pandas 丢掉 dataframe 中重复的行-ag捕鱼王app官网

本教程介绍了如何使用 dataframe.drop_duplicates() 方法从 pandas dataframe 中删除所有重复的行。

dataframe.drop_duplicates() 语法

dataframe.drop_duplicates(subset=none, keep="first", inplace=false, ignore_index=false)

它返回一个 dataframe，删除 dataframe 中所有重复的行。

使用 dataframe.drop_duplicates() 方法删除重复的行

import pandas as pd
df_with_duplicates = pd.dataframe(
    {
        "id": [302, 504, 708, 103, 303, 302],
        "name": ["watch", "camera", "phone", "shoes", "watch", "watch"],
        "cost": ["300", "400", "350", "100", "300", "300"],
    }
)
df_without_duplicates = df_with_duplicates.drop_duplicates()
print("dataframe with duplicates:")
print(df_with_duplicates, "\n")
print("dataframe without duplicates:")
print(df_without_duplicates, "\n")

输出：

dataframe with duplicates:
    id    name cost
0  302   watch  300
1  504  camera  400
2  708   phone  350
3  103   shoes  100
4  303   watch  300
5  302   watch  300 
dataframe without duplicates:
    id    name cost
0  302   watch  300
1  504  camera  400
2  708   phone  350
3  103   shoes  100
4  303   watch  300 

它会删除所有列的所有值都相同的行。默认情况下，dataframe 中每一列都有相同值的行才被认为是重复的。在 df_with_duplicates dataframe 中，第一行和第五行对所有列都有相同的值，所以第五行被删除。

设置 subset 参数以仅基于特定列删除重复项

import pandas as pd
df_with_duplicates = pd.dataframe(
    {
        "id": [302, 504, 708, 103, 303, 302],
        "name": ["watch", "camera", "phone", "shoes", "watch", "watch"],
        "cost": ["300", "400", "350", "100", "300", "300"],
    }
)
df_without_duplicates = df_with_duplicates.drop_duplicates(subset=["name"])
print("dataframe with duplicates:")
print(df_with_duplicates, "\n")
print("dataframe without duplicates:")
print(df_without_duplicates, "\n")

输出：

dataframe with duplicates:
    id    name cost
0  302   watch  300
1  504  camera  400
2  708   phone  350
3  103   shoes  100
4  303   watch  300
5  302   watch  300 
dataframe without duplicates:
    id    name cost
0  302   watch  300
1  504  camera  400
2  708   phone  350
3  103   shoes  100 

在这里，我们将 name 作为 subset 参数传给 drop_duplicates() 方法。第四行和第五行被删除，因为它们的 name 列的值与第一列相同。

在 drop_duplicates() 方法中设置 keep='last'

import pandas as pd
df_with_duplicates = pd.dataframe(
    {
        "id": [302, 504, 708, 103, 303, 302],
        "name": ["watch", "camera", "phone", "shoes", "watch", "watch"],
        "cost": ["300", "400", "350", "100", "300", "300"],
    }
)
df_without_duplicates = df_with_duplicates.drop_duplicates(subset=["name"], keep="last")
print("dataframe with duplicates:")
print(df_with_duplicates, "\n")
print("dataframe without duplicates:")
print(df_without_duplicates, "\n")

输出：

dataframe with duplicates:
    id    name cost
0  302   watch  300
1  504  camera  400
2  708   phone  350
3  103   shoes  100
4  303   watch  300
5  302   watch  300 
dataframe without duplicates:
    id    name cost
1  504  camera  400
2  708   phone  350
3  103   shoes  100
5  302   watch  300 

它删除了所有的行，除了最后一行与 name 列值相同的行。

我们设置 keep=false 来删除任何一列中具有相同值的所有行。

import pandas as pd
df_with_duplicates = pd.dataframe(
    {
        "id": [302, 504, 708, 103, 303, 302],
        "name": ["watch", "camera", "phone", "shoes", "watch", "watch"],
        "cost": ["300", "400", "350", "100", "300", "300"],
    }
)
df_without_duplicates = df_with_duplicates.drop_duplicates(subset=["name"], keep=false)
print("dataframe with duplicates:")
print(df_with_duplicates, "\n")
print("dataframe without duplicates:")
print(df_without_duplicates, "\n")

输出：

dataframe with duplicates:
    id    name cost
0  302   watch  300
1  504  camera  400
2  708   phone  350
3  103   shoes  100
4  303   watch  300
5  302   watch  300 
dataframe without duplicates:
    id    name cost
1  504  camera  400
2  708   phone  350
3  103   shoes  100 

它删除了第一、五、六行，因为它们的 name 列都有相同的值。