Pandas Tricks

Basics

Imports:

import pandas as pd
import numpy as np

Advanced

Shuffle rows:

df.reindex(np.random.permutation(df.index))

Make index incremental again (e.g. after a shuffle):

df.reset_index(drop=True)

Drop rows that contain NaN:

df.dropna()

Split dataframe in half:

df1,df2 = np.array_split(df, 2)

Recipes

Merging

Basic merge on shared columns (inner, outer, left, right):

df1 = pd.DataFrame({'A': [1,2,3], 'B': [1,2,3]})
"""
   A  B
0  1  1
1  2  2
2  3  3
"""
 
df2 = pd.DataFrame({'A': [3,4,5], 'C': [1,2,3]})
"""
   A  C
0  3  1
1  4  2
2  5  3
"""
 
df1.merge(df2)  # inner (default)
"""
   A  B  C
0  3  3  1
"""
 
df1.merge(df2, how='outer')
"""
   A   B   C
0  1   1 NaN
1  2   2 NaN
2  3   3   1
3  4 NaN   2
4  5 NaN   3
"""
 
df1.merge(df2, how='left')
"""
   A  B   C
0  1  1 NaN
1  2  2 NaN
2  3  3   1
"""
 
df1.merge(df2, how='right')
"""
   A   B  C
0  3   3  1
1  4 NaN  2
2  5 NaN  3
"""

Leave a Reply