Programming

How to fill missing dates in Pandas

Create a pandas dataframe with a date column: import pandas as pd import datetime   TODAY = datetime.date.today() ONE_WEEK = datetime.timedelta(days=7) ONE_DAY = datetime.timedelta(days=1)   df = pd.DataFrame({’dt’: [TODAY-ONE_WEEK, TODAY-3*ONE_DAY, TODAY], ‘x’: [42, 45,127]})import pandas as pd import datetime TODAY = datetime.date.today() ONE_WEEK = datetime.timedelta(days=7) ONE_DAY = datetime.timedelta(days=1) df = pd.DataFrame({‘dt’: [TODAY-ONE_WEEK, TODAY-3*ONE_DAY, TODAY], ‘x’: …

How to fill missing dates in Pandas Read More »

How to display a Choropleth map in Jupyter Notebook

Here is the code: %matplotlib inline import geopandas as gpd import matplotlib as mpl # make rcParams available (optional) mpl.rcParams[’figure.dpi’]= 144 # increase dpi (optional)   world = gpd.read_file(gpd.datasets.get_path("naturalearth_lowres")) world = world[world.name != ‘Antarctica’] # remove Antarctica (optional) world[’gdp_per_person’] = world.gdp_md_est / world.pop_est g = world.plot(column=’gdp_per_person’, cmap=’OrRd’, scheme=’quantiles’) g.set_facecolor(’#A8C5DD’) # make the ocean blue (optional)%matplotlib …

How to display a Choropleth map in Jupyter Notebook Read More »

Python script for geocoding a text file

Assume that you have a file with some locations as text with one location per line. For example, here are some school names in Copenhagen, Denmark, stored in schools.csv: Hyltebjerg Skole Heibergskolen Ellebjerg Skole Katrinedals Skole Peder Lykke Skolen Amager Fælled Skole Tingbjerg Heldagsskole Øster Farimagsgades Skole Sankt Annæ Gymnasiums Grundskole Lykkebo Skole Randersgades Skole …

Python script for geocoding a text file Read More »

Easy parallel HTTP requests with Python and asyncio

Python 3.x, and in particular Python 3.5, natively supports asynchronous programming. While asynchronous code can be harder to read than synchronous code, there are many use cases were the added complexity is worthwhile. One such examples is to execute a batch of HTTP requests in parallel, which I will explore in this post. Additionally, the …

Easy parallel HTTP requests with Python and asyncio Read More »

How to compute the pagerank of almost anything

Whenever two things have a directional relationship to each other, then you can compute the pagerank of those things. For example, you can observe a directional relationships between web pages that link to each other, scientists that cite each other, and chess players that beat each other. The relationship is directional because it matters in …

How to compute the pagerank of almost anything Read More »

How to export CSV file to database with Python

Pandas and SQLAlchemy offer powerful conversions between CSV files and tables in databases. Here is a small example: import pandas as pd from sqlalchemy import create_engine   df = pd.read_csv(’mydata.csv’)   engine = create_engine(’sqlite:///mydata.db’) df.to_sql(’mytable’, engine)import pandas as pd from sqlalchemy import create_engine df = pd.read_csv(‘mydata.csv’) engine = create_engine(‘sqlite:///mydata.db’) df.to_sql(‘mytable’, engine) Read more: pandas.DataFrame.to_sql sqlalchemy …

How to export CSV file to database with Python Read More »

How to use non-default profile in boto3

Given an AWS credentials file that looks like this: [default] aws_access_key_id = DEFAULT aws_secret_access_key = SECRET1   [dev] aws_access_key_id = DEV aws_secret_access_key = SECRET2   [prod] aws_access_key_id = PROD aws_secret_access_key = SECRET3[default] aws_access_key_id = DEFAULT aws_secret_access_key = SECRET1 [dev] aws_access_key_id = DEV aws_secret_access_key = SECRET2 [prod] aws_access_key_id = PROD aws_secret_access_key = SECRET3 You can use …

How to use non-default profile in boto3 Read More »