Programming – SKIPPERKONGEN

How to fill missing dates in Pandas

Leave a Comment / Analytics, Programming / By kostas

Create a pandas dataframe with a date column: import pandas as pd import datetime TODAY = datetime.date.today() ONE_WEEK = datetime.timedelta(days=7) ONE_DAY = datetime.timedelta(days=1) df = pd.DataFrame({’dt’: [TODAY-ONE_WEEK, TODAY-3*ONE_DAY, TODAY], ‘x’: [42, 45,127]})import pandas as pd import datetime TODAY = datetime.date.today() ONE_WEEK = datetime.timedelta(days=7) ONE_DAY = datetime.timedelta(days=1) df = pd.DataFrame({‘dt’: [TODAY-ONE_WEEK, TODAY-3*ONE_DAY, TODAY], ‘x’: …

How to fill missing dates in Pandas Read More »

Cosine similarity in Python

Leave a Comment / Algorithms, Analytics, Machine Learning, Programming / By kostas

Cosine similarity is the normalised dot product between two vectors. I guess it is called “cosine” similarity because the dot product is the product of Euclidean magnitudes of the two vectors and the cosine of the angle between them. If you want, read more about cosine similarity and dot products on Wikipedia. Here is how …

Cosine similarity in Python Read More »

How to display a Choropleth map in Jupyter Notebook

Leave a Comment / Analytics, Programming, Spatial stuff / By kostas

Here is the code: %matplotlib inline import geopandas as gpd import matplotlib as mpl # make rcParams available (optional) mpl.rcParams[’figure.dpi’]= 144 # increase dpi (optional) world = gpd.read_file(gpd.datasets.get_path("naturalearth_lowres")) world = world[world.name != ‘Antarctica’] # remove Antarctica (optional) world[’gdp_per_person’] = world.gdp_md_est / world.pop_est g = world.plot(column=’gdp_per_person’, cmap=’OrRd’, scheme=’quantiles’) g.set_facecolor(’#A8C5DD’) # make the ocean blue (optional)%matplotlib …

How to display a Choropleth map in Jupyter Notebook Read More »

(Integer) Linear Programming in Python

Leave a Comment / Algorithms, Analytics, Optimization, Programming / By kostas

Step one: brew install glpk pip install pulpbrew install glpk pip install pulp Step two: from pulp import * prob = LpProblem("test1", LpMinimize) # Variables x = LpVariable("x", 0, 4, cat="Integer") y = LpVariable("y", -1, 1, cat="Integer") z = LpVariable("z", 0, cat="Integer") # Objective prob += x + 4*y + 9*z …

(Integer) Linear Programming in Python Read More »

Python script for geocoding a text file

Leave a Comment / Programming, Spatial stuff / By kostas

Assume that you have a file with some locations as text with one location per line. For example, here are some school names in Copenhagen, Denmark, stored in schools.csv: Hyltebjerg Skole Heibergskolen Ellebjerg Skole Katrinedals Skole Peder Lykke Skolen Amager Fælled Skole Tingbjerg Heldagsskole Øster Farimagsgades Skole Sankt Annæ Gymnasiums Grundskole Lykkebo Skole Randersgades Skole …

Python script for geocoding a text file Read More »

Easy parallel HTTP requests with Python and asyncio

2 Comments / Programming / By kostas

Python 3.x, and in particular Python 3.5, natively supports asynchronous programming. While asynchronous code can be harder to read than synchronous code, there are many use cases were the added complexity is worthwhile. One such examples is to execute a batch of HTTP requests in parallel, which I will explore in this post. Additionally, the …

Easy parallel HTTP requests with Python and asyncio Read More »

How to compute the pagerank of almost anything

Leave a Comment / Algorithms, Programming / By kostas

Whenever two things have a directional relationship to each other, then you can compute the pagerank of those things. For example, you can observe a directional relationships between web pages that link to each other, scientists that cite each other, and chess players that beat each other. The relationship is directional because it matters in …

How to compute the pagerank of almost anything Read More »

Quick introduction to RabbitMQ and Celery

Leave a Comment / Programming / By kostas

I like to code in Python. I also like the concept of asynchronous workers to build loosely coupled applications. Luckily, RabbitMQ and Celery can help me do exactly that. This post is based on a very nice YouTube video by Max Mautner (the one below). For easy repeatability, I have transcribed the video in this …

Quick introduction to RabbitMQ and Celery Read More »

How to export CSV file to database with Python

Leave a Comment / Databases, Programming / By kostas

Pandas and SQLAlchemy offer powerful conversions between CSV files and tables in databases. Here is a small example: import pandas as pd from sqlalchemy import create_engine df = pd.read_csv(’mydata.csv’) engine = create_engine(’sqlite:///mydata.db’) df.to_sql(’mytable’, engine)import pandas as pd from sqlalchemy import create_engine df = pd.read_csv(‘mydata.csv’) engine = create_engine(‘sqlite:///mydata.db’) df.to_sql(‘mytable’, engine) Read more: pandas.DataFrame.to_sql sqlalchemy …

How to export CSV file to database with Python Read More »

How to use non-default profile in boto3

Leave a Comment / Cloud, Programming / By kostas

Given an AWS credentials file that looks like this: [default] aws_access_key_id = DEFAULT aws_secret_access_key = SECRET1 [dev] aws_access_key_id = DEV aws_secret_access_key = SECRET2 [prod] aws_access_key_id = PROD aws_secret_access_key = SECRET3[default] aws_access_key_id = DEFAULT aws_secret_access_key = SECRET1 [dev] aws_access_key_id = DEV aws_secret_access_key = SECRET2 [prod] aws_access_key_id = PROD aws_secret_access_key = SECRET3 You can use …

How to use non-default profile in boto3 Read More »