How to fill missing dates in Pandas

Create a pandas dataframe with a date column: import pandas as pd import datetime TODAY = datetime.date.today() ONE_WEEK = datetime.timedelta(days=7) ONE_DAY = datetime.timedelta(days=1) df = pd.DataFrame({‘dt’: [TODAY-ONE_WEEK, TODAY-3*ONE_DAY, TODAY], ‘x’: [42, 45,127]}) The dates have gaps: dt x 0 2018-11-19 42 1 2018-11-23 45 2 2018-11-26 127 Now, fill in the missing dates: r = […]

How to fill missing dates in Pandas Read More »

Cosine similarity in Python

Cosine similarity is the normalised dot product between two vectors. I guess it is called “cosine” similarity because the dot product is the product of Euclidean magnitudes of the two vectors and the cosine of the angle between them. If you want, read more about cosine similarity and dot products on Wikipedia. Here is how

Cosine similarity in Python Read More »

Terms used in shipping

Now that I work in shipping, it is necessary to learn a bunch of new terms. Shipping is regulated under Admiralty Law and there are traditional documents and parties involved. Knowing what these are is crucial to understanding shipping. Legal documents There are three key documents involved with shipping: Bill of Lading (sometimes abbreviated as

Terms used in shipping Read More »

How to display a Choropleth map in Jupyter Notebook

Here is the code: %matplotlib inline import geopandas as gpd import matplotlib as mpl # make rcParams available (optional) mpl.rcParams[‘figure.dpi’]= 144 # increase dpi (optional) world = gpd.read_file(gpd.datasets.get_path(“naturalearth_lowres”)) world = world[world.name != ‘Antarctica’] # remove Antarctica (optional) world[‘gdp_per_person’] = world.gdp_md_est / world.pop_est g = world.plot(column=’gdp_per_person’, cmap=’OrRd’, scheme=’quantiles’) g.set_facecolor(‘#A8C5DD’) # make the ocean blue (optional) Here

How to display a Choropleth map in Jupyter Notebook Read More »

Create a European city map with population density

Datasets: – Urban morphological zones 2000 (EU): https://www.eea.europa.eu/data-and-maps/data/urban-morphological-zones-2000-2 – Population count (World): http://sedac.ciesin.columbia.edu/data/set/gpw-v4-population-count-rev10/ – Administrative regions (World): http://gadm.org/ The map is European since the “urban” data from the European Environmental Agency (EEA) only covers Europe. Caveats The UMZ data ended up in PostGIS with srid 900914. You can use prj2epsg.org to convert the contents of

Create a European city map with population density Read More »

How to create a world-wide PostgreSQL database of administrative regions

The GADM database contains geographical data for administrative regions, e.g. countries, regions and municipalities. As always, once you have the data in the right format, it is easy to import it into a database. The data is available from GADM in several formats. All data has the coordinate reference system in longitude/latitude and theWGS84 datum.

How to create a world-wide PostgreSQL database of administrative regions Read More »