Create a pandas dataframe with a date column: import pandas as pd import datetime TODAY = datetime.date.today() ONE_WEEK = datetime.timedelta(days=7) ONE_DAY = datetime.timedelta(days=1) df = pd.DataFrame({‘dt’: [TODAY-ONE_WEEK, TODAY-3*ONE_DAY, TODAY], ‘x’: [42, 45,127]}) The dates have gaps: dt x 0 2018-11-19 42 1 2018-11-23 45 2 2018-11-26 127 Now, fill in the missing dates: r = […]
How to fill missing dates in Pandas Read More »
Cosine similarity is the normalised dot product between two vectors. I guess it is called “cosine” similarity because the dot product is the product of Euclidean magnitudes of the two vectors and the cosine of the angle between them. If you want, read more about cosine similarity and dot products on Wikipedia. Here is how
Cosine similarity in Python Read More »
Now that I work in shipping, it is necessary to learn a bunch of new terms. Shipping is regulated under Admiralty Law and there are traditional documents and parties involved. Knowing what these are is crucial to understanding shipping. Legal documents There are three key documents involved with shipping: Bill of Lading (sometimes abbreviated as
Terms used in shipping Read More »
Here is how to sample from a softmax probability vector at different temperatures. import numpy as np import matplotlib.pyplot as plt import matplotlib as mpl import seaborn as sns mpl.rcParams[‘figure.dpi’]= 144 trials = 1000 softmax = [0.1, 0.3, 0.6] def sample(softmax, temperature): EPSILON = 10e-16 # to avoid taking the log of zero #print(preds) (np.array(softmax)
How to sample from softmax with temperature Read More »
https://seaborn.pydata.org/generated/seaborn.heatmap.html
How to explore two-dimensional data with a heatmap Read More »
Here is the code: %matplotlib inline import geopandas as gpd import matplotlib as mpl # make rcParams available (optional) mpl.rcParams[‘figure.dpi’]= 144 # increase dpi (optional) world = gpd.read_file(gpd.datasets.get_path(“naturalearth_lowres”)) world = world[world.name != ‘Antarctica’] # remove Antarctica (optional) world[‘gdp_per_person’] = world.gdp_md_est / world.pop_est g = world.plot(column=’gdp_per_person’, cmap=’OrRd’, scheme=’quantiles’) g.set_facecolor(‘#A8C5DD’) # make the ocean blue (optional) Here
How to display a Choropleth map in Jupyter Notebook Read More »
Step one: brew install glpk pip install pulp Step two: from pulp import * prob = LpProblem(“test1”, LpMinimize) # Variables x = LpVariable(“x”, 0, 4, cat=”Integer”) y = LpVariable(“y”, -1, 1, cat=”Integer”) z = LpVariable(“z”, 0, cat=”Integer”) # Objective prob += x + 4*y + 9*z # Constraints prob += x+y = 10 prob +=
(Integer) Linear Programming in Python Read More »
I was looking at arial photos of north-western Europe in Google Maps when I noticed a big white dot on the map! I thought, what the hell? To satisfy my curiosity I decided to zoom in for further investigation. It turns out that the big white dot is a giant surface mine. The 48 km²
Things that are visible from space, the Garzweiler Surface Mine Read More »
Datasets: – Urban morphological zones 2000 (EU): https://www.eea.europa.eu/data-and-maps/data/urban-morphological-zones-2000-2 – Population count (World): http://sedac.ciesin.columbia.edu/data/set/gpw-v4-population-count-rev10/ – Administrative regions (World): http://gadm.org/ The map is European since the “urban” data from the European Environmental Agency (EEA) only covers Europe. Caveats The UMZ data ended up in PostGIS with srid 900914. You can use prj2epsg.org to convert the contents of
Create a European city map with population density Read More »
The GADM database contains geographical data for administrative regions, e.g. countries, regions and municipalities. As always, once you have the data in the right format, it is easy to import it into a database. The data is available from GADM in several formats. All data has the coordinate reference system in longitude/latitude and theWGS84 datum.
How to create a world-wide PostgreSQL database of administrative regions Read More »