Pyro’s introduction to probabilistic programming
https://pyro.ai/examples/intro_long.html
How to Draw an Owl
Taken from lecture 1 of Statistical Rethinking course (around the 44 minute mark). The course material is also on Github. How to draw an "owl" version 1: Create generative simulation (GS) Write an estimator Validate estimator using simulated data Analyze real data: … Reuse 1 to compute hypothetical interventions How to draw an "owl" version …
How to sort numbers with an evolutionary algorithm (CMA-ES)
Yes, this is clearly nonsense. Sorting is not a hard problem and standard algorithms such as quicksort and mergesort have O(x^2) and O(n log(n)) complexity. But let me scratch this itch of sorting numbers using an evolutionary algorithm, specifically Covariance matrix adaptation evolution strategy (CMA-ES). Technically, we will use what I think is the original …
How to sort numbers with an evolutionary algorithm (CMA-ES) Read More »
How to draw lines on map in Databricks
Imports: import plotly.graph_objects as go Plot: fig = go.Figure() fig.add_trace(go.Scattermapbox( mode = "markers+lines", lon = [10, 20, 30], lat = [10, 15,30], marker = {'size': 10})) fig.add_trace(go.Scattermapbox( mode = "markers+lines", lon = [-50, -60,40], lat = [30, 10, -20], marker = {'size': 10})) fig.update_layout( margin ={'l':0,'t':0,'b':0,'r':0}, mapbox = { 'center': {'lon': 10, 'lat': 10}, 'style': …
How to call an API from PySpark (in workers)
Tested in Databricks import pyspark.sql.functions as F import requests # create dataframe pokenumbers = [(i,) for i in range(100)] cols = ["pokenum"] df_pokenums = spark.createDataFrame(data=pokenumbers, schema=cols) # call API def get_name(rows): # take the first item in list (API doesn't support batch) first = rows[0] url = f'https://pokeapi.co/api/v2/pokemon-form/{first.pokenum}' try: resp = requests.get(url) name = resp.json()['pokemon']['name'] …
How to use bnlearn to learn causal structures
This article on causal machine learning covers a practical example of how to learn structural causal models (SCM) directly from data. We will use bnlearn, which is an open-source library for learning the graphical structure of Bayesian networks in Python. Check out my Github repo for additional code examples. For other frameworks, checkout my page …
Learn some machine learning fundamentals in an afternoon
Here is a plan to learn ML fundamentals in an afternoon by watching some videos on youtube: Follow this plan Machine learning fundamentals: What is Bias and Variance? – 6 minutes What is a Tensor? – 12 minutes [Stop and drink coffee, eat a snack] How to address bias and variance: Regularization (e.g. L2/Ridge) – …
Learn some machine learning fundamentals in an afternoon Read More »
How to make interactive plots in Jupyter
Python has great options for plotting data. However, sometimes you want to explore data by changing parameters and rerunning plots to explore the effect of those changed parameters. This slows down the cycle of exploration. Luckily, Jupyter offers you a way to make you plots interactive, so you can see the effect of parameter changes …
Yummy 3D plots
Very nice interactive 3D plots with Plotly. import plotly.graph_objects as go import numpy as np import pandas as pd # Read data from a csv Z = pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/api_docs/mt_bruno_elevation.csv').values # Actually not necessary to provide X and Y… X = np.linspace(0, 1000, Z.shape[0]) Y = np.linspace(0, 1000, Z.shape[1]) fig = go.Figure(data=[go.Surface(x=X, y=Y, z=Z)]) fig.update_layout(title='Mt Bruno Elevation', …
RM papers
Optimal Dynamic Pricing of Inventories over Finite Horizons