kostas

kostas

How to draw lines on map in Databricks

Imports: import plotly.graph_objects as go Plot: fig = go.Figure() fig.add_trace(go.Scattermapbox( mode = "markers+lines", lon = [10, 20, 30], lat = [10, 15,30], marker = {'size': 10})) fig.add_trace(go.Scattermapbox( mode = "markers+lines", lon = [-50, -60,40], lat = [30, 10, -20], marker…

How to call an API from PySpark (in workers)

Tested in Databricks import pyspark.sql.functions as F import requests # create dataframe pokenumbers = [(i,) for i in range(100)] cols = ["pokenum"] df_pokenums = spark.createDataFrame(data=pokenumbers, schema=cols) # call API def get_name(rows): # take the first item in list (API doesn't…

How to make interactive plots in Jupyter

Python has great options for plotting data. However, sometimes you want to explore data by changing parameters and rerunning plots to explore the effect of those changed parameters. This slows down the cycle of exploration. Luckily, Jupyter offers you a…

Yummy 3D plots

Very nice interactive 3D plots with Plotly. import plotly.graph_objects as go import numpy as np import pandas as pd # Read data from a csv Z = pd.read_csv('https://raw.githubusercontent.com/plotly/datasets/master/api_docs/mt_bruno_elevation.csv').values # Actually not necessary to provide X and Y… X = np.linspace(0,…

Getting back into operations research

For the last five years, I have been fascinated with machine learning techniques, but that fascination is slowly running out. I increasingly consider ML a tool in my tool box among others, not a panacea to all problems. In particular,…