Pandas and SQLAlchemy offer powerful conversions between CSV files and tables in databases. Here is a small example: import pandas as pd from sqlalchemy import create_engine df = pd.read_csv(‘mydata.csv’) engine = create_engine(”) df.to_sql(‘mytable’, engine) Read more: pandas.DataFrame.to_sql sqlalchemy engines
Given an AWS credentials file that looks like this: [default] aws_access_key_id = DEFAULT aws_secret_access_key = SECRET1 [dev] aws_access_key_id = DEV aws_secret_access_key = SECRET2 [prod] aws_access_key_id = PROD aws_secret_access_key = SECRET3 You can use any profile, say dev, like this in…
You may ask yourself, if I’m a machine learning person then what kind am I? See for yourself in Jason Eisner’s Three Cultures of Machine Learning.
While Redshift does not offer native support for spatial data, indexes and functions, there exists a partial workaround. Redshift supports Python UDFs and can also load custom Python libraries. Out of the box, Redshift has numpy, scipy, pandas and many…
You can use the shell to extract a random sample of lines from a file in *nix. The two commands you need are “shuf” and “head” (+ “tail” for CSV files with a header). The shuf command will randomly shuffle…
At Spark Summit Europe 2015, several presenters made use of Apache Zeppeling, which is a notebook (a la IPython) for Spark. I immediately wanted to try it out myself. I also highly recommend you to download and try it out…
After pip install bybrain, the PyBrain the quick start essentially goes as follows: from pybrain.tools.shortcuts import buildNetwork from pybrain.structure import TanhLayer from pybrain.datasets import SupervisedDataSet from pybrain.supervised.trainers import BackpropTrainer # Create a neural network with two inputs, three hidden, and…
There is no excerpt because this is a protected post.
Fing logger (finglogger.sh): #!/bin/sh FING_LOG_FILE= # append current public ip echo `date +”%Y/%m/%d %T”`”;publicip;”`curl -s ipecho.net/plain`”;;;” >> $FING_LOG_FILE # append current fing output /usr/bin/fing -r1 -o log,csv,$FING_LOG_FILE,1000 –silent Add to cron (run every hour): 0 * * * *
The problem: Given two random samples, s1 and s2, of size k over two disjoint populations, p1 and p2, how to combine the two k-sized random samples into one k-sized random sample over p1 ∪ p2? The solution: k times,…