Difficulty level 3 of 5: Intermediate
Here is an analytical query that you (and I) will often need to do if you work in e-commerce, marketing or similar domain. It answers the question, within each group of items (e.g. partitioned by territory, age groups or something…
Wikipedia contains a wealth of knowledge. While some of that knowledge consists of natural language descriptions, a rich share of information on Wikipedia is encoded in machine-readable format, such as infoboxes and other specially formatted parts. An infobox is rendered…
I like to code in Python. I also like the concept of asynchronous workers to build loosely coupled applications. Luckily, RabbitMQ and Celery can help me do exactly that. This post is based on a very nice YouTube video by…
Scrapy is an excellent Python library for web scraping. For example, you could create an API with data that is populated via web scraping. This article covers some basic scrapy features, such as the shell and selectors. Install scrapy in…
After pip install bybrain, the PyBrain the quick start essentially goes as follows: from pybrain.tools.shortcuts import buildNetwork from pybrain.structure import TanhLayer from pybrain.datasets import SupervisedDataSet from pybrain.supervised.trainers import BackpropTrainer # Create a neural network with two inputs, three hidden, and…
Fing logger (finglogger.sh): #!/bin/sh FING_LOG_FILE= # append current public ip echo `date +”%Y/%m/%d %T”`”;publicip;”`curl -s ipecho.net/plain`”;;;” >> $FING_LOG_FILE # append current fing output /usr/bin/fing -r1 -o log,csv,$FING_LOG_FILE,1000 –silent Add to cron (run every hour): 0 * * * *
Inspired and simplified from a set of slides on using RDBMS for storing, managing, and querying graphs: < pre lang=”sql”> with recursive fib(i,j) as ( select 0,1 union all select j, i+j from fib where j
Docker allows you to develop, ship and run any application, anywhere. The metaphor is that of the standard shipping container that fits on any ship, can be handled by any crane, and loaded onto any train or truck. In a…
PhantomJS is a headless browsers that you can use, e.g. to test web UIs and to screen capture webpages. I will focus on the last use case. Since PhantomJS knows how to execute Javascript, it can create a screen shot…
This is a mini-workshop that shows you how to work with Spark on Amazon Elastic Map-Reduce; It’s a kind of hello world of Spark on EMR. We will solve a simple problem, namely use Spark and Amazon EMR to count…