Quick benchmarking of Python code with timeit

How long does [insert code snippet] take in Python? Find out with the timeit module, in Python since 2.3.

Below I time how long it takes to make a list with a 1000 zeros, [0,0,0,…], using different Python code snippets. Notice that you can time snippets directly on the commandline, without writing a script file (.py) or running the Python interpreter interactively.

Benchmarks ordered by running time

Using numpy:

$ python -m timeit 'import numpy' 'a = numpy.zeros(shape=(1,1000))'
100000 loops, best of 3: 2.38 usec per loop

Read more

Never forget your password again, version 1

The recommended practice is to have different passwords on different websites. But how do you remember all those passwords without storing them somewhere? The tricks is, you don’t. You remember a single strong password, and use a mechanism to generate other passwords from that.

This is not for securing government secrets, but should work for your twitter account.

Create a single very strong password

There are many ways to do this: http://xkcd.com/936/

Read more

Benchmark: Reading uncompressed and compressed files from disc

In this post I’ll compare the running time of reading uncompressed and compressed files from disc.

I’ll run a test using two files, data.txt (858M) and data.txt.gz (83M), that have the same content.

About cat and zcat

The well-known command cat, prints the contents of a file. The lesser-known zcat, prints the contents of a GZIP’ed file.

Read more

GDAL and OGR Python tips

Online documentation for GDAL/OGR Python is sparse. Here I show some recommended ways of learning more about GDAL/OGR in Python.

Using Python interpreter

You can learn about GDAL and OGR from inside the Python interpreter.

Start python interpreter:

$ python

Import the modules:

from osgeo import gdal,ogr,osr

Learn about the modules using ‘help’ and ‘dir’ in Python. These built-in functions work for any type of object (module, class, functions etc):

help(ogr) # display help for the ogr module
dir(ogr) # what's contained in ogr module?
help(ogr.Geometry) # display help for ogr.Geometry class
dir(ogr.Geometry) # show contents of an object, like functions on a class

Using gis.stackexchange.com

gis.stackexchange.com is a Q&A site about GIS. Many people here know about GDAL and ORG, also about the Python bindings.

Some good questions I’ve found:

Transformation using OGR in Python

Will add more as I find them…

Importing data from a CSV file into a Postgres table

Simple CSV file import

You have a CSV file called “data.csv”. It has a header line, and is delimited using “;”. You want to import it into Postgres and a table called “your_table”:

Create the database table. Set column-types so the string fields in the CSV file, can be cast to values in columns.

CREATE TABLE your_table
  -- Your columns

Read more