Fast compression algorithms like Snappy, QuickLZ and LZ4 are designed for a general stream of bytes, and typically don’t treat byte-sequences representing numbers in any special way. Geospatial data is special in the sense that it often contains a large…
Question: When can a distance function d(x,y) be called metric, pseudo-metric, quasi-metric or semi-metric? Constraint Metric Pseudo Quasi Semi Non-negativity: d(x,y) ≥ 0 x x x x Identity of indiscernibles: d(x,y)=0 ⇒ x=y x x x Symmetry: d(x,y) = d(y,x)…
Rtree is a ctypes Python wrapper of libspatialindex that provides a number of advanced spatial indexing features for the spatially curious Python user.
A friend of mine, who is the CEO of a company that develops an embedded database, asked me to do a presentation on spatial indexing. This was an opportunity for me to brush up on R-trees and similar datastructures. Download…
Spacebase is a spatial datastore that began life as military-grade software, which at least sounds kind of cool. It’s an in-memory database, really, so switch off the cluster and the data is gone. Apparently the same thing was (unknown to…
A distribution algorithm is used to map keys to servers in a distributed key-value store. There are several different ones, implemented in different systems, and with different properties. In this blog post I’ll briefly cover the best-known key hashing schemes,…
Install KC: wget tar xzvf kyotocabinet-1.2.76.tar.gz cd kyotocabinet-1.2.76 ./configure && make && make install # takes a couple of minutes
Note: It is worth noting that (the way I interpret it) one of the authors of LevelDB (Sanjay) recommends using TokyoCabinet if the workload is more read-oriented. In turn the TokyoCabinet authors recommend using KyotoCabinet :-) (see video with a…
Background When I’m at the library, I’d like to be able to go to the toilet, without collecting all my stuff from the table. Part of the solution is to have a camera installed that films all the tables, but…
BigQuery is an API developed by Google for querying big data using an SQL like language. For beginners I can recommend this video presentation on BigQuery. It also covers the open source version (Apache Drill) of the software (Dremel) underlying…