Difficulty level 4 of 5: Advanced
Having reinstalled PostgreSQL with support for Python and pointing at my non-system python, it is time to test whether I can use the convex optimizer library I’ve installed in my Python 2.7 (pip install cvxopt). Install PL/Python if not already…
Prepare the database by installing PL/Python and a function: CREATE EXTENSION IF NOT EXISTS plpythonu; CREATE OR REPLACE FUNCTION hello_lp() RETURNS float[] AS $$ import subprocess output = subprocess.Popen ( [ “”, “” ], stdout = subprocess.PIPE, stderr = subprocess.STDOUT…
Here is a project I’m working on called the Web Co-Processor (credit for name goes to Marcos Vaz Salles). There is a demo you can try if you clone the project. Read the following four bullets and you’ll understand what…
This video was mentioned on highscalability.com, so I thought I’d have a look. Knowning this stuff is useful when you’re in the business of delivering large amounts of geographical data to a large amount of clients. C10M = 10 million…
Non-strict orders: ≤ The symbol ≤ denotes a generalization of “less than or equal”, and it defines either a partial or total ordering over a set P (in the table below a,b ∈ P): Constraint (Non-strict) partial order (Non-strict) total…
Fast compression algorithms like Snappy, QuickLZ and LZ4 are designed for a general stream of bytes, and typically don’t treat byte-sequences representing numbers in any special way. Geospatial data is special in the sense that it often contains a large…
Question: When can a distance function d(x,y) be called metric, pseudo-metric, quasi-metric or semi-metric? Constraint Metric Pseudo Quasi Semi Non-negativity: d(x,y) ≥ 0 x x x x Identity of indiscernibles: d(x,y)=0 ⇒ x=y x x x Symmetry: d(x,y) = d(y,x)…
A friend of mine, who is the CEO of a company that develops an embedded database, asked me to do a presentation on spatial indexing. This was an opportunity for me to brush up on R-trees and similar datastructures. Download…
A distribution algorithm is used to map keys to servers in a distributed key-value store. There are several different ones, implemented in different systems, and with different properties. In this blog post I’ll briefly cover the best-known key hashing schemes,…
Warning: This is work in progress, so this post will be updated TODO: Make a table of the file systems mentioned e.g. in CRUSH paper (related work section). Design space A subset of properties that can considered when designing a…