Image search by sketching – continued

It’s a simple question

Can you search for images by sketching a similar image?

I went looking online for a search engine that had implemented this feature, which I’ll call image-search-by-sketching.

Update: Since I wrote this piece, GaZoPa no longer exists. In the meantime Google has implemented image-search-by-image. You can’t sketch, but you can use an existing image.

Googles implemetation of image-search-by-image is did both a good and bad job when I tried it last (December 2011). When I tried with my test image (dog-shape below), I got this blog post, which is good. But the related images are way off, number one related image is a picture of a shoe?

I can see the similarity to my dog-shape in the results that Google suggested, but I didn’t get a dog. No doubt it is a hard problem, and what I wish for is highly semantic, in the sense that I want the search engine to recognize that I’m looking for a dog. In my test below, GaZoPa could have gotten it right for a number of reasons. Maybe they simply had many fewer items in their database to match the dog against, and the best match happened to be… a dog? I guess I’ll never know. R.I.P. GaZoPa.

And so I went looking for such a search engine…

First thing I did, was ask this question on Stackoverflow and got an reply which pointed my to a couple of cool websites.

These are all cool websites, but at first not exactly what I was looking for. After trying GazoPa I realized that the website is almost exactly what I was looking for (a service that allows you to sketch-up an image query).

Trying GazoPa

GazoPa allows you (among other things) to upload an image, and performs a search for similar images. I’m not quite sure which images are in its index, but I proceeded with the following experiment. I drew up a rather crude dog in Dia, and uploaded this image to GazoPa. Here is the dog:

It actually gave some pretty decent results, with this one being the first hit:

It is not hard to imagine a site that combines the sketching I did in Dia with the GazoPa service.

Update: Unfortunately GazoPa no longer exists. I guess you combine Google image search with a drawing program, but it would be more fun to do it with an indie search engine.

Image search by sketching in 2007

This is a post in my technology archaeology series.

What is search by sketching?

The idea is to search for images by drawing a sketch that roughly resembles what you are looking for. The sketch is your query. This idea was mentioned in years 2007, 2010 and sometime in the late 90’s (according to my friend Rasmus)

The idea is not new. A friend told me about an art search engine (i forget the name) where you could search for works of art by splashing colors on a crude web canvas, e.g. drawing some purple in the top, some yellow in the corner, and voila: “Is this the painting you where looking for?”

That is, based on your quick sketch, the algorithm finds matches in an art image database.

Applications of the technology

Here are some ideas for applications of the top of my head

  • Search for vector data in a spatial datasource. The user draws a sketch on top of a map (to get scale correct), and relevant vectors are returned. I and my colleague talked about how Denmark looks like the word Foo.

    So we naturally thought about something geographical that looks like the word Bar. This could be a chain of islands or a series of lakes. In essence you’d draw the word “Bar” and ask for vector data that looks similar.

Online mentions of search by sketching

There is a blogpost that also talks about the idea and mentions concrete technology:

This guy has something that looks like a product and even a youtube video

Also Microsoft in Asia apparently has been working on this

But where is it? Why doesn’t Google support this on their image search?

I’ve asked on stackoverflow

http://stackoverflow.com/questions/5458174/image-search-by-sketching-who-has-implemented

BitTorrent for geodata was big in 2005

Big in 2005…

Today I’m trying to find out whether BitTorrent + geodata is a “thing”. I have found out that it WAS a thing… in 2005! Just like Coldplay, Gorillaz, Eminem, 50 cents, James Blunt, Green Day… but it never really took off.

  • In 2006 Chris Holmes had a blog post titled Distribution of Geodata, where he said stuff like «What is needed is a standard, so that clients tile up the earth in the same way and make the same requests.» and «instead of asking the server to return a set of tiles that represents an area, it could ask a p2p network»
  • In 2005 Ed Parsons has a blog post titled Peer to Peer Geodata anyone ?, where he said stuff like «The idea of distributing large geodata datasets as small chucks is quite appealing and I have no doubt that when open geodata becomes more mature – this will be the obvious mechanism of supply» and «peer to peer means piracy in many minds, an unfortunate perception».
  • He and others mention GeoTorrent.org, a site offering geographical datasets via bittorrent.
  • In 2008 people ask: What happened to geotorrent.org?
  • In 2011, I’m asking the same thing: What is going on with P2P and geodata? Either I’m hopelessly old school, or a good idea simply went missing without a trace…

Ok, so people are still talking about P2P+geodata in 2006, 2007 and 2008, but the fact is that it has not seen a wide breakthrough in 2011. Or am I missing something?

GeoTorrent.org no longer answers HTTP requests, but it is still registered. GeoTorrent.org was run by ERMapper, who was bought by Leica Geosystems, who merged with Erdas, according to some person in 2008. It was a site devoted to offering geodata via bittorrent. Richard Orchard was one of the people behind GeoTorrent.org. Maybe he knows what happened to geotorrent.org?

Using the keywords “P2P” and “geodata” I went looking on scholar.google.com. I did not find that much, and nothing that has been generally adopted (see some of the hits in the Links section below).

What am I looking for in 2011?

What I’m looking for is something like a plugin for GeoServer, or a web-gis framework that fetches tiles via P2P, or something like GeoNode with a P2P twist. Actually GeoNode could be it… is GeoNode it?

Conclusion: Some pros and cons of P2P geodata

  • In 2009 a guy on a mailing list said: «Pure P2P solutions are great for exchanging large files, but typically have too much latency to be practical»
  • In 2010 some chinese guys said: «P2P technology offered a novel solution to spatial information online service and a good platform for sharing mass spatial data, it can avoid “single point of failure”and “hot spots bottleneck” problem»
  • In 2007 some austrians said: «As disaster management inherently happens in highly dynamic environments, these applications suffer from deficiencies with respect to maintaining connections to the server representing their sole source of information. We propose to exploit peer-to-peer networks to interconnect field workers.»
  • They also said: «P2P oriented raster geo-data online services have been widely applied, whereas vector geo-data online services still have many issues that can′t be handled, such as vector geo-data organization pattern, segmentation, lossless reconstruction etc»
  • In 2006 Chris Holmes said: «The damn brilliant thing about using an architecture of participation for geospatial data information is that as a layer gets more popular it scales perfectly, since more people downloading and checking out the layer means that more people are serving it up to others.»

If «P2P oriented raster geo-data online services have been widely applied», then where has it gone now? I’d like to find out…

Links