This is a post in my technology archaeology series.

What is search by sketching?

The idea is to search for images by drawing a sketch that roughly resembles what you are looking for. The sketch is your query. This idea was mentioned in years 2007, 2010 and sometime in the late 90’s (according to my friend Rasmus)

The idea is not new. A friend told me about an art search engine (i forget the name) where you could search for works of art by splashing colors on a crude web canvas, e.g. drawing some purple in the top, some yellow in the corner, and voila: “Is this the painting you where looking for?”

That is, based on your quick sketch, the algorithm finds matches in an art image database.

Applications of the technology

Here are some ideas for applications of the top of my head

• Search for vector data in a spatial datasource. The user draws a sketch on top of a map (to get scale correct), and relevant vectors are returned. I and my colleague talked about how Denmark looks like the word Foo.

  

  So we naturally thought about something geographical that looks like the word Bar. This could be a chain of islands or a series of lakes. In essence you’d draw the word “Bar” and ask for vector data that looks similar.

Online mentions of search by sketching

There is a blogpost that also talks about the idea and mentions concrete technology:

• Search photos by sketch or image with Retrievr and ImgSeek

This guy has something that looks like a product and even a youtube video

• Chris Harrison: Search by sketch

Also Microsoft in Asia apparently has been working on this

• Interactive Image Search: Search by Color Sketch – Just Released in Microsoft Bing Image Search

But where is it? Why doesn’t Google support this on their image search?

I’ve asked on stackoverflow

http://stackoverflow.com/questions/5458174/image-search-by-sketching-who-has-implemented