Dose – alpha entering final contractions

Thursday 27th November, 2008

We’re nearing the completion of the first Alpha of our new content-based image retrieval (CBIR) engine. It seems wrong to let it be born without a name, so I’ve settled provisionally on Dose which stands for Distributed Object Similarity Engine.

It’ll be a little while before we’ve got any user-facing products to show for all our hard work, but we’ve learnt a lot and should have something good at the end of it.

Idée’s Multicolr Search Lab

Thursday 27th November, 2008

This morning I had another play with the Multicolr Search Lab from visual search outfit Idée Inc and decided to make some notes, which I’ve posted below.

Idée in context

Idée are one of the bigger players in the visual search space, although they currently occupy quite a different market to my new team Pixsta.

Idée’s biggest product so far is TinEye, which is used to find uses of a single specific source image across the Internet. The main commercial use appears to be detection of copyright infringment, a service they provide to photographers and copyright owners for a fee. To my (admitedly limited) knowledge they’re the only company offering this specific service.


The Multicolr Search Lab (MSL) is a proof-of-concept that demonstrates Idée’s ability to index image by colour. As an image-based ‘labs’ project, the UI naturally reflects a mixture and Google Labs functionality and Flickr’s Web 2.0 styling. Its clean and simple. I like it a lot.

Naturally as a ‘labs’ project it has no direct revenue stream, but it’s a nice demo and there may well be use for this type of technology in some areas: Interior design for example; add a simple hook-up to a printing service like Photobox and you’ve got a revenue stream for photographers and your own service.

As Multicolr is currently running at an adequate speed over a stated 10 million images I’d guess that the underlying technology is ready for at the least enterprise scale applications, if not internet scale.

MultiColr engine

From my previous play with Multicolr I has certain expectations as to how it worked internally.

In any search application you need precision, but you also need to be able to bring back close matches if no direct matches exist. Since MultiColr is using RGB hex values as query terms I’d suspected that they’d rounded those hex values to match their chosen quantisation, and were matching on those approximate colours.

This isn’t what they’re doing, as I discovered when I changed the colour specified in their nicely RESTful URL to a subtely different colour (one that should fall within the same quantisation bin). If they were using naive quantisation then the results would have remained the same. In fact they changed.

So although they may still be using some form of quantisation to avoid gaps in queries, it seems like they’re also allowing the raw colour value (either in RGB or a different colour space) into the index, and scoring images appropriately based on each query, i.e. the quantisations seen in the UI may well be totally arbitrary.

All in all, an good piece of work. Well done Idée.

Incogna goes live

Tuesday 25th November, 2008

Congratulations to the Incogna guys for getting their visual search engine out of Alpha. Good work.

I Love Visualisations

Thursday 20th November, 2008
I love visualisations

Visualisation du jour

I love visualisations. Not in a short-lived fiery way, but in that way that’s based on mutual understanding, respect for each other’s space, and not hogging the remote control.

A good visualisation can sell an idea. A good visualisation can let the human eye’s abilities take a set of data and run with it. A good visualisation can inform a decision, or even prompt entirely new questions.

Visualisation of the day

I’m not going to try and explain this visualisation on the left, partialy because it wouldn’t make much sense to anyone but one or two people I work with, but trust me, I found it really useful.

It reveals quite clearly the aspects of this data I wanted to isolate, highlights where I’m going wrong, and has told me a few things about the underlying system that I hadn’t realised. The only thing I’m not happy about is that it’s not in colour.

I don’t think I need a second opinion. I’m a data geek.

Update: Have now given in and created a colour version where the green RGB value is set to intensity, and red is proportional to standard deviation. It rocks my dweeby little world.

Image search and Incogna’s beta

Saturday 15th November, 2008

I’ve written previously about work by Canadian image search technologists Incogna to harness the power of graphics processors to index images, but I’ve only just found time to try out their image search beta.

A bit of context

The challenges surrounding image search are significant and numerous (just like the opportunities). To start with, an image search engine has to have most of the main features and properties of a text search engine, so problems include some degree of natural language processing, semantic indexing, and all the problems that come with creating a distributed search index.

When you add image comparison into the mix that puts even more load on your infrastructure in terms of bandwidth, storage, the CPU required to extract usable data from images, and the data structures required to hold that visual data.

While those aspects of a search engine design can be tricky to get right, I don’t see them as the hardest problem. I think the hardest problem is caused by the fact that a picture paints a thousand words. An image contains so much information that it’s hard to know exactly what aspect of it a user is looking at… and hence what they mean by similarity and relevance. For example, if a user uses a holiday snap from Yosemite as a query image, are they looking for other pictures of Yosemite, or for shirts like the guy in the picture is wearing?

The point I’m getting to is that nobody has really nailed this problem yet, because it’s hard.

Incogna Beta

Playing with this search engine is fun. The user interface is slick and responsive, and search results are returned in a fraction of a second (it’s hard to overstate how important the perception of speed is to user experience). Looking under the hood the interface is using Prototype to handle animation and talk to a JSON API, which is a solid choice.

Getting results

From the results returned from a few similarity searches, they seem to be calculating similarity using some sort of shape analysis, some colour data, and also making significant use of non-visual metadata.

The input data they’ve crawled seems to have come from a fairly wide range of sources, although it’s hard to see how big their index is in total since they return quite a tight result set, opting for precision rather than recall.

That leads me to my only criticism of the beta, which is that precision is only useful if you can be sure you have a reasonable understanding of what is relevant to a user. The beta doesn’t offer the user a mechanism to help determine relevance; the “thousand words” problem I described above. In order to become really useful, users have to be able to help the search engine decide what is relevant. That said, I don’t have all the answers myself, and Incogna seem like a smart bunch. I expect them to have some very interesting ideas over the next 12 months.

Trust in the Software Market

Sunday 9th November, 2008

So it seems that Spore’s creators are on the receiving end of a couple of law suits for secretly installing unremovable software on customers computers.

While I don’t know exactly what chance these suits have, I wish them the best of luck. Why? Because markets depend on trust, a point that the people who decided to quietly install SecuRom with Spore seem to have missed.

Trust is required between vendors and customers; trust that the price will be fair, trust that the goods will be as described, trust that the payment will be valid, and trust that privacy will be respected. Electronic Arts’ decision to ship this DRM payload breaks that trust because they try to hide it, and because they take the liberty of making dangerous and irreversible modifications to your property.

As George Akerlof highlighted in his seminal 1970 paper ‘The Market for Lemons‘, markets without trust (he describes a car vendor’s hidden knowledge of a vehicle’s state as “information asymmetry“) can be difficult and dysfunctional, with the whole market affected by a few bad eggs.

If trust deteriorates between software vendors and software consumers, the software market will become a worse place for everybody.

Iggins: Conceived, not yet born

Tuesday 4th November, 2008

So I’ve started my first open source project. Aiming small I’ve chosen to implement a language identification tool on Appspot. It’s named Iggins, after ‘Enry ‘Iggins from Pygmalion.

Lets see if I can make the time to get it running.