Progress, Trust, and Going to the Pub: London Search Social write-up

Thursday 14th January, 2010
The Elgin on Ladbroke Grove

The Elgin, via flickr/Ewan-M

There’s a theory that claims expertise and associated salaries increase more rapidly in cities than they do in the country because physical proximity decreases the cost of sharing ideas (I’m desperately trying to dig up the source amongst the noise; which is either ironic or poetic depending on how you look at it).

The interwebs are a different beast. Proximity doesn’t exist in the same way, perhaps instead becoming a cultural rather than a geographic measure of separation. The cost of spreading an idea on the internet is more related to trust than physical distance.

On the internet, that trust must be earned by expertise and clear communication. In the physical world people behave differently, people tend to trust each other more quickly and be more open when they can look each other in the eye, or buy each other a beer.

[Only on a technology blog would you find a justification this obscure for going to the pub.]

On Tuesday we held this month‘s London Open Source Search Social at The Elgin in Notting Hill. This was the first time we’d used our shiny new Meetup account to organise the event, so it was nice not to have to send out reminders manually (laziness #ftw).

A few notes from the evening for those whose memories are as bad as mine

There’s plenty missing, and some of this may be fictitious.

Bruno from Jobomix talked about his use of Hadoop to detect duplicate job data, leading to a conversation about Pig and Cascade, then other distributed systems like Scala. Ben from OneIS brought up the subject of Duby, a Ruby-like-but-tidier language targeting the JVM, and when prompted gave us an outline of his company’s free-text graph store.

We talked about duplicate detection in various fields, thresholds, and the cost of false positives. We touched on human relevance testing; Richard told us he’d found people generally need to be paid to do it and not for more than 30 minutes at a time.

Joao from the Royal Library of the Netherlands told us how they digitise and index millions of pre-digital documents per month. Ben told us about a method of querying Xapian from Postgres using an SQL JOIN.