A Round-Up of Mobile Visual Search Companies

This post is an overview of all the companies developing mobile image recognition search engines. For people to whom that means nothing, see these posts. My robotics readers will have to forgive me for my sudden obsession with this topic, but I work on visual search engines for my PhD, and I’m giving serious thought to joining this industry in one form or another in the next six months, so I’m rather excited to see commercial applications taking off.

Right now is the perfect moment for this technology. New platforms like iPhone and Android are offering the first compelling mobile web experiences, and have built-in cameras of acceptable quality. Coincidentally the computer vision research on large scale object recognition has just recently reached the necessary level of maturity. Three search engines launched in the past three months, and a few more appear to be in the works. Some have been available for a while via mms or email, but it’s only as iPhone apps that they are beginning to be seriously useful.

Visual search is still very small. Right now I expect it’s generating almost no cash flow, so every company in the field is also pursuing alternative uses of the technology. Some of these are interesting businesses in their own right. The most popular application is enhanced print advertising, where newspaper or magazine readers are encouraged to take a picture of an advertisement and send it to a shortcode. The image is then recognised, and the user gets back extra content or is entered into a prize draw, etc. Advertisers get enhanced engagement, and crucially they also get a way to measure the effectiveness of their offline advertising spend. There is serious advertiser hunger for this, and the services are already growing quickly. A recent Kooaba campaign took the idea a step further.
Attractive as these applications are, I think they’re mainly good to pay the bills and raise awareness in the short term. Longer term I think more typical search applications are going to dominate.

So, the players:

Update, October 2009:

Since writing this post, I have co-founded my own visual search company, Plink. Plink was spun out of the computer vision and robotics research groups at Oxford University, and we have some of the best technology and and best visual search engineers anywhere on the planet. Our Developer API gives anyone access to the system, and entry level usage is free.
Our own first app built on the platform is called  PlinkArt. The app lets you identify famous paintings just by taking a photo, and we are trialing it as a replacement for traditional audio guides in museums. The app was selected as one of the $100,000 prize winners in Android Developer Challenge, and an iPhone version is coming soon.

My original, unedited thoughts from 2008 are preserved below. The main news since then is that SnapTell were acquired by Amazon, and that in the last few days Google pre-announced their internal Google Visual Search or “Google Googles” project. You can read all my posts on the subject here.

Original post continues:

SnapTell

SnapTell arguably leads the field at the moment. I believe they currently have the largest search index at 2.5 million items. This includes most book covers, games, CDs and DVDs.  Their recognition technology is first rate, even with poor images. Despite their search engine being so impressive, their main focus seems to be on their enhanced print advertising business, where they have a number of high profile clients like Sports Illustrated and Rolling Stone. This focus shows – they have slick marketing materials aimed at magazine advertisers, but rather poor landing pages for their search engine. However, their search engine app just got a rave review on TechCrunch, so perhaps search will be moving up their agenda as traffic goes up. Venture funding from NEA.

Kooaba

Kooaba are another company with first rate recognition technology (check out some examples). They’re a spinout from ETH Zurich, and are run by some well known computer vision researchers. As with SnapTell, their main current revenue stream is from enhanced print advertising, and some interest extensions of the concept. They also have an iPhone app for movie search. The number of items indexed is still quite small, though no doubt it will be expanding soon. Their technology certainly scales to millions of indexed items.
For other companies looking to acquire visual search capability, kooaba is a peach.

TinEye Mobile

Idée are a Canadian company involved in several types of image search. Their TinEye Mobile app for iPhone launched late last month, and currently recognises CD covers. Their main current revenue stream is from another product called PixID, which monitors print and online publications for unauthorized use of copyright images from clients like the European Pressphoto Agency. You can try the same system for web searches at TinEye.com, where they have a little over a billion images indexed. They also have a cute colour-based Flickr search.
The technology behind TinEye is quite different to the Kooaba/SnapTell approach. It can jump through hoops in its intended application of large scale near-duplicate detection for PixID, however, it’s not quite so good at dealing with some of the challenges of camera phone images, particularly geometric distortions. However, Idée strike me as a company with excellent corporate DNA and good PR management. They had 23 employees and were entirely privately funded as of last year. So while their technology is not quite as good as SnapTell or Kooaba, I think they’re one to watch.

Evolution Robotics

Evolution have a number of vision based products, notably for robot navigation and theft prevention in shops.  They released a video preview of an iPhone visual search engine back in April, but so far it’s not available on devices. A different version of their software is apparently big in Japan, (can’t resist), though it’s hard for me to assess that.
In addition to their own play, Evolution are taking a parallel tack as a technology provider. They are the primary licensee of David Lowe’s SIFT patent, which is a key ingredient of many of these search engines. Other players such as Mobot and possibly others have licensed from them. They recently took a chunk of investment from Vodaphone, which is suggestive.

Pongr

Pongr are a company I only came across a few days ago. They offer price comparison, optionally driven by image recognition. I haven’t had a chance to try their software yet, but based on their video demo their recognition technology is reasonable but not fantastic. Possibly licenced from somewhere? The image database is currently small, only thousands of products. However, where they might win is with a focus on UI, Facebook integration and all those other ingredients that make up a good experience.

SnapNow

SnapNow are another player in the enhanced advertising space. My impression is that they don’t have the necessary technology to roll out a full search engine. They’ve been concentrating on Japan since 2005, and have just launched in the UK, US and China. Judging by their patents, their recognition technology is out of date, but that’s not to say that it doesn’t work. Unless they’ve changed technology, they will likely have some trouble scaling to large index sizes. Without a smart phone client, the experience is also pretty clunk – you have to send a picture by mms (expensive) and get back a link in an SMS message (slow). However, they have a fairly established presence in Japan, and have already signed up a number of initial magazine clients in the UK.

Vodafone

Now we come to the 800-pound gorillas. Vodafone demoed a visual search engine called Otello at CeBIT in March, where it was amusingly panned by the German Chancellor for failing to recognise her. (In fairness, none of these engines can do face recognition). I must admit that I don’t know anything else about Otello other than what’s in the Engadget post. Nokia also have some dubious patents in the area. I’m highly skeptical that any major carrier can really pull off a project like this. If they want to own this space they’ll be acquiring start-ups. Nokia have a $250 million fund for supporting exactly that.

Microsoft

Microsoft research is exceptionally strong in computer vision – they employ many of the top people leaving academia. While there is a long history of research outfits where cool things are invented and then gather dust, MS Research have taken several ideas all the way to shipped product, the main example being Photosynth. They also have a prototype visual search engine called Lincoln. There is an article discussing the system here. Whether Microsoft has the motivation to make this a hit, I’m not sure, but they certainly have the setup to follow quickly if visual search takes off.

Google

Google are as usual a huge unknown. They haven’t announced anything, but they certainly have many good computer vision people from their acquisition of Neven Vision back in 2006. Neven Vision even hold a patent on mobile visual search engines, and YouTube has some pretty strong related technology for detecting uploads of copyrighted video. They also hired Henrik Stewénius, who published one of the seminal papers on visual search. On top of that, Eric Schmidt recently had some suggestive things to say about interactive magazine advertising, talking about how “young audiences will want a more interactive experience in print.” Make of that what you will.

Others

Several other players out there at early stages or with lesser offerings.

IQ Engines has a good demo, but is still pre-release. Based on their research papers, their technology is quite a different approach to anyone above. It’s an interesting direction in computer vision at the moment, but I wouldn’t have guessed it would be ready for prime time. The demos look good though.

Thrrum/23half claim to do general visual search based on text extraction, though I’m skeptical to what extent their technology actually works. Mobot do enhanced print advertising using the Evolution Robotics engine. No smart phone app at present, so the experience is not much fun. Mobile Acuity are another similar outfit, I am not sure that they are still active.

Whew, long post. As far as I know the above is a complete list of companies in this space. The really significant players are SnapTell, Kooaba, Idée and to some extent Evolution. To put my cards on the table, I’m thinking seriously about joining someone on that list or starting a visual search company myself. I think this is going to be a big hit, and there are still many niches unfilled.

If anyone knows of a company I’ve missed, please drop a comment below.