The Search is On
The web was once all about text and an occasional picture, but broadband begat bandwidth, and bandwidth spawned a boom in multimedia content through which video and music sites like YouTube became wildly popular and were bought and sold for huge sums of money. Yet in spite of all this rich content, the major search engines remain mostly text-based, relying on titles, tags, and metadata when searching for multimedia content.
The big free search engines lack a way to search inside multimedia content, something that has been available for 10 years in searching text. In order to delve deeper into multimedia, you need to use a specialized search tool from the likes of blinkx, Nexidia, Podzinger, or TVEyes. As one analyst describes it, multimedia search is still in the crawling stage, but sometime in the not-too-distant future, it’s going to grow up and take off fast.
From Genesis to Revelation
In the beginning, there was text search. We had some keywords and a title, and we thought it was good. Long ago, text search matured to search every word in a document (full-text search), but according to Suranga Chandratillake, CTO and founder of video search technology company blinkx, until recently, search tools also focused exclusively on text. He says, "Search engine technology has always been built around the idea that you are looking for text. In that sense it’s self-descriptive because computers can read text and you know what words are there and therefore you have an idea of what’s relevant. Obviously there is a lot of detail around what is more relevant, but at least you can make pretty good decisions."
Chris Sherman, executive editor of the site Search Engine Land, who has followed search since its early years, says most multimedia search has advanced only to the text search evolutionary equivalent of 1993 or 1994, when it looked at titles, links, and keywords. According to Sherman, "A lot of these multimedia sites that people call search sites really aren’t. They use things like tags and other types of text information to figure out what’s out there and refer people to multimedia files. They are not ‘full’ search like we understand full-text document search."
The challenge, Chandratillake says, is moving beyond text search because video and audio may have some text available that could be searched, but that won’t give you a full sense of the contents. "Now, because of the way the web is, there will usually be some associated text. There will be files, metadata, and text on the web page around the video or image and that’s how the majority of multimedia search has occurred in the past." He points out that if you do a video or image search on Google or AOL, it’s not really searching the video itself or the image itself. He says, "It’s looking at stuff like: Based on the fact that this image seems to be tagged with the words ‘George’ and ‘Bush,’ it’s probably a photo of the president and so on."
Other key issues, according to Sherman, are the subtle elements of multimedia content. Humans understand these elements intuitively, but today’s technology does not. "When you have streaming content," says Sherman, "you have a lot of information contained that’s non-textual, like body language or inflection. Is somebody being sarcastic when they say something or are they being straightforward? It’s still very difficult for a computer to understand that."
Are We There Yet?
We are not at the final destination by a long shot, but multimedia search is beginning to move beyond simple text, even if Sherman thinks we still have a long way to go. He says, "If you look at true multimedia, like video, audio, or other types of streaming content, really nobody is doing search in the sense of, ‘Let’s get in and really understand this content, analyze it, and make it searchable.’ Some companies are doing simple things like speech detection analysis or looking at waveforms in audio and doing similarity analysis, but it’s still basic and doesn’t work consistently well."
Related Articles
Viewers will be able to search over 600 daily weather clips to get local or international reports.
24 Aug 2010
Agreement brings wealth of entertainment clips to leading video search site.
17 Aug 2010