The Search is On
However, there are companies beginning to move forward with some interesting approaches. Chandratillake says blinkx is using speech recognition technology in conjunction with visual analysis of videos to provide a way to expose what’s inside the video or audio. "Our obsession is how much can you get the computer to actually understand the video itself or, in our case, the audio as well. To that end, we specialize in technology that not only finds the video that’s out there on the web and reads text around it, but watches the video and also listens," he says.
Gary Price, who runs the website ResourceShelf and who is also the director of online information sources at Ask, sees TVEyes and Nexidia as two companies pushing multimedia search forward. "Right now for some of the technology I’ve seen, whether it be what TVEyes or Nexidia is doing, is really what I call the state of the art—that is, being able to work with [multiple] languages and breaking down the text into phonetic sounds, so it’s much more accurate and cost-effective in terms of computing cycles and it’s much faster." He adds that using a phonetic approach is also more effective than trying to do pure speech recognition, which is trying to resolve a word against a word in a dictionary and is much slower.
Drew Langham, SVP of media at Nexidia, views this as a great advantage. "We take any recorded audio or video source and break it into a purely phonetic index. So what happens is we create this index, depending on the processor you are using, at about [340] times real time. This means one hour of media gets processed in about 12 seconds and rendered searchable. When someone wants to search for a piece of information, they type a text query. It gets parsed to the phonetic equivalent and matched to the exact point in the audio or video where it was said."
TVEyes monitors broadcasts for its subscriber clients using a hybrid of the phonetic and dictionary approach. David Ives, the company’s CEO, says TVEyes’ approach depends on the language being spoken. He admits there are limits to speech recognition technology, but he says it gets you further than looking at tags or other text information.
"We look at the audio track as the primary means of determining the content and context of what rich media is about. In a consumer-generated video like on YouTube, with lots of background music or a visual stunt, our approach would not be effective, but the vast majority of content has spoken content and we offer an economical way of determining the content," he says.
Clients report that TVEyes is an effective way to monitor mentions in broadcasts, a task that would be impossible without this technology. Ellen Davis, senior director of strategic communications at the National Retailer Federation, uses TVEyes to monitor news about any of its members or the organization itself; she says it’s particularly useful during the holidays when mentions skyrocket. Davis manages this by getting a daily digest of all mentions by email. What’s more, she can access broadcasts and email meaningful snippets to her boss. "With this service we can very easily splice a three-minute segment and email it to the CEO so she can watch it on her computer."
Podzinger takes yet another approach by using voice recognition technology developed for the U.S. government by parent company BBN. Alex Laats, Podzinger’s CEO, says that by using this approach and then categorizing the audio and video files by subject, Podzinger makes it easier to pinpoint the desired file and the desired element within the file. "The basics of what we do is making audio and video searchable using full-text search and we also do natural language processing in order to topic-classify it and to make the content more readily accessible," Laats says. This means that if you are looking for a mention of New York Yankees third baseman Alex Rodriguez, you can enter his name in the search box, or you could search for "sports" and "baseball" to find him. Once found, Podzinger not only displays the relevant podcast or video, it shows you exactly where in the broadcast the mention occurs, sparing you the time it would take to listen to the whole thing.
Related Articles
Viewers will be able to search over 600 daily weather clips to get local or international reports.
24 Aug 2010
Agreement brings wealth of entertainment clips to leading video search site.
17 Aug 2010