Moving Beyond Text Search

In the area of gesture recognition, researchers are attempting to categorize particular facial and arm movements to track emotion over time. Searches for particular types of gestures currently require keywords, such as sad, happy, etc. Just like AltaVista and several other search engines that have allowed users to search for still photos that bear similarities to the original image, however, the research into gesture recognition is also rapidly moving into attempts to search video based on other video clips.

In the area of focus of attention, new research has enabled an indexing system to track where in a room a person is looking, as well as on what or whom she is focusing attention, and for how long. Demonstration videos already show this in action, albeit in rudimentary form. But the essence is there, and AMI is now beginning to seek out locations to transfer the technology from the lab to the practical meeting scenario.

Which brings us to the Torino project. Dodging tourists yesterday, just like we did when the Olympic Winter Games invaded my hometown of Lake Placid, New York, in 1980, we were escorted into a beautiful old building whose ongoing remodeling is the initial project in a corporate university for a major European bank group. The request to meet for a few hours had come less than 10 days prior, but the meeting was well worth the long, rapid trip from a snowed-under New York City: one of the topics on the agenda was an indexing system that would handle multiple modalities for indexing and search.

Interestingly, the request came not from a technician or someone in the IT department but from the business side of the house. A senior executive within the human resources arm of the bank group was concerned that those attending meetings and training weren’t able to fully capture the nuances of meetings they were attending, and that the failure to record and search the content of these meetings meant that the company was less effective than it had the potential to be. Also interesting was the fact that the company needed to be able to search for content in one language that may have been spoken or recorded in another language, as the bank group had recently merged with a bank in another part of Europe.

Could a strong relationship between the researchers and a strategic customer with a business need address all the limitations that plague current video search systems? Probably not. But as with any large issue, the puzzle is solved one piece at a time.