June 21, 2019
By Jun Heider Director of technology, RealEyes Media
Short Cuts

Video: How to Make MLaaS Work for You

Learn more about machine learning and AI at Streaming Media's next event.

Watch Jun Heider's complete presentation, VES103. Enhancing Media with Machine Learning in 2019, in the Streaming Media Conference Video Portal.

Read the complete transcript of this clip:

Jun Heider: Let's talk about how to make MLaaS work for you. I think the biggest thing to do is ask yourself some questions first. What kind of content do you have? What is your use case, what are you trying to do with your content? How accurate do you need the results?

Somebody that I was talking to in a previous session told me they work for a firm that deals with content that is government legislation and things like that. When you're talking about government legislation, especially in Europe, you want to make sure that that is a really accurate transcription--otherwise people may get mad. So that's a case where you want high, high accuracy.

Do you have any developers on your team? A lot of the services expose an API, and often you're going to want to use their API to actually build some useful applications around what you're doing with these machine learning services.

Finally, your content is your content, and it's not my content; it's specific to you. Some of these services are going to work better with certain types of content, because they haven't had the ability to label every single video that ever existed in human history, and run it through their model. So, some of these are going to be tuned to sports, some of these are going to be more tuned to nature videos that have a lot of animals in them. So you're going to want to take your content and leverage these trials they make available, to see which ones are the best ones for you.

What is your content profile? Movies, user-generated content, nature clips--play with them. If you have a developer on your team, or you are a developer, or you have a team of developers, definitely take a look at the API.

The nice thing about Video Indexer is they actually let you play with their services in their API documentation, so that's a really cool feature. And then those spreadsheets that I shared with you, this is kinda what they look like. We went from Microsoft Video Indexer, Valossa, Google Video, Intelligence, and AWS Recognition, and we pulled all the tags out. We checked to see how accurate they were, and if they weren't, what are our thoughts on why they weren't?

For instance, the Video Indexer recently said that there was a gun in the video, but then when I went back to the frame where it supposedly saw a gun, it was actually an old-style camera. So the old-style, with the flash bulb at the top, and somebody was side-cocking the camera, which I could think is why it may have thought it was a gun. Another example is, I ran a very old black-and-white cartoon and there was a villain that was kind of doing this, and it thought it was a bat, because it was black and white and he had a big black cloak on.

Feel free to take a look at this, see where the services return similar tags, different tags, where they get tags wrong. This will get you started, but the thing to note is, we did this in 2018. These models are constantly learning, they're constantly training these models, so I can guarantee you, if we went back and did this again, it would be different. But this will at least give you a feel for what kind of stuff to get out of these services.