Video: What AI Platforms Can and Can't Do

Learn more about video AI at Streaming Media's next event.

Read the complete transcript of this clip:

Jun Heider: As we're going through, looking at all these labels from these various services, we came up with some interesting thoughts. We found out that Google really excelled at human and animal anatomy. So it could say, “Hey that's an elbow on an arm, or it's a beak on a bird.”

So that was interesting. Interesting things that you'll find out as you start getting your feet wet and playing.

AWS, apparently, is very good with plant genus and cloud formations. My favorite part is the machine learning algorithm that detected a cumulus cloud taught my developer what a cumulus cloud is. That's pretty cool.

So the machines are teaching the humans. Valossa does really well with actions--smiling, laughing etc. And then Microsoft, they can also detect actions and there are different actions there.

But Microsoft is especially good at fighting scenes. So that was interesting. Because some of the other services, when we ran the content through, when there was fighting, they said video game. But Microsoft says, no that's fighting, that's not a video game.

So my recommendation to you is utilize these trial accounts and pick the content profile that is a subset of the type of content that you have. If you're a news organization, get a bunch of content that's people on a news team talking about things. If you're a nature company, get that nature video library and create these models.

But before you do that, just play with these services. Play with the Video Indexer, play with Valossa, play with Veritone etc.

And, I definitely, AI isn't here to replace us, it's here to help us. So it's going to be humans and AI, it's not just AI and it's not just humans. A really good example is with model training, I submitted this black and white cartoon to one of the services, I can't remember which one, and maybe I'm saying that because I don't want to get one in trouble, but it was a black and white cartoon from like 1930. And this guy is hopping across like this with this black cape. What do you think it detected? Not a guy in a black cape. It detected a bat. And it's like, “okay, I can see why that's a bat, I can see.”

It's like a five year old. It's never quite seen something so it's using what it understands through its own trained model to make it's best guess. And that's where that confidence level comes in. And I don't think it was very confident that it was a bat, but I thought it was funny that it thought it was a bat because I could see how it thought that it was a bat. Just like the mirror example earlier with the computer monitors.

So AI and humans, if you want to go down this unified path of leveraging both, Clarifai has a really interesting model where you can just do machine object detection, or you can combine it with their team of manual taggers or you can combine it with your own team.

So it's like best of both worlds. You're getting some basic stuff in and then you're also leveraging human metadata taggers to help out. Hive has two different services.

One is called Predict, which is machine learning, and the other one is humans, it's Hive Data. And make sure to put those checks and balances in. Maybe it's good enough for you to say, hey I'm going to use Video Indexer and any of the labels that come back, I want to make sure it's 90% confident. And then you just go through some of those initial index videos and just make sure it's helping you out in the way you need it to.