Video: How to Use Microservices to Get Data Out Quickly
Learn more about microservices and OTT at Streaming Media's next event.
Watch the complete video of this panel from the OTT Leadership Summit at Streaming Media West, OTT102. OTT Moves Toward Microservices, in the Streaming Media Conference Video Portal.
Read the complete transcript of this clip:
Steve Miller-Jones: We built a new service called Smart Purge a couple of years go. It was to solve a specific problem of how do you, if you need to, very quickly get rid of something that's in a cache? If you have a video chunk that you misencoded, or it has the wrong bug in it or take-down request or whatever, and this was a real challenge, we took a real design-lead approach and I think this is critical in not only microservices but any software development. With microservices in particular, if you're going to break things down so that individual service is self-contained and you have a meaningful output for support so you can quickly triage and fix bugs, iterate and release fixes, for example.
So we had a real design-lead focus on the requirements of how quickly it should take to do anything. And this took us to a point where we had five key parts to the system we had to develop. So we had to create a mechanism to create some form of query language that people we're going to use through a common API to describe what they wanted to purge. That in itself--the mechanism of describing and writing that to a database, for want of a better word--was a key microservice in itself.
Batching those, putting them together into groups that could be distributed and acted upon, executing, or batched, we shouldn't batch them, and we should be able to apply different levels of quality of service to different customer types, thinking about batching was another different microservice that we needed to use in order to put these queries together.
Then we looked to distribution and state-tracking. Did we distribute something, and can we track the state of that request when it's gone out? So we wrote a hold over our approach goals for managing the distribution of these queries and these batches and determining if they'd got out to the edge. So that we could be sure we were hitting the requirement of being a 99.5% sure that we're going to actually get rid of that object that's in cache.
Because the worst thing when you're purging is, "Yeah, I told you I purged it with your next request still working on an object." So those three things--distribution, state tracking and the messaging protocols--we considered to be independent services too. To approach all of those and then give that to an operational team who were gonna get speeds and feeds from the system, how many requests are we doing, how quickly are they being executed. Are we breaching our SLAs on purging, and being able to escalate quickly to someone who can implement and change the system was really important.
So the other part of the microservice and self-containment of it is being able to get core data out quickly so whether you use Zabbix or whatever your monitoring systems are, that's a really cool part of microservice design.
Related Articles
StackPath's Nathan Moore and EZDRM's Olga Kornienko explain the process and benefits of monolith-to-microservice transition, such as identifying points of failure, in this clip from Streaming Media West 2019.
01 May 2020
Cisco's Nermeen Ismail, id3as' Dom Robinson, and Twitch's Tanu Aggarwal resolve the age-old microservice/container conundrum in the streaming world in this clip from Streaming Media West 2019.
29 Apr 2020
Twitch Director of Enginerring Tanu Aggarwal explains the basics of microservices and the advantages of designing and deploying self-contained, single-purpose services in this clip from Streaming Media West 2019.
24 Apr 2020
Netflix Senior Software Engineer Sujana Sooreddy describes how microservices work within Netflix' architecture in this clip from her OTT Leadership Summit panel at Streaming Media East 2019.
07 Oct 2019
Imagine Communications' Mark Myslinski and Bitmovin's Paul MacDougall discuss advantages if microservices for broadcast applications such as agility and scalability in this clip from their OTT Leadership Summit panel at Streaming Media East 2019.
27 Sep 2019
MediaKind chief technology and strategy officer Mark Russell discusses common pitfalls of microservice-building in this clip from his panel at the OTT Leadership Summit at Streaming Media West 2018.
14 Dec 2018
MediaKind chief technology and strategy officer Mark Russell discusses microservice software development in this clip from his panel at the OTT Leadership Summit at Streaming Media West 2018.
12 Dec 2018
Streaming Media's Tim Siglin interviews MediaKind CTO & Head of Strategy Mark Russell at Streaming Media West 2018.
21 Nov 2018