How Do You Ensure Streaming Quality?
Tavant's Charles Olson explains why streaming and quality engineering go hand-in-hand.
If you have recently ditched any of your streaming services because you finished binge-watching a specific show, your trial period expired, or your customer experience was marred by video or audio quality issues – you have a lot of company. In its 16th annual Digital Media Trends Survey, Deloitte finds that subscription video on-demand (SVOD) services suffer from an unhealthy turnover rate among subscribers. The rate is even higher among Millennials and Generation Z, so-called digital natives who grew up with email, smartphones, and Netflix and who have high expectations for a digital content experience. We’ll leave it to business leaders to solve the problems of binge-watching and trial subscription offers. But technology can address program and ad quality issues in ways that can directly cut your churn rates in half.
The key to ensuring quality across multiple content types, and multiple devices for content consumption, is an automation framework for streaming quality engineering (QE) that enables a “dual shift” approach – combining test planning, design and creation with an operational focus, monitoring quality across content and device types in real time. A dual shift approach to quality engineering ensures your software will meet your design specifications, while operational emphasis ensures performance and reliability.
The FIRE Testing Framework
The framework used by Tavant to enable the “dual shift” approach is called the FIRE framework. This proprietary system and process allows media clients to accommodate content-centric testing needs across mobile, linear, and web-based platforms. The FIRE framework also offers users:
- Non-functional, load and endurance testing
- Capacity planning and monitoring
- A robust set of APIs for tool-agnostic testing and integration with other QE systems
- A rich set of analytics tools that use the data produced through FIRE to drive cognitive analysis
The FIRE testing framework allows users to plan and execute testing across web, mobile, NRF, and API dimensions while enabling analysis on the health of any dimension This framework was developed by Tavant through our work with video quality engineering at one of the leading business news channels in the U.S. Starting in 2012, this blended team was tasked with designing and creating a framework that:
- Optimizes release turnaround time for new applications
- Is platform independent and extensible
- Has device testing capability as well as simulation capability
- Facilitates automated application build promotion
- Has network proxy capabilities
- Provides actionable and timely reports
The framework is designed to enable test execution from any location, with UI, API, mobile, web and native application testing capabilities. Cloud-enabled, FIRE can easily be integrated with other cloud-based tools in your ecosystem.
The modularized approach to functional libraries results in capabilities that can be reused and repurposed for many needs. Multiple testing functions may be run in parallel. The system supports BOD, has a plug-in for CI/CD enablement, and is easy to maintain through a modular, reusable library approach. Reporting capabilities enable real-time monitoring of vital operational functions and alerts that allow staff to address issues as quickly as possible.
The FIRE framework makes integrations with defect tracking and testing management systems a breeze. FIRE integrates with Slack and is cloud-enabled. Common function libraries simplify the development of core testing functionality, while business layers make client-specific functions trackable and testable. Reporting modules enable the creation of executive dashboards.
A sample report from the FIRE test automation framework
Success Story: A Leading Business News Channel
Tavant began developing quality engineering capabilities at a leading business cable news network in 2012. Since then, the FIRE framework has allowed technical staff to automate and operationalize video and audio testing across content types and consumption devices. One of the first challenges Tavant addressed was the automation of test routines that could enable an Agile, sprint-based approach to QE, allowing users to go beyond “after the fact” testing to a real-time approach.
This automation focus also reduces regression execution time, reduces regression cycles through better management capabilities, and creates a predictable cadence for everyday release testing. The team uses performance metrics to improve the functional value of support services. Other metrics are now used in tech-design sessions, promoting the “testability” of new functions. FIRE increases collaboration between testers and operational staff when apps are managed and/or debugged. The testing framework has also promoted a new level of governance for all related testing functions. Dashboards showing test execution results and product “health” at any time enable action to overcome quality issues. The client and Tavant created a formal testing review and audit process. Test failure alerts are now done in real-time.
The testing program has been so successful at this business news channel that the client expanded the scope of Tavant’s responsibilities to include multiple news organizations (and even their leading morning show). Within one year alone, the budget for FIRE-enabled testing has almost doubled, showing the immediate benefits of FIRE.
Engagement Models for Quality Engineering
Tavant engages on quality engineering projects in several ways:
- Project-based engagements deliver QE capability in a “time and materials,” timesheet-based fashion
- Resource-based engagements offer Tavant QE employees, working either onsite or offshore, as testing resources
- DevOps engagements, where a Tavant team manages pushing new applications into development, can often include QE services
- Custom development engagements should always include a robust testing component
- Testing capabilities may also be required when new partners or systems are brought into a client ecosystem
The Tavant Edge
We’d love to start a conversation on offering video and audio quality engineering services to your organization. We think the value of our FIRE testing framework becomes obvious to new clients within days of its implementation.
https://www.tavant.com/media/ott-services/testing
This article is Sponsored Content
Related Articles
The industry had a very fruitful and interesting 2022. In 2023 we will see continuations and finalizations of some popular initiatives.
14 Feb 2023