The Business of Stream Metrics: Quantifying Quality

What About Other Approaches?
No single strategy for measuring stream quality over an IP network wins the unanimous support of the experts in this field.

One of the objections to the Keynote and Streamcheck approach for measuring stream quality is that it does not actually measure the "true" end user experience. While the agents simulate the end user experience of a specific profile of user, people find themselves in diverse situations when requesting a stream and these different environments may not all be accurately reflected by the network of measurement computers. Alternative approaches to measuring quality have been tried but for business reasons have not survived the test of time.

The rub comes with the question of how a company (either a streaming service provider or a content publisher) can use this information to reduce costs or generate new revenues. Since in 2001 and 2002 streaming media was not generating copious revenues on a per stream basis model and consumer and entertainment sites struggled with their advertiser-reliant business models, the providers of streaming metrics faced a limited opportunity for growth. Under these circumstances, the first to market (Streamcheck and Keynote) secured most of the accounts that needed independent verification of a service provider’s Service Level Agreements (SLAs).

There are other reasons why streaming quality metrics might be of value. For example, to optimize the bandwidth usage by streamers, quality metrics might be the basis of intelligent routing instead of overprovisioning on expensive backbones. Backbone capacity in the Internet remains so underutilized,though, that this argument is not very persuasive.

Another theory in late 2001 and early 2002 was that business customers would find significant value in superior quality stream experiences and that the viewer of higher quality streaming content would be willing to pay a premium for these experiences.

Led by streaming industry veteran Pat Greer, Broadstream (which later changed its name to Netelligence) developed another way to use the information gleaned from end user based experience monitors. Netelligence built a client software that ran on the end user’s desktop PC and provided data about the actual end user’s experience. At intervals during the stream delivery, the software sent that quality data back to a Netelligence-managed server. The first business model Netelligence tested was that content providers could be persuaded to pay a premium for higher quality delivery supplied by best of breed service providers. The extra cost would then, presumably be passed on the customer or rationalized as brand protection.

In the second business model Netelligence tested, the company proposed to help streaming media content providers manage the stream delivery within their own networks through better end user metrics. Finally, Netelligence offered to use the metrics to initiate route changes during a stream. Based on business rules entered by the service provider or streaming content owner, the Netelligence server in a POP would switch the stream from one backbone provider to another (or apply other rules that would improve the delivery of the content) when the experience dropped below an agreed upon level. "Unfortunately," recalls Greer, "this third business model appealed to large enterprises but failed to attract the support of the venture capitalists. The service providers felt that quality was good enough and the cost of provisioning the content on multiple networks in case of congestion on any one network was prohibitively high due to the fixed cost of storage."

Another developer of intelligent end user experience (audio video quality) monitoring software, Streamcenter, also found it difficult to monetize their technologies. Streamcenter attempted to license its end point-based stream metric technologies to the streaming media format providers or streaming media server vendors. Industry consolidation and difficult economic conditions facing the remaining streaming service providers caused the potential market for quality measurement technologies to decline. Like Netelligence, Streamcenter was unable to raise the capital necessary to sustain its business until other revenue streams could be identified.

In the end, the only business model that has persisted is the one used by Keynote and Streamcheck where on a contract basis, the company’s scanning agents request specific streams at known intervals and compile the information over time.