The Business of Stream Metrics: Quantifying Quality

The Architectural View
Architecturally, the approach both companies took was well established by others conducting Internet measurements and the same that Exodus’ Service Metrics group had developed. You take servers, load them up with databases and a mixture of custom and off the shelf software (including browsers and streaming media players), and program the players to request files from streaming media servers. Then, other software on the server measures the stream response time (time to start), any interruptions in the transmission, packet arrival statistics (packet loss, out of order, and jitter) and frame rate associated with the requested stream.

What differentiated both Keynote and Streamcheck from predecessors and competitors in the business of gathering streaming metrics was that the data analysis produced a single number. By placing weights on different metrics, the final rating of a stream reflects the relative importance of different performance parameters studied. The objective is to quantify the quality of the viewing experience. Keynote calls this a performance "scale." The 0 to 10 Streaming Perspective scale is based on subjective testing of stream viewing experiences in a laboratory environment and assigning a value from worst to best on files in which a controlled condition has been applied. A stream that mimics a high definition television experience is rated a 10 and a 0 is no stream at all.

Keynote was already a successful company whose business model was (and remains) to publish indexes on the performance of 100 high traffic corporate and commercial web sites. Each week the Keynote Index is released and permits companies to compare their web site performance with that of their competitors. In Keynote’s third quarter 2003 fiscal year earnings report, issued on July 22, 2003, the company earned $9.4M from measuring the response time of over 7,600 web pages for approximately 2,300 customers worldwide. Today the measurement agent network includes 1,500 measurement computers in 50 cities taking 14 million measurements per day.

Only a small fraction of the URLs Keynote agents open contain streaming media. Keynote’s foray into streaming was the result of a collaborative project with the then-giant Akamai but leveraged facilities around the world where Keynote had already deployed web page performance agents. At the same time, Streamcheck’s agent network grew (assisted in part by financial support from Mark Cuban) but remained narrowly focused on streaming service providers.