Ten Questions: Signing a Service Level Agreement

Select—Review the ratings of your top three CDNs, review benchmarks, and consult with your colleagues.

Here are some suggestions for scorecard categories ranking the CDN:

—Size of audience
—Location of audience—Choose a random sample that reflects your actual target audience
—Method of end-user evaluation (feedback form or usersurvey)
—Content of webcast to be used for the pilot (i.e., audio only, video, slides, polls, etc. )
—ROI expectations—Are you saving money? What traditional forms of communication are you projecting savings from by using streaming media? Set metrics that will measure this. For example, if conference call replay cost is $1,000 per month, and presenting an on-demand, audio-only file is $3 per month, this will save your company $11,964 per year for a monthly call.
—Server-side metrics—In a controlled pilot, you will be able to better determine if the end-user evaluations and the metrics on performance and user behavior make sense. Test with multiple users from the same location from behind a firewall and see if the unique hits match up on the server-side reports. They often do not, so here is your chance to ensure you are getting metrics that will help determine ROI or not.

Create a scorecard that reflects all the criteria that is important to the success of your streaming media initiative.

Once we sign an SLA, how do we know it’s working and how will we enforce it?
Formal review meetings should happen at least quarterly, along with an annual review of the rate card. Monthly reviews for the first year of a new SLA may be required, especially if service delivery has not met the criteria of the SLA. The CDN service environment is complex and is always undergoing significant changes, so monitoring is critical.

Prepare to negotiate changes if any significant circumstances arise from changing business or service needs, significant variations from agreed-upon service standards, and unanticipated events.

The specifics of how these reviews will take place—including key contacts, methods of notification, and how metrics on performance and usage will be communicated—should be articulated in the SLA.

I have seen enterprise SLA relationship managers neglect their roles even after the most rigorous SLA negotiation processes. This results in CDNs making outrageous guarantees based on the idea that no one will ever be able to enforce them. The complexity of interpreting the deliverables of an SLA can be discouraging, and remediation is often not sought. This lack of expectation for fulfillment can render the whole process a ridiculous exercise indeed.