How to Jump-Start Your Multi-CDN Strategy and Deliver Every Time

Some CDNs offer their own health-check APIs to measure their throughput and provide feedback to systems or users. We’ve found those really valuable, particularly as multi-CDN has increased demand for these features across the industry. Generally, these APIs offer good, quick insights.

The Solutions

Figure 2 shows a sampling of current providers of solutions for multi-CDN delivery. These products and services represent a spectrum of options and approaches that we’ll explore here.

Figure 2

CONVIVA

Conviva’s Precision Delivery Intelligence system API (Figure 3) is all about data. Conviva does not offer a CDN solution that’s plug and play in any sense, but it provides multi-CDN monitoring metrics that you can take action on. The most common approach is to plug in the data Conviva’s Precision API provides to make up-front decisions about which CDNs to assign to which users. Integrate this into your content management system, and feed the data to your users, and it will adjust on-the-fly to give them, in theory, the best initial video experience they could have.

Figure 3

Conviva Precision data can also be used midstream by the client- or server-based solutions to move traffic as issues arise. There is a powerful logic layer to determine the best solution for specific users based on their IP address, so there’s potentially high value in applying it to any client-specific solution if you are already using Conviva. Generally speaking, from time of issue to Precision data indicating that a change should take place, it is potentially quicker than from 1 to about 3 minutes maximum delay, according to Conviva.

You can also use Conviva for midstream adjustments. Let’s say you’re starting to see an issue, and you want to shift 30% of your users over to CDN X. You can use the Conviva API data to help drive that as well. But once again, you have to have a system in place that’s going to then take those actions.

CITRIX/CEDEXIS

Citrix/Cedexis offers one of the more robust systems available. It also can be one of the most expensive. On a daily basis, the Cedexis system collects information from more than 50,000 different networks and 130 clouds and acquires 14 billion ROM datapoints. It has node-check systems all over, and if you use it, it requires that you put a little bit of client-side code in it that’s also going to be doing node checks, pulling some data, and checking performance. Citrix’s Radar Live system (Figure 4) also aggregates this data to give you a live view of the internet at any time that will show you where the greatest congestion and other issues are happening.

Figure 4

The potential downside to Cedexis is if the node-check payloads accurately reflect the actual performance on stream chunks due to potential CDN false optimizations or just the difference in payload size and cache systems. The other consideration with any DNS-based solution is if you are trading one point of failure for a different one. What is your disaster recovery plan if its DNS system fails?

The benefit of the Cedexis solution is that once you get it configured and route your DNS tracker through it, it becomes a mostly turnkey solution to integrate. There is a tiny bit of client-side code for validation, but it can provide a powerful system in place, and you don’t have to do very much to get it up and running. And it’s driven by a lot of data from across the network.

Some of the biggest players out there doing multi-CDN are definitely using Citrix/Cedexis, and some of them aren’t. But it is a great option for a bigger approach.

DLVR

DLVR, like Citrix/Cedexis, is a DNS-based solution. It differs from Citrix/Cedexis in that instead of just routing the individual request, it actually modifies the manifest, and it has a trademark responsive manifest for each video request made to a DLVR host name. The modified manifest has the start-video segments that are actually going to be delivered from DLVR. So instead of being delivered from your CDN, the first one to three segments of each video are sent from DLVR and used as one of its main key metric monitoring points. DLVR maintains that it can deliver as well as any CDN for small volumes.

Customers we know who have used DLVR have experienced fairly good delivery. Aside from polling, DLVR actually checks a lot of things going on within the CDN as requests are flowing through the system and looks at the manifest requests to also determine if delays or buffer issues are occurring. DVLV has built some interpretive and prescriptive logic around what it believes is happening at the individual clients, and it aggregates that based on requests that clients make. Thus, it can say, “If we know this is a 6-second segment, and they’re now experiencing a greater delay between requests, we know that buffering is happening.”

The downside of DLVR is that some clients don’t like the fact that they end up picking up more of the traffic. In some systems, that doesn’t deliver the performance that it does in others, especially if you’re on a Comcast network. Is that really going to be the most optimal route, getting content delivered from DLVR and AWS versus the actual Comcast CDN? That said, DLVR tries not to do that too much, and limitations aside, it can offer an interesting control.

DLVR tries to provide a cost-neutral approach. It charges for the smaller segments it delivers off its network. But in theory, that cost is offset because those segments aren’t delivered by the CDN. The downside is that DLVR can end up delivering more of that traffic. We’ve been told that as much as an eighth or even a quarter of the traffic can be routed through DLVR.

DLVR’s competitive advantage over tag-based measurement like Citrix/Cedexis is that no client-side code or tags are required (Figure 5). There’s nothing you have to put in the client for DLVR. Thus, its message is, “Set this up. Point us over. We do the configuration, and you now have multi-CDN.”

Figure 5

DLVR also offers robust CDN decisioning. It can set up your business rules and real end-user measurements of video streams based on your actual video streams. Instead of trying to use non-video, tag-based measurement, DLVR uses your actual video segments to make decisions. Measuring on the video that’s actually going to be delivered does provide some potential extra value.

STREAMROOT

Streamroot Compass (Figure 6) is a client-side solution that intercepts and adjusts requests on-the-fly for individual segments and swaps them there. The advantage of this approach is that you don’t have to change your overbearing routing system, and it can still do segment-based swaps. Compass gives you good control over the routing, and the implementation is actually pretty simple. We implemented this solution recently, and it was up and running within an hour or two.

Figure 6

We’ve worked with Streamroot to help handle some very complex solutions for the future in regard to dynamic ad insertion and multi-token authentication as they come into play. Right now, it has really solid support on all the web-based platforms, and it has support moving forward on some of the native platforms. It’s not on all platforms yet, but it’s working toward that.

PEER5

Peer5’s client-side concept (Figure 7) is similar to Streamroot’s. It’s on fewer platforms at present—just web for now.

Figure 7

If you have no CDN yet and don’t have a lot going on too far into your player, Peer5 can offer some solutions to take care of it all for you. If you’re just starting out with no CDN agreements, and you don’t want to line them up yourself, Peer5 is a good solution. If you’re further along, it’s probably not the right choice for you.

These solutions are not the only options out there, but they should give you an idea of the range of what’s available. Citrix/Cedexis and DLVR may be the top two for the DNS-based approach. Streamroot is a great choice for client-side, as long as you’re not looking to hit all platforms out of the box.

Implementation Strategy

The challenges you’ll face with any multi-CDN solution depend on your implementation strategy, which is defined by your answers to a few key questions. What are the supported platforms that you intend to target? Are you looking for stream-based or node/tag-based measurement? I would definitely recommend, if possible, stream-based measurement, but tag-based can also provide a viable solution.

CDN platform-specific capabilities and support are another key concern. Do you want to also support “in-network” CDNs? Some customers may require that, in addition to Akamai, Limelight Networks, or CenturyLink, you also support routing users who are on a specific network to a specific private CDN like Comcast or Verizon. This is becoming more popular with some cellular or cable providers, as they can offer optimal performance for users who are solely on their networks. If you’re at home and pay for Comcast internet, and you get content off of the Comcast network, you are going to probably outpace everyone else from a performance standpoint, because Comcast controls the end to end down to the last mile. But that works only if you’re a Comcast customer, so sometimes we have to check and say, “If this user, wherever he or she is, is on the Comcast network, let’s prioritize the Comcast network before we try to send stuff over to Akamai or CenturyLink.”

These network CDNs can add in interesting complexities. Something analogous happens at the mobile level. Verizon also has its own CDN, called Verizon Digital Media Services (VDMS). If users are on a Verizon device and they’re trying to access content, getting it from the Verizon network will, in theory, give them a good performance benefit. If you’re going with a multi-CDN strategy and you are looking to leverage these types of networks or have agreements in place to leverage them, talk to the providers or whoever you’re going to work with before building out a solution to deal with that.

Dynamic ad insertion is another important consideration. When you have to deliver an ad in a multi-CDN implementation, you need to ask, “Which CDN do I load that from? Do you try to load that from the CDN your user is already on, and if so, how do we make adjustments for that?” That’s something a lot of the systems overlook. Working through this can be a big challenge. If you’re using a DNS-based solution, it should work out fine, but even that can get complicated based on your content origin and your ad delivery and decisioning configuration. You’ll also need to make sure your ad-decisioning engines are playing well with it; at a minimum, you will want to likely return ad content pointers for each CDN within your ad calls. If you’re using a client-side solution, you’ll need to handle ad insertion and selection more carefully if you want sticky CDNs for ad content. Keep that in mind for the metrics you gather as well. Server-side ad solutions (SSAI) are also working to catch up a bit in the multi-CDN world, especially for non-DNS-based solutions—although, once again, a lot of this also comes down to having all the data flow from ad services to players and other providers that need the right multi-CDN information.

Other key factors in your multi-CDN implementation strategy include security, access, and token authentication. Will you have to get new tokens upon switching, or will you do that up front? We’ve found that our customers didn’t want to have unified tokens across all CDNs. Even if Citrix/Cedexis doesn’t go so far as to require it, it pushes hard for it on its customers. Essentially, its approach is, “If you’re using token auth, you have to have the exact same tokens work across your different CDNs.” That does make your life a heck of a lot easier.

However, there are challenges with that approach for CDN configuration, whether CDNs use long tokens for authentication or tokens in the path. By and large, our customers like having different tokens, and this approach can offer an additional layer of security. You’ll have to consider if it’s worth the headache.

If you ever do a hard push of all your users—for example, if you’re seeing an issue on CDN 1, and your solution is to move all your users to CDN 2—be careful. If you’re working on big events, moving a very large number of users too quickly to an alternative CDN can have negative implications. You can cause a stampede, which would potentially give you far degraded service on that CDN while it tries to warm up all the regional caches and their points of presence (PoPs).

One of the things we built in years ago for one of our players is enabling the client to do redirects. You could design it to execute the redirect immediately, although we’ve also built in an option to specify a duration. So, you could say, “Over the next 5 or 10 minutes, start randomly moving people over.” That’s easy logic that you can do on the client. You simply have it generate a number between 1 and X seconds and set a time-out. Wait until that executes, and then do your move over. That can be a good way to delay and spread things out a bit.

Choosing the Right Multi-CDN Solution for Your Use Case

There are a number of options out there for moving relatively quickly into multi-CDN delivery. Which one you choose will be determined by what your use case is, what your tolerance is for how much implementation you have to do, what your budget is, and how you want to go about it.

You can also work toward building a system. We’ve built some in the past as well. It just depends on what target platforms you’re trying to hit and so forth. For the systems that we built out, we used some of the manifest manipulation at the client level. We’ve built systems across all the primary platforms, including web, iOS, TV iOS, Android, and Fire TV. There are a number of good options out there; which one you choose depends on where you want to go with it.

[This article appears in the September 2019 issue of Streaming Media Magazine as "How to Jump-Start Your Multi-CDN Strategy."]