Transparent Internet Caching: An Industry Perspective
In the past two years, I've encountered a lot of confusion about the differences between Transparent Internet Caching (TIC) and Content Delivery Networks (CDN). This article dives into the different applications, features, technologies, legal issues and purposes.
There was a lot of debate about TIC at the 2011 Content Delivery Summit in New York. Dan Rayburn wrote some good articles here and here, and you can watch a video of the transparent caching panel from the Content Delivery Summit at the end of this article.
These are the main differences between TIC and CDNs:
- A CDN lets you store, distribute, and deliver on-net (walled garden) premium content, whether this is for IPTV, OTT, web, or mobile delivery. There's no use for a transparent caching platform for this purpose.
- A CDN lets you store, distribute and deliver off-net premium content. You can monetize the CDN by renting its capacity to third-party content owners. There's no use for a transparent caching platform for this purpose.
- Both a CDN and transparent caching let you offload traffic from the internet. By deploying edge nodes in strategical locations, the telco can offload traffic not just from their peering and transit links, but also from their backbones and metro networks. More modern solutions even let you deploy tiny edge nodes at the outer edges of the network, mobile base stations for instance.
The technical differences between TIC and CDN aside, it is above all a matter of legal and commercial relationships.
TIC: Save Money
A Transparent Caching platform can be deployed and used without end users and content publishers being aware of this, without the need for any commercial or legal agreement. TIC is basically an intelligent network facility that allows the telco to offload traffic. The more transparent the TIC has been designed, the less it affects communication between end users and content publishers and the less it interferes with the content publishers' business.
CDN: Save Money, Make Money
A CDN requires a relationship with the content publisher. CDNs don't just cache content from the web, they only store, distribute and deliver known content. In most cases, the CDN is offered as a commercial, professional service, with service levels and contracts, and more and more with QoS guarantees: guaranteed capacity, guaranteed performance, guaranteed quality. Compared to TIC, which is purely a technical game, a CDN requires sales and marketing.
Traffic Savings
Basic inline TIC implementations can offer up to 30%-40% traffic savings. More advanced TIC implementations with central intelligence and decentralized edge nodes claim to save even 60-70% traffic, I haven't seen facts that prove these claims though. A modern CDN with a central intelligence service and decentralized edge nodes can offload the same high percentages.
TIC or CDN? No: TIC and CDN
If you are purely looking into offloading traffic in the short term, then TIC is an interesting approach. But that is not a long-term strategy. Most telcos I've talked to say they need a dual-strategy: they want to offer their own walled garden content via a CDN that can guarantee them performance, quality and control. They want to monetize their network and their CDN to offer the same performance, quality and control to third parties. And to catch and offload a large chunk of the rest of the web's traffic they want to implement a TIC alongside. Some telcos want to integrate some additional technologies as well: DPR, realtime transcoding etc.
I can't stress enough that CDNs and TIC serve different purposes and that telcos needs BOTH.
Content owners don't like TIC
Funny that no one really asked content publishers what their opinion is about TIC. We did. We happen to own a very popular streaming video CDN in Europe called StreamZilla. We know a lot of content publishers so I asked around. In general, they don't like TIC.
Telefonica did the same, as they shared with us on CDN World Forum. They reached out to the top video traffic generators on their network if they would prefer free TIC offloading or a paid CDN service. All of them preferred to pay for a CDN service.
Content publishers use CDNs to get their content distributed in a controlled way. They pay the CDN to deliver their content in a controlled way: the edge servers (or better: delivery nodes) are controlled by the CDN, no one else. Todays global CDNs can only guarantee best effort: global CDNs don't control the last miles so they have no control over what is happening beyond their network. Content publishers don't want anyone beyond their CDN to touch their content: no caching, no shaping, no DPI, no transcoding.
Legal issues
Telcos tend to look at TIC, CDNs, DPI, etc from a technical perspective. What they are not realizing enough sometimes is that they are part of a larger value chain. And that traffic isn't just about terabytes but about actual content. Important content. Valuable content. Someone else' s business. Content publishers have contractual obligations towards content owners. Let's say that a studio licenses a broadcaster to publish a film on the internet. In many of these licensing contracts it is strictly forbidden to change a single bit of the pre-encoded content. In many of these contracts it is mandatory to prevent third-party caching beyond the trusted CDN.
TIC breaks these contracts. In some countries telcos are allowed to cache internet content, in fair use. Fair use means that if content is offered via HTTP and does not state ‘no-cache' the content may be cached. But I've heard a lot of telcos and vendors stating that they ignore the no-cache statement from global CDNs, YouTube, and others to improve their cache hit ratio. Which is illegal.
I've also heard telcos and TIC vendors talk about caching non-HTTP traffic to further increase the effectiveness of the TIC offloading. RTMP traffic for instance. Many content owners force content publishers to use non-HTTP delivery protocols to prevent caching. Since RTMP and RTSP were never designed to be cached, you can't state ‘no-cache'. Caching non-HTTP traffic is illegal.
Caching content is one thing, but actually touching content is another. Most content publishers are not aware of this, but more and more telcos (especially in the mobile space) are using DPI to block, frustrate, or accelerate content. Other mobile telcos go a step further: they transcode content in realtime to lower bit rates. I wouldn't be surprised if governments will enforce net neutrality rules and make this illegal, if it isn't already illegal. I wouldn't be surprised if content publishers will sue telcos for this: these content publishers need to guarantee content quality contractually towards the content licensors.
On a side note: many TIC vendors claim that over 90% of the Internet's traffic is going to HTTP based. I doubt that. We don't see HTTP adaptive streaming cannibalize RTMP, MMS and RTSP streams. Virtually all content publishers I've talked to have no intention to switch over to HTTP adaptive streaming. It is added to the mix to support additional clients, but it is not a replacement. Content publishers need reach. Maximum reach can only be achieved if they support multiple video clients by using multiple streaming technologies with their specific protocols. Going 100% HTTP adaptive would kill their reach.
QoS and Control
There's another argument that I've heard from content publishers: quality and control. For most content publishers, web video has been a niche gadget so far. But this market is about to explode and needs to professionalize. Consumers demand the same QoS (not just QoE!) for OTT content as what they are used to get from digital cable or IPTV. Instant on video. HD quality. No buffering, no delay. Global CDNs can't offer this because they don't own the pipes to the end users. Global CDNs can have the best infrastructure, with 99.99999% uptime, once they dump their traffic over the hedge into a telcos network, it's best effort.
TIC does not improve this either. TIC makes it worse because the trusted CDN doesn't deliver to the end user anymore. There's a man in the middle. TIC offers no capacity guarantee. No quality guarantee, no QoS, no SLA, no contract. QoS and control is the main reason why content publishers are very interested to work together with telcos who deploy a CDN. It's time to become friends. Telcos who 'secretly fiddle around with our content' are not making friends.
Anti-Caching
Most content publishers are not aware of all these TIC initiatives. Others are worried. Content publishers don't trust telcos to play nice and are afraid that their ‘no-cache' statements are not respected. We have been asked to implement extra anti-caching mechanisms in our CDN software, a feature which is currently in R&D. These mechanisms are intended to frustrate non-trusted caches beyond the CDN's own trusted caches. This feature gives content publishers back the control they need over THEIR (licensed) content. And it gives them a stronger legal position if third party caches are deliberately trying to bypass protection mechanisms.
>