Provisioning for Live Streaming Scale in Real Time at Al Jazeera and YouTube
How do large-volume streamers anticipate, provision for, and capacity-plan for real-time traffic bursts during tentpole and breaking news events? Al Jazeera Senior Streaming Media Architect Dilip Bharadwaj and YouTube Head of Live OTT Engineering Sean McCarthy discuss real-world challenges and preparation and remediation strategies in this discussion with SVTA Subject Matter Expert Bhavesh Upadhyaya at Streaming Media Connect 2026.
Monitoring Audience Growth as News Breaks
Upadhyaya kicks off the conversation by referencing the virtual impossibility of predicting scale and demand when streaming news live as it breaks. “How do you make a determination in real time of how much you need to scale and how much your potential audience may grow?” he asks. “Do you have any tools in place?”
"For this, we use Dataminr," Bharadwaj replies. "Let's suppose, as of now, there is no critical event. All of a sudden, something happens somewhere on the globe, and we can see the number of the subscribers has increased five or ten times. On our end, we have on-prem encoders that are capable of handling this traffic."
But the challenges are different on the CDN side. As a news broadcaster, he continues, "I have encountered a couple of scenarios, like for the Ukraine war, where the traffic has increased all of a sudden. In that case, we use Conviva and Grafana as analytics tools to see how the subscriber [number] grows and how it can be multiplied by four or five or 10. And in that case, we secure the advance bandwidth to make sure that our stream and our channel is capable of scaling up without affecting the end user delivery. So we take all these precautions in advance to ensure that the stream will be seamless in case of any critical event or any important vital news that we are going to receive from the ground."
Taking a Multi-CDN Approach to Capacity Planning
Bharadwaj then directs a question to McCarthy concerning times when those same news-related spikes cause Al Jazeera to send larger-than-expected streams to their YouTube channel.
“Let's suppose we are streaming to YouTube, and all of a sudden we have noticed that the number of viewers watching has increased from 40,000 to 100,000,” he begins. “In that case, we don't have visibility on the YouTube side [as for] how they are scaling up our services for the seamless delivery for the YouTube users. How do you manage this scaling up on the YouTube side? We published the streams, the main and backup RTMP to YouTube ingest. But after that, we don't have any visibility on how the scaling is happening. And our monitoring parameters only see the channel on YouTube, and we can calculate the number of subscribers who are watching our channel concurrently, but how do you manage the scaling?"
McCarthy replies that Bharadwaj's scenario and question mostly concerns "CDN capacity-scaling. We are in a similar situation as most CDN operators and infrastructure providers, which I would suggest is not the fun position to be in, but that's capacity planning, always having headroom, and having to manage that headroom and prepare for these bursty traffic patterns. There's an art and a science to it. To be frank, there's no real secret sauce to scaling the platform other than having more capacity. Now, there is a secret sauce to getting the most density out of a particular box: serve the highest number of bits as fast as possible from a particular server. We have optimisations down to the kernel level to facilitate that, but that's more of a CDN problem. We operate around CDNs, so it becomes our problem."
Drilling down further on the delivery side, McCarthy continues, "When it comes to managing multiple CDNs--which is not something that we typically do at YouTube, [given that] it's all kind of our own monolithic platform--but putting on my Paramount hat from years in the past and just seeing how others in the industry do this, I think that the power of API-driven CDN infrastructure management is not leveraged intelligently enough. So the way that we designed a multi-CDN system in the past to account for exactly this was to have one orchestration layer, and one API that can orchestrate your Fastly, your Akamai, your CloudFront, your CacheFly, whatever it might be, or however many CDNs you have in your arsenal and have feedback loops. By default, we always had two CDNs. Not everyone agrees with that. It doesn't work for everyone's business model, but assuming it does, you always have two CDNs just for reliability and you only introduce a third when it makes sense when you're either trying to mitigate risk or you just need to. And you don't always know that."
Going on to address the role observability plays in this strategy, McCarthy says, "Having your Conviva data or your client-side metrics or some indicator--even total bandwidth across all CDNs--and measuring it against what your reserved capacity is (if you have it), should trigger an API call after a certain threshold is met to your orchestration layer to introduce a new CDN to the mix. Now, there is risk in doing that, but we would mitigate that risk by doing pre-warming of the cache, making sure all of the POPs were online and everything was propagated before introducing that, or slowly staggering it out to end users through either DNS load balancing, manifest manipulation, or content steering."
He concludes, "There are a lot of tools in the kit I think that go under-leveraged for a multi-CDN strategy to solve this bursty traffic pattern problem, but at the end of the day it's infrastructure providers" that are best equipped to address it.
Join us 12-14 May 2026 for more thought leadership, actionable insights, and lively debate at Streaming Media Connect 2026! Registration is open!
Related Articles
A moment's buffering during a game might send fans running. This is where the need for scale, reliability, and quality converge. So, how do providers deliver seamless, captivating experiences for millions of viewers? The answer isn't found in a solo technology or company, but at the intersection of technology and smart collaboration.
17 Mar 2026
How media organisations are tackling the toughest bottlenecks in live video delivery
06 Aug 2025
What are some of the biggest issues encountered when handling interactive streaming at scale? The widespread increase of highly interactive streaming such as auctions, sports betting, and gaming has put pressure on networks while increasing the need to integrate interactivity into more traditional methods of OTT content delivery.
01 Nov 2022