How to Monitor and Troubleshoot Your Live Streaming Workflow
What are some of the typical technical issues with workflows for live streaming at scale, and how can they best be handled? Eric Schumacher-Rasmussen, Chair, Streaming Media Conferences, and CMO, id3as, discusses this topic in detail with a panel of experts from Paramount, Amagi, TAG VS, and Nomad Technologies.
Schumacher-Rasmussen begins by asking Peter Wharton, Chief Strategy & Cloud Officer, TAG Video Systems, “Once you've optimized each stage or element of the workflow, how do you ensure the entire holistic package is working as it should be and stays working as it should be?”
Wharton emphasizes that these systems need a certain amount of sophistication to achieve the best technical efficiency and cost-effectiveness. “We are looking for a level of intelligence in these systems,” he says. “We're looking at something that monitors all these points and actually could go deep enough to look at content in those points and not just say, ‘there's data flowing here.’ Because a lot of the problems you might have would be misrouted signals where it's the wrong thing airing in the wrong place because you're building these workflows where you might be routing away for commercial breaks and not coming back or breakaway and not getting the right returns, so, it is complex, but there are the tools out there.”
Corey Smith, Sr. Director, Advanced Production Technology, CBS Sports Digital, Paramount, highlights the importance of partners needing to be just as invested in a seamless live stream as the organization running the workflow. “If they're like, ‘Hey, go ahead and don't worry about this. Just send us your stream, we'll go ahead and distribute it’…that would be my first red flag. If you're not invested in making sure you're sitting down at the table with me, and we're walking hand-in-hand in this, and you have a Tier 1 event you're trying to pull off, and there's an incredible market and PR value to you as an organization…and it appears that nobody cares on the other side, that is a problem.”
Wharton agrees and says, “When you start working with partners and other companies, you need to get those analytics back into your system and be able to watch them too. And sometimes you even put your own monitoring at the edge to provide yourself the trust that they're actually doing what you think they're doing.”
“Trust and verify,” Smith says.
Adam Miller, CEO, Nomad Technologies, notes that it is essential to plan for failure to some extent to ensure a successful outcome. “I think most people don't follow through the scenarios that could actually happen. Like an encoder stops or an internet stops, or the camera stops. They just assume that their backup plan will just be figured out on the fly. In reality, you kind of have to try it. And most people, they're frantic before these events. So the last thing they want to do is unplug things and see if it really works. Most people are afraid to do that, but you have to stress test those systems. Turn on the monitoring, ‘Hey, look, the flag went off! Look, the monitoring said something's wrong. What do we do now?’ And go through that stuff ahead of time.”
Smith says, “Don't be afraid of unplugging. If your encoders are dual path on an ethernet line, pull one of the lines in a test scenario. Your successful failures should happen in a lab. They shouldn't happen live to your customers.”
Schumacher-Rasmussen says, “It would seem like monitoring is an element of the workflow that would lend itself naturally to AI. Any thoughts on AI? Seems like Brian, you might have had some.”
Brian Ring, Senior Director, News & Sports Solutions, Amagi, says, “I guess two things. So, on monitoring things, it's true. We can have automated monitoring, and we can trigger alarms and events and things like that. But for Tier 1, many customers pay us to have eyes on the glass. I think the other point I was going make around AI, it's not quite this kind of reliability thing, it was the broader definition of quality. I grew up in this business. I remember having that first insight. These guys are talking about quality, they were talking about encoding the picture. I was like, what? Even from my early times, I was like, quality in TV means much more than that, right? And that's true today.”
Ring points out the broader benefits of AI, especially regarding FAST linear. “We want to make sure the ad slots are filled, we want to hit all the seamless changes and so forth and so on,” he says. “And then the redundancies to all the platforms. There are just so many different things. One of those things is putting the ad breaks in the right spot so you can use sort of narrow AI for those purposes. There's lots of great stuff in AI that has been happening and will continue to happen, and you can kind of QA those things [like] typos in the subtitles. And so there's lots of great level narrow use cases [to] make the better video experience.”
Learn more about live streaming workflows at Streaming Media East 2023.
Related Articles
Guaranteeing a satisfying end user experience, whether you're delivering content live or VOD, requires resiliency, ensuring that the stream doesn't break down regardless of the scale, bursts, or other fluctuations in delivery demands. And the challenges are different for live and VOD, with live proving significantly more challenging in most instances. TAG Video Systems' Michael Demb, DAZN's Bob Hannent, and the CDN Alliance's Mark de Jong discuss the key challenges and how to address them in this clip from Streaming Media Connect 2023.
09 Jan 2024