AI's Streaming Stack: Graphical Representations
AI is a funny thing in that everyone likes to claim they use it in their products, but often during conversations with vendors, I find myself asking, “Is this AI or not?” (While they rarely say it’s not AI, sometimes its implementation seems marginal at best.) Another issue that comes up is that not everyone wants to talk about what exactly they are using AI to accomplish, or they prefer to talk about future AI features rather than the capabilities of their currently available products or services. There’s not much point in trying to write anything worthwhile about vapourware or features that might sound great but can’t be tested or aren’t in use or available to users.
The companies covered here offer some very interesting approaches for using AI to accomplish what I would describe, essentially, as graphical representation in three very different areas:
- AI-powered UX for building live-streaming workflows
- AI interpretation for ad targeting
- AI-generated video clip preview for streaming services
Norsk
Norsk Studio enables customers to build multi-source live-streaming workflows using a graphical drag-and-drop UI or by providing instructions to an AI-powered interface. It is definitely a tool that you will find easier to use the more knowledge you have of building streaming workflows. The demo Norsk provided showed a simple-to-understand UX. There are drag-and-drop functions for enabling a user to architect their workflow in what previously required manual stitching together of different capture, graphics, encoding, and delivery applications.
Norsk Studio sits on top of the company’s own streaming engine and supports multiple capture formats, which can be seen in the left-hand panel of the interface navigation, with input buttons for RTMP, SRT, UDP, and Video Test Card. You can accept default settings or tweak as you see fit. Other building blocks you can add include a source switcher, a monitoring window, HTML overlays, subtitles/captions, and customisable and saveable output encode settings to send channels to a CDN, YouTube, Twitch, etc. “Customers see only what they need to see to manage their production or channel,” says Norsk chief business development officer Dom Robinson.
Norsk Studio
Norsk’s use of AI manifests in two areas. The first is enabling you to utilise AI to create workflows. There is the ability to use AI coding tools like Claude or Cursor via a model context protocol (MCP) to describe what you want, and the system can build it for you. The second allows you to put AI to work monitoring streams, making decisions about what to switch/reconfigure using AI agents.
“Norsk’s technical foundations are entirely built around carrier-class availability,” Robinson says. “With Norsk Manager or your own Kubernetes orchestration, you can achieve five-nines availability.”
Norsk Studio supports most industry-standard ingest formats, ranging from uncompressed video via NDI and 2110 and full-frame capture from SDI/ASI to compressed formats such as H.264, H.265, and other common codecs; transport streams; SRT; RTMP; UDP; SRT with encryption; and secure WebRTC (WHEP) outputs.
Norsk Studio input and output format support
“Broadcast and streaming service operators and enterprise video teams are our biggest customers,” Robinson reports, “with users themselves being streaming engineers, broadcast engineers, system architects, systems integrators, and live event producers.”
Norsk Studio launched in 2023. Previously, the company (as id3as) focused on custom development. Annual pricing is $2,000 per channel, where a channel is a single continuous workflow from source to output, regardless of how many sources (cameras, archived files, etc.) or outputs (CDNs, YouTube, Twitch, etc.) are used.
Norsk Studio runs on-prem or in cloud environments. Potential additional costs include third-party integrations with an LLM. Norsk can provide granular detail around token use and even allow users to dynamically optimise token use on-the-fly.
ThinkAnalytics
Advertisers have forever been trying to figure out how to better target ads. ThinkAnalytics is connecting disparate datasets to solve this problem. The company helps publishers use audience intelligence—aka behavioural context—to get premium CPMs based on better targeting. Its platform, ThinkMediaAI, looks at content, viewer interaction, and ad creative. “We analyze each of those to create a more complete picture for understanding the entirety [of the viewing experience], rather than siloed approaches,” says James Shears, SVP of advertising at ThinkAnalytics. Without behavioural context, ad inventory is much harder to quantify.
ThinkAnalytics’ ThinkMediaAI
Also missing are the requirements for personally identifiable information (PII). “We’ll take the data anonymised and push it back to the publisher or wherever it needs to go, and they can do the matching on their side so they can actually tie it to however it is that they typically do their business,” says Shears. “Most people will take in IP addresses and really create targeting around them. We’re not doing that. We’re trying to help people create viewing profiles that are based around behaviours and based around the content as the driver rather than any kind of offline or online identifier. It’s about curating a marketplace or inventory and allowing them to push that through their workflow.”
What this delivers, Shears contends, is “an opportunity for new ad targeting, inventory packaging, and reporting. We’re trying to provide richer datasets, enrich everything that’s there, and actually answer questions as to why people watch what they do.”
To understand how the company accomplishes this, think about matching mood, subgenre, sentiment, or topic with advertising. “It’s identifying a host of characteristics within the content,” Shears explains. “If you think about the way that advertising is bought and sold today, most of it still happens on the audience level. That’s never going to go away. There’s so much content available that it’s not really an inventory problem. It’s more about, how do you package the inventory up?”
Matching ads to audience is the problem digital marketers have complained about for years: Who is watching, and what exactly do I know about them? “Understanding why I watch specific content—sometimes that’s not going to be as obvious if I’m watching the World Cup and I’m watching shows with my children or I’m watching other shows,” says Shears.
Finding what environment ad creative fits into can be very dynamic within a piece of content. The company’s ThinkAdvertising solution uses AI to analyse for different themes within the content and within the ad creative and can surface a Venn diagram of overlapping interests.
“What most people are trying to determine right now is how to actually effectively work with content signals,” Shears says, “and a lot of that has to do with brand safety first and then other additional pieces after that. How can you take some of the inventory that’s perhaps difficult to sell and make it look more like the inventory that is selling? It’s characteristics. And I’m not saying it’s nefarious in any way. These programmes share the same characteristics. These viewers share the same characteristics.”
By normalising taxonomies across ad categories and subcategories (including IAB standards), ThinkAnalytics applies weighted confidence scoring from AI models trained on global viewing data.
This does sound a bit like describing a black box, but the same is true with many ad tech products that focus on targeting. Customers of
ThinkAdvertising often use ThinkAnalytics’ other products, but the advertising solution within the ThinkMediaAI product group may also be used as a standalone service. ThinkAdvertising has been on the market for 7 years. No public pricing is available.
Media Distillery
Media Distillery’s Search & Discovery Suite is a platform for streaming services and content owners that are looking for a better way to help viewers search for content. On average, according to Martin Prins, Media Distillery’s head of product, viewers “spend 12 to 40 minutes to find something to watch.” Search & Discovery Suite generates “episodic images” to “help people decide more quickly.” These images can also help viewers “discover content that [they] might not know and then quickly select it or prevent them from selecting something that [they] don’t like.”
The application analyses video and not just generates still images, but also identifies a compelling automatic edit of the content. Because these clips are used on platforms that are family-friendly, in addition to not showing spoilers, they can also flag nudity and violence and not include either within the clips.
Most customers want the clips to give the viewer a good idea of what the content is about and also give a good representation of the programme itself. These clips are typically 15–30 seconds long. “When we started training this with our customers, they wanted 40-second clips,” Prins says. “But by showing the first results, they also noticed that [40-second clips are] quite long. By checking more and more content, we saw that, for instance, for an action movie, which has fast pacing, you can use a shorter scene. With a documentary, where the pacing is slower, [you need to] make sure you select a clip that’s slightly longer.”
“Episodic images,” courtesy of Media Distillery’s Search & Discovery Suite
Does Media Distillery use existing metadata or create metadata to generate these clips? According to Prins, “We ask our customers, ‘Can you classify what this type of programme is?’ Then we can tune it a bit better. But it’s not a hard requirement.”
Media Distillery then deploys AI “to understand context and select scenes,” Prins says. “For a comedy show, it should be a comedic clip, and there should be a punch line at the end of a joke. For The Graham Norton Show, you do not want to have a clip just with the host. With a documentary like Planet Earth, if you look at the background, it’s more slowly paced, more narrative-driven. There, you want to provide a slightly longer clip to give the viewer a better idea of what it’s about. They also have to make sure that they’re not generating a new representation for a piece of content.”
As for the quantity of clips delivered, Prins explains, when Media Distillery begins working with a customer, “we deliver approximately 100 clips per day for primetime programmes primarily, or 3,000 per month. We’re already at the [stage of] 45,000 clips being generated automatically, all being used in [our customers’] UI.” Previews, he adds, are created in the original aspect ratio.
Some customers, Prins says, “want us to provide the clips that they can ingest and enter into their CMS, and they will publish them. Other customers just give us the pointers because we already have the asset, and then we just use it as a virtual bookmark so you still have the source video.”
Media Distillery’s auto-generated “spoiler-free previews”
Prins cites a few different playback use cases. Some customers use Media Distillery as a large preview window, while others have an animated hero. For some, it is an auto-start, while others access it following a user interaction. On mobile, it scrolls.
Search & Discovery Suite has been on the market for 1 year and is a managed service. “We have products that you purely run continuously on streams like TV channels, and then you pay for an amount of TV channels per year where we do analysis of content, like finding all of this metadata and enriching it,” Prins says. “There’s another model that’s more à la carte, like what we did for the World Cup, where there were only 104 matches. Then you get more into pricing per asset. Pricing then depends also on what models are needed and which features you use.”