How WePlay Studios is Transforming the Future of Live Event Storytelling

By 2032, the esports market is expected to grow to $9.29 billion, bolstered by a global player count and fan following that both continue to climb. The 2023 League of Legends World Championship Tournament alone drew in an audience of over 6.4 million. Con

Grass Valley, California(02 May 2024)

By 2032, the esports market is expected to grow to $9.29 billion, bolstered by a global player count and fan following that both continue to climb. The 2023 League of Legends World Championship Tournament alone drew in an audience of over 6.4 million. Considering esports’ massive fan following, many of whom tune in via dedicated live streams, the pressure to deliver high-end live productions that align with fan expectations is real for companies like WePlay Studios. With dual headquarters in Kyiv, Ukraine, and Los Angeles, California, the award-winning content production company fuses gaming, technology from companies like AJA Video Systems, and storytelling to craft second-to-none gaming shows and esports tournament experiences for top-rated titles like Dota 2, CS:GO, Valorant, and Rocket League. But, as WePlay has uncovered, there’s also a huge demand for its talents on live event productions like The VTuber Awards, which its team completed a full virtual production for last year.

Hosted by Filian in partnership with talent management agency Mythic Talent, the five-hour event celebrated the top virtual creators online. WePlay Studios helped bring the event to audiences worldwide with a virtual broadcast by blending physical production facilities and equipment with extensive virtual production engineering and design. “Storytelling and technological innovation drive every show we do, and we pride ourselves on creating iconic content that leaves a lasting viewer impression; the VTuber Awards were no exception,” shared Head of Virtual Production Aleksii Gutiantov. “While we’d previously incorporated AR into live esports productions, this show marked our first foray into a fully virtual event managed with virtual cues; it’s the most challenging technological endeavor we’ve ever taken on.”

Linking physical and virtual environments

To successfully pull off the event, Gutiantov managed and coordinated the production in Los Angeles remotely from his laptop from Europe, using intercom communication with over 16 team members and orchestrating eight days of non-stop pre-production to deliver the broadcast. His team first created a real-time rendering of an entirely virtual Filian to incorporate into the live production using motion capture (mocap) technology. They tapped 20 witness cameras for comprehensive, full-body performance capture, including precise finger movements, and combined it with additional technology to stream facial mocap data. 

The live event stream included a vast virtual arena, but Filian's character was located on a smaller stage, encircled by a digitally reconstructed version of WePlay's physical LA arena. To ensure every physical pan, tilt, and focus pull translated directly into the virtual render environment, WePlay Studios’ camera operators managed three cameras that were synced to virtual cameras. Camera operators in the practical/physical set were then able to switch among various angles within the virtual stadium using iPads connected to virtual cameras, creating the illusion of using a dozen cameras instead of three.

To make the production look more authentic, WePlay Studios connected the physical stage lights to the corresponding virtual lights, which allowed the team to manipulate the virtual stadium's lighting environment through the activation of a real environment via a lighting control console. Video playback was also integrated into the virtual world, with software for live event visuals connected to the virtual venue used to launch and control the graphics displayed on the virtual stage's screens. AJA KONA 5 video I/O boards played a crucial role in the 12G-SDI signal chain, and the final SDI feed was forwarded to an AJA KUMO 3232-12G video router for availability across the entire broadcast pipeline.

“Our KONA 5 cards were instrumental in allowing us to receive 12G-SDI signals, integrate them into an Unreal Engine 5 environment, and composite the final in SDI. It’s the best product on the market,” explained Gutiantov. “And, our KUMO routers let us build infrastructure for large remote and on-site productions like this one and manage everything from a single, convenient web interface thousands of kilometers away. We also love that we can save pre-programmed salvo routing configurations for SDI signals, and we never have to worry about them going down; I've been working with them since 2017 on various projects, and they've never failed me.”

 Leveraging KONA 5 with Unreal Engine

KONA 5 enabled WePlay Studios’ team to leverage the power of Unreal Engine to create a comprehensive virtual production hub capable of handling 12G-SDI workflows. This allowed them to fully harness the potential of AR technology, from camera tracking to motion capture and data-driven graphics, while ensuring flawless live virtual production broadcasts without any technical mishaps in compositing. It also allowed them to produce UltraHD fill and key signals from one card in all known formats, using Pixotope as a keyer for 4K with the failover features known from FHD workflows.

“The KONA 5 user interface is simple enough to understand and control, even amidst the pressures of live production, and we love that we can preview last-minute changes in real time. It also offers up to four reconfigurable I/Os, from SD to 4K, along with support for AES/EBU, LTC, RS-422/GPI, which is key for transforming video from interlaced to progressive formats if we are working in Saudi Arabia or China,” added Gutiantov. “KONA 5 really helps accelerate operations on projects like this, which requires a lot of compute power for motion-adaptive deinterlacing. Furthermore, the card’s multi-channel hardware processing accelerated compute-intensive operations so that we could combine multiple video sources into a single output in Unreal Engine 5, up/down/cross-scale, and mix/composite for all resolutions. These processes are essential for handling video content of any resolution, ensuring that the final output meets the broadcast quality standards.”

 Meeting flexible recording needs

In addition to KONA 5 and KUMO, WePlay also leveraged a cluster of AJA Ki Pro Ultra 12G recorders to meet the high-quality recording standards demanded by the project without any interruptions or compromises. “The flexibility and reliability of our Ki Pro Ultra 12G recorders were essential,” Aleksii Gutiantov added. “The devices are indispensable; they allow us to support multi-channel HD recording or single-channel UltraHD, and we can swap out recording media on the fly, which is convenient and reliable, especially for long-format live broadcasts and when clients require high-bitrate UltraHD materials for post. Plus, they continue to run smoothly for more than 12 consecutive hours.” 

Establishing a preview infrastructure

Due to the unique setup of WePlay Studios’ Los Angeles facility, the team developed a preview infrastructure comprising a series of Mini-Converters to facilitate 12G-SDI signal down conversion and forward 3G-SDI signals to their AJA KUMO video router. Using AJA HD5DA SDI distribution amplifiers, the team was then able to spread preview signals across all arena monitors for more straightforward management of all preview signals. The setup, which also used salvo routing configurations for SDI signals regardless of the data source's nature, enabled precise control over the view of the production that WePlay Studio provided to its partners, talent, camera operators, motion capture team, and the entire production crew at any moment. AJA ROI-DP DisplayPort to SDI Mini-Converters proved a key part of this preview infrastructure design, allowing the team to duplicate computer monitors into the broadcast pipeline to manage conversion with region-of-interest scaling.

The intricate setup behind WePlay Studios’ VTuber Awards production is just one of many examples of how virtual production technology is transforming modern entertainment experiences. According to Aleksii Gutiantov, the level of interactivity it brings to the table unlocks exciting new possibilities for live entertainment genres, blurring the lines between viewers and the virtual worlds we create. He noted, “WePlay is not just staying within the confines of the gaming industry; we're branching out to music and broader entertainment directions. We're currently in the early stages of planning and discussions for projects that straddle these new frontiers.”

WePlay Studios plans to open up a new virtual production studio in Los Angeles this year that will feature a screen area larger than 2,500 sq ft with a 1.8mm pitch and the first-ever Pantone-certified LED color pipeline, utilizing advanced flip-chip technology — which will be dedicated to film and entertainment projects beyond gaming and esports. Gutiantov concluded, “There’s even more exciting news on the horizon from WePlay Studios, so stay tuned for more to come.”