The Building Blocks of the Metaverse
The digital twin of our world mirrored in the metaverse is already being constructed, with companies like Epic Games, Nvidia, and Google as the architects. The ability to scan the real world and translate it digitally is big business. For example, in 2019, Epic Games acquired Quixel, a company with a library of Megascans comprising tens of billions of pixel-precise triangles.
In an article for NAB Amplify, Ball says that the ability to map the real world is becoming a key source of IP. "This dynamic explains why companies such as Epic … and Unity … choose to buy and build up real world scans, rather than build from zero. In the coming years, it's likely we'll see intense competition in the category, with the likes of Nvidia, Autodesk, Facebook, Snap, and Niantic all choosing to build up their databases."
However, the creation of 3D assets still requires highly skilled technicians and artists, presenting a bottleneck for metaverse growth. Khronos Group developers think this might be solved with mass market LiDAR (Light Detection And Ranging) technology. New cellphones (such as the iPhone 12) contain LiDAR, putting this technology in the average user's pocket.
Rumors abound that the iPhone 13 Pro could contain a second-generation LiDAR scanner, which, combined with machine learning algorithms, could turn the stills we take every day into three dimensions almost overnight. "Many experts think 3D snapping is as inevitable as digital photography was in 2000," reports TechRadar.
It's not just still images either. LiDAR could hold the key to user-generated volumetric video. As pointed out by Apple Insider, patents published by Apple in 2021 refer to compressing LiDAR spatial information in video using an encoder, "which could allow the A15 chip to simulate video bokeh based on the LiDAR's depth info, while still shooting high-quality video."
3D media management platforms like Sketchfab and Poly are based on interoperability standards such as glTF and already enable viewing and interactive manipulation of 3D models via a web browser. "LiDAR technology … now allows anybody with the latest iPhone, to mass render the physical world, translate it into machine readable 3D models and convert them into tradable NFTs which could be uploaded into open virtual worlds very quickly populating them with avatars, wearables, furniture, and even whole buildings and streets," writes Jamie Burke, CEO and founder of venture capital firm Outlier Ventures in a website post.
Burke is leading a parallel attempt to lay the groundwork for the open metaverse. Outlier Ventures invests in cryptocurrency, blockchain, and startups in the emerging Web 3 stack. The company states, "The Internet is being fundamentally restructured from the bottom up by a convergence of decentralised technologies to form a new data economy. Where the last 20 years have been dominated by digital globalization of extractive and increasingly anti-social platforms and ‘the cloud,' the next 20 will be defined by a redistribution of value to networks and an unbundling of platform monopolies. From the sovereignty of the platform to the sovereignty of the individual user."
Outlier Ventures wants to accelerate that mission by promoting what it calls The Open Metaverse OS, a shared and open OS building on the success of decentralized protocols like NFTs. It explicitly ties digital currencies and the "on-chain transferability of assets through NFTs" to the core of the emerging metaverse economy.
Ball and Navok agree with this approach. They say that blockchain is considered the one major interchange technology that "retains most of the values and benefits of an open standard, and also looks likely to thrive in the Metaverse."
A synthetic camera image and corresponding "ground-truth" data (Image courtesy of Nvidia)
An Open OS for the Metaverse
Among the framework technologies identified by Burke are LiDAR, Pixar's USD, and Nvidia's Omniverse. He thinks they can be better monetized in global and open markets than any one closed platform and that "increasingly the world of Web 3 and crypto is … converging [and] with new environments like gaming and VR there is a generational shift away from Web 2 platforms. … The Open Metaverse OS is best understood as an evolving collection of highly composable technologies that will increasingly, but selectively, be used to make aspects of … the Metaverse progressively more open."
Other platforms are hoping for similar social outcomes. OWAKE is described as a "Real-time Moment Sharing System that enables communication between people and people, people and machines, and machines and machines." It's developed by Kronosa, whose mission is "to build The Sustainable Human Society with a next generation internet for humans to use both the virtual and the real world as their residence and workplace." In addition, the Open Metaverse Interoperability Group is "focused on bridging virtual worlds by designing and promoting protocols for identity, social graphs, inventory, and more."
Nvidia Drive Mapping delivers scalable, highdefinition mapping and localization capabilities for autonomous vehicles. (Image courtesy of Nvidia)
Hardware in the Metaverse
Metaverse or not, the sheer amount of data heading across networks demands more than a software response. Optimizing streaming bandwidth, latency, and reliability is deemed essential. "If we want to interact in a large, real-time, shared, and persistent virtual environment, we will need to receive a superabundance of cloud-streamed data," say Ball and Navok in another article. "Cloud data streaming is also essential if we want to seamlessly jump between different virtual worlds."
Latency is a current bugbear in live sports, of course, but the general feeling is that by optimizing technologies like LL-HLS and adaptive bitrate, the delay from capture to screen can be reduced to a broadcast-like 5 seconds. That may be fine for the NFL or Premier League, but it's nowhere near good enough for esports, gambling, or online multiplayer games, let alone the instant social interaction on which the metaverse is predicated. "Slight facial movements are incredibly important to human conversation—and we're incredibly sensitive to slight mistakes and synchronization issues (hence the uncanny valley problem in CGI)," say Ball and Navok. It doesn't matter how powerful your device is, they argue, if it can't receive all of the information it needs in a timely fashion. As a result, they say, the availability and development of computing power will constrain and define the metaverse.
Compute, 5G, and Edge for the Metaverse
The combined force of 5G and the build-out of infrastructure at the edge (in data centers or cellphones) are considered key. Aside from the regular telco developments in this area, there are fresh innovations targeting the metaverse. Los Angeles-based start-up LionShare Media is launching the THIN/AIR platform for premium entertainment and immersive media experiences. According to its website, this is a direct-to-consumer, cloud-native, 5G decentralized media distribution platform that empowers creators with their own media channels called Projects. Projects, it explains, are spatial 3D Web apps that feature a "hyper-cube UI/UX design that leapfrogs the experience of OTT [MPEG-5] video, social media, and livestream sites."
But even if we improve the computing power of consumer devices, move more enterprise computing power closer to consumers, and build out more centralized infrastructure, we're still likely to fall short.
Ball and Navok's idea is that a form of peer-to-peer networking will take place in which the available compute power of every local PC and device will be used to sate demand. Owners will be paid for the use of their CPU and GPU compute power. They think this is possible because the transaction will be conducted by blockchain. "Every computer, no matter how small, would be designed to always be auctioning off any spare cycles. Billions of dynamically arrayed processors will power the deep compute cycles of even the largest industrial customers and provide the ultimate and infinite computing mesh that enables the Metaverse."
Related Articles
Some say NFTs (non-fungible tokens) are a novelty, a status symbol for the crypto rich, and a bubble that's bound to burst. Others argue that NFTs will fundamentally change business models in the creative industries. It's early days, but there's plenty of momentum behind NFTs and blockchain. Here's what you need to know.
13 Dec 2021
Interest in VR has waned, even as the technology has improved. But combine it with augmented reality and mixed reality to form extended reality (XR), and things get interesting.
09 Jul 2021