The Metaverse is Becoming Real
— By Frederic Chemaly
What is the Metaverse, exactly?
A metaverse is a fully decentralized, 3D virtual world you can own and develop your virtual land or wander around and interact with what other users have created. You can also meet other people that are exploring it at the same time. The landscape can include anything from static 3D scenery to interactive objects, buildings, art, games, or massive events that attract thousands of users.
It’s basically a better version of Minecraft, or GTA 5, except that it’s decentralized. In these examples, the world is controlled by a single entity that can change the rules as they desire, censor you, or ban you. Or they could even go out of business, in which case you lose everything you’ve built. But with the metaverse, the world belongs to the community.
The LiDAR technology
Imagine a camera, but that takes videos of its surroundings in 3D, instead of giving you a flat image. A LiDAR is a new technology that works on the same principles as radars. It doesn’t need light as it emits its own rays and uses them to recreate a 3D point cloud of its surroundings.
This point cloud can then be used to detect other objects like cars on a highway, pedestrians, road signs, etc. One of the huge advantages of this technology, compared to a regular depth camera, is the ability to fuse data from multiple sources to re-create an environment as big as the application’s needs. You’re restrained to a certain field of view from a camera, you can add two cameras, but then you’ll have two pictures, not a single 3D representation of the combined space.
This technology can be used to detect people in open spaces, like a venue, the airport, or train stations. The managing company can have a people flow monitoring solution, and apply the needed logistics decisions to optimize its operations.
Bridging point cloud to the metaverse
Adding real-time data to a pure digital metaverse could be a game-changer as it becomes linked to real-life events that are happening at the same time. One way of doing it is by integrating a LiDAR’s point cloud, to an existing place in the metaverse, that is an actual copy of the same real-life location. When applied globally, it could bring more engaged consumers, create new revenue streams, and reach younger, digitally native users.
The real issue with the metaverse is that it’s a purely digital world. It can attract the early adopters for now, but to really reach mass adoption, and for companies to use it, we will need a real-time integration with stuff happening in the physical world.
Use Case: Football Coverage
You’re a huge football or basketball fan, but your favorite team is located miles away from you. You can only enjoy the games by broadcasting them on a big screen. That’s exactly where we could use the LiDAR x Metaverse integration to three-dimensionally stream the game. You begin by buying an NFT that grants you access to the game. It could be a season, one game, or lifetime access ticket. Then you hop in the metaverse and go to the stadium. This stadium should be already built by the team selling the tickets. You can then interact with a smart contract that can identify your identity and that you’re holding a ticket that grants you access to the stadium at this time. Once you’re in, you will be able to watch the game as if you’re in the actual stadium, especially if you’re using Headsets and Glasses.
This can be made possible only after equipping the stadium with the right equipment. We would need a set of LiDARs to cover the entire pitch. The players should already have been 3D scanned and integrated into the ecosystem. The algorithm will then link the position and movement of the players from the 3D point cloud, with their scanned body, thus creating the perfect combination of aesthetics and usability.
Manchester City recently announced that they are now investing in the metaverse and started recreating their stadium under a digital version. They did not disclose details about possible live streams as previously stated in the article, they stated that it’s going to be a place where fans can experience new stuff. The possibilities are endless here, once they have recreated the stadium, anything is possible using the right technology.
How it could work
To actually exist on the metaverse, this whole technology needs to be decentralized. It can get pretty complicated since this process involves physical and digital assets that need to be used simultaneously. One way under which this could work is by having the venue owner invest in equipment that will make this possible, in addition to building the digital version of this venue. Now that the environment is ready, we need to stream the data from the sensors, to the metaverse, in a decentralized way. We will need decentralized servers run by the community, holding the content needed to render it in 3D graphics for everyone to enjoy. All of this should happen in real-time to have the best user experience possible and to avoid delays relative to the physical events happening in the venue. This would work like any other IoT solution linked to the blockchain, except that we need to handle large data really quickly.
This article is not intended to provide a deep technical solution of how this could happen, rather than giving a brief introduction to its huge potential. We could think of something like a proof of processing, where users process raw LiDAR data, generate the tracked objects, and synchronize them on the blockchain. This will then be used by a contract that reads from this database and moves objects in the metaverse accordingly. The rewards would then be distributed to those processors from the event’s ticket sale, or whatever was happening that needed this physical-digital twin. A big portion of the reward will need to be allocated to the venue owner, that needs to make up his investments. But the tickets aren’t his only source of income as he can sell collectibles during the event, or even use it to widen his client base.
Lidars are currently operating at a maximum of 20Hz, while cameras can go up to 60Hz for TV broadcast levels. Of course, this frequency is meant to go up with the new technology being developed, but in the meantime, this means the need for additional sensors to track fast objects, like a ball in a football game.
Another issue is the raw data size. Data getting out of the LiDAR is enormous, around 32 mB/s for a good LiDAR, so processing this amount of data in the cloud will be hectic. That’s why companies like Outsight developed the first LiDAR Pre-Processor. It could be one of the most efficient ways to integrate 3D LiDAR Data into cloud and blockchain processing.
Here you can see an example of how they can successfully recreate a 3D representation of an airport in real-time while tracking the passengers. It wouldn’t be impossible to do the same thing, but instead of broadcasting the data to a centralized server, it could be done in a decentralized server and sent to a smart contract to interact with the metaverse accordingly. By doing so we solve the issue of having large data being sent, as we only send tracked objects, which are relatively small and easily sent to a cloud processor. The bitrate depends on the number of objects being tracked but it is less than 1 MB/s for 500 persons per frame.