Acoustics in the metaverse

Most of us by now have heard the term ‘metaverse’. From Facebook’s interestingly odd Meta rebranding presented by Mark Zuckerberg, to Second Life’s creator describing metaverses as a ‘three-dimensional internet that is populated with live people’.

However, few really have a grasp on exactly what this means. For starters, nearly everyone describes it as ‘the metaverse’ – when the reality is that there is no singular metaverse in the sense that we talk about ‘the internet’. The term in fact doesn't really refer to any one specific type of technology, but rather a broad shift in how we interact with technology.

From Matthew Ball's 'Metaverse Primer':

“The metaverse is a massively scaled and interoperable network of real-time rendered 3D virtual worlds which can be experienced synchronously and persistently by an effectively unlimited number of users with an individual sense of presence, and with continuity of data, such as identity, history, entitlements, objects, communications, and payments.”

The elements that generally make up the typical understanding of (the) metaverse include:

• Access via virtual reality, augmented reality, PCs, game consoles, and even phones
• Persistent virtual worlds that continue to exist even when you're not ‘playing’ [1]
• A digital economy where users can create, buy, and sell goods
• In more idealistic visions it's interoperable, allowing you to take virtual items like clothes and cars from one platform to another
• Hosting events, trading and socialising in your own virtual real estate
  

To simplify the above though, it’s often described as ‘an evolution of the internet – online spaces where people can socialise, work and play as avatars’. Those spaces are shared and always available; they don't disappear when you've finished using them, like a Zoom call.

The idea behind the deluxe metaverse (the one that currently requires a headset) is an immersive, 360-degree digital world. You'll have your own avatar, which you will be able to design, and ownership of digital assets, the titles of which will likely be recorded on a blockchain. Some foresee the possibility of buying plots of digital land and building online houses, in which you can entertain your friends (or at least their avatars).

So, you can define the look and feel of your virtual world, and many architects and interior designers have already dipped their toes in on the visual aspect [2]. Given the sensory envelopment (‘immersiveness’) that people may want from a metaverse experience, what about how it will sound? This all depends on the level of immersion providers want to offer, which ultimately would be driven by the consumer. Is spatial audio important? What format should this be delivered in? (b-format, binaural, Dolby Surround Sound and Atmos, etc.) Do we need full immersion in the metaverse? Or will simple audio be enough?

Spatial audio combined with VR (what we here at Cundall define as ‘Virtual Acoustic Reality®’ or ‘VAR’, not to be confused with Video Assistant Referee, which came after!) is key to conveying a sense of presence in 3D platforms and has an enormous impact on our ability to consume and digest information. A recent article by Forbes states that “the confusion and inability to locate or identify who's speaking using our ears can lead to a condition often referred to as ‘Zoom Fatigue’.” It's a significant drawback of webinars and virtual events, one that's been difficult to solve until the arrival of VAR.

Some example implementations of VAR:

• Enhanced online presentations – 3D sound can make presentations much more exciting and engaging. With sound originating from different areas in the room, it creates a stronger sense of presence – meaning that you can conduct events remotely without losing engagement.
• Virtual tours – imagine a tour through a museum or historical building with sound effects that help to create a more engaging experience. Or a new product launch where customers get to visualise and experience the product before it’s even launched.
• Online concert – using spatial audio, attendees will be able to experience sound from every angle, from the stage performance to the band session as well as the shouts from the concert crowd.
• Sports events - Manchester City FC are planning to offer metaverse tickets to their matches, where spectators can sit in different seats and view the match from a variety of angles.

All of the above examples require a level of acoustic design to the virtual space in order to create the acoustic environment that the user hears. The geometry of the space, the type of hard or soft finishes, furniture and even the number of virtual people in the space would affect the acoustic environment. This needs to be quantified by acoustic simulations that represent the physics of sound in the real world.

The exciting thing is that this is already here. Cundall originally pioneered VAR back in 2016 and things have developed significantly since then. With consistent increases in real-time processing power, coupled with financial investment from major game producers, game engine developers and acousticians alike, it's only a matter of time until we’re experiencing a real-time Virtual Acoustic Reality® in the metaverse that is indistinguishable from the real world.

[1] Think of The Framework in Agents of SHIELD for you Marvel fans out there.
[2] Examples are Zaha Hadid who are investing heavily in metaverse and Roar who has also invested in ‘virtual real estate’ in Decentraland.