Meetings in Your Own Metaverse: Enter the Metaverse

VR headsets, and the metaverse, are something that is always presented to us as the future - Meta invested $13.7 billion in it last year! But if it is the future, what will we actually use it for? One popular use case we are always told about is the ability to meet your friends and colleagues in 3D meeting rooms and spaces.

Meeting Software

Recently, I have been evaluating the feasibility of scanning a room and meeting in this in VR. In part 1, I explored the various scanning methods available for generating a 3D scan of a room and decided on Polycam as my scanning app. Now that we have our room scans, the next step in our journey to travel to our meeting room in VR is to decide on which meeting software to use. There are various meeting software available that allow this, each with its pros and cons.

Why am I doing this?

Since the conceptualisation of the metaverse, a virtual world you can visit by putting on a VR headset, there has been a lot of hype surrounding VR as a future of work. Waterstons works with a lot of clients who are intrigued by this field, such as manufacturers, architects, and engineers, and who are also interested in digital twins and 3D models. I set out to investigate the feasibility of this. How easy would it be to achieve? Which tools are available? What is the experience like?

Why is meeting in the metaverse a good thing?

• It can give a more realistic feeling of speaking face-to-face with others, which can often be lost when using video calls.
• It allows for more interactivity than video calls, with the ability to interact with your environment.
• It’s really fun!!!

What meeting software is available?

Horizon Workrooms

*A Meta Horizons Workroom taken from their website.

Horizon Workrooms is free and offers pre-built meeting rooms and allows cool interactions such as the ability to write on whiteboards for brainstorming, and the ability to import your laptop and use this in VR as if you were sitting at your desk. Unfortunately, there doesn’t seem to be a direct way of importing our meeting room – we can customise and add logos/banners to current meeting spaces, which means I sadly cannot use Horizons to import my 3D models.

MeetInVR

This is similar to Meta Horizons and is free for up to 10 users. They charge you to import your models, which requires you to contact meetinVR directly for them to do this for you. I am looking for an easy way to import my models into VR myself and travel there straight away, so I will not be using this platform.

Mozilla Hubs

When you visit pre-defined rooms created by Mozilla hubs, there is instant excitement about the possibilities of this platform. For example, Polycam was able to to scan an old abandoned miner’s village Cerro Gordo in California’s Sierra Nevada mountains, which was once home to 4500 miners. They then took this scan and imported this into Mozilla Hubs to enable virtual tours, which allows the history of the town to be preserved for future generations.

Unfortunately, when trying to create scenes yourself, there is a large learning curve that needs to be overcome in order to use the required tools.

• Spoke: This is Hubs online scene editor, which allows you to create scenes for your hubs environment from scratch or by importing your own models.
• Blender: This is a 3D modelling tool which can be used to create or edit 3D models. Hubs has introduced a blender plugin which allows you to create scenes within Blender which have compatibility with Spoke, such as adding media frames, or creating a navigation mesh to map out the walkable areas of your scene. These can then be uploaded to Mozilla Hubs through Spoke.

Hubs allow for collaboration through the use of whiteboards and screen sharing. You can import your objects into a scene. They can create media frames, which can be appended to TVs so it looks as though you are streaming your screen to the TV within hubs. Pretty cool right? Well, not when it comes to practicality. First, to change the size of the media frame, you need to use Blender. After spending a day teaching myself the basics of the editing platform, I could not get this to work, as you have to walk up to the screen and drop the screen onto it. Unless you are standing up while wearing the VR headset this is very difficult. It’s not very intuitive and that’s disappointing as I saw a lot of potential for Mozilla hubs.

You can view the Mozilla Hub I created by going right HERE!

Spatial

*Our meeting room scan inside Spatial, viewed on desktop.

Spatial makes the process of creating a room and importing a model much more streamlined. However, it could do with having similar functionality to Mozilla’s scene editor Spoke in terms of moving objects around, as it’s very difficult to place objects in an exact location, as this is purely drag and drop, rather than having a navigation arrow that allows you to move things, meaning things can’t be moved in one direction very easily. Spatial also allows you to import furniture, share your screen, and interact with your environment. It’s just a shame that the ease of use is again not very intuitive.

FrameVR

FrameVR is, in my opinion, the best choice for VR meeting software. Its free version allows you to have up to three active meeting rooms at a time, each allowing 8 participants. It allows you to upload custom environments, which you can then customise with objects which can be downloaded directly from Sketchfab, a platform which allows you to share and download 3D models, with some models freely available.

Before you upload your model, FrameVR provides great documentation on how to optimise the model for frameVR before you upload it and also allows you to upload your model to a sandbox before you upload it to frameVR.

Once a model has been uploaded, FrameVR provides the ability to easily snap sharescreen onto a TV, and snap a whiteboard to a wall, which was difficult in other meeting software. One thing to bear in mind is that complex models cannot be uploaded to FrameVR, for example, I was initially unable to upload a lidar scan of our office cafe taken with Polycam. This turned out to be due to the textures within the model being extremely large. to address this, when I rescaled the model using Blender I exported textures as .jpg to significantly reduce the size of the file so that it is small enough to be uploaded to FrameVR.

*Our geometry scan inside FrameVR, viewed on desktop.

Here is our room scan, which maps out the geometry of our office cafe. FrameVR allowed me to add a stage, coffee tables, chairs, and pool tables. This means that you can customise your environment to meet your needs.

*Our LiDAR scan inside FrameVR, viewed on desktop.

And here is our LiDAR scan of the same Cafe, which allows us to walk around the space and add the same assets as in our geometry scan.

It’s important to note that these VR meetings are not constrained to meeting in an office, once a point cloud of an object/room has been obtained these can be uploaded to meeting software and viewed in VR, so there is huge potential for the use of these technologies in digital twins, immersive experiences and VR training. For example, VR has been used to gamify training for fixing offshore wind turbines, and the AA is using VR to assist in training driving instructors. There are many models available for free online. I took this museum scan from Sketchfab, created by user artfletch, and was able to scale this by x2, upload it to FrameVR, and walk around the environment. This requires a navigation mesh, which maps out the walkable areas of the 3D model. For more information on this check out this blog post.

*Museum scan inside FrameVR, viewed on desktop.

*Museum scan in FrameVR, viewed in VR.

Why is this scan better than the one I created?

This scan was created using photogrammetry, using the Sony A7 III camera to take 33,284 photos. This top-of-the-range camera can pick up on a lot more detail than using our iPad or iPhone, and can capture more depth of the environment. These photos were then processed in an app called Reality Capture, and then heavily manually cleaned up to improve its quality.

“A craftsman is only as good as his tools”

Emmert Wolf/Dani Southall

My top tips for using FrameVR are:

1. Make sure the scan you are trying to upload is not too complex, this can be analysed using the sandbox.
2. If your model is too complex to upload, it might be due to the textures in your image, exporting your textures as .jpg from Blender should significantly reduce the size of your model.
3. Make sure your model is to scale.
4. To explore more of your model in VR, take some time to learn how to create a navigation mesh.

How can you access these meeting rooms?

Most meeting software allows you access on your PC, phone, and VR headset, allowing you to visit the room from whichever device you have available. Some of these provide this through an App, such as Meta Horizons, while others, like Mozilla hubs and FrameVR use a website (WebXR). I found that the use of a website was great when on a laptop or your phone, but this was difficult to use in VR. This was especially true for Mozilla Hubs, which requires you to input a long URL, or a code in order to join rooms. FrameVR’s URL are a lot smaller, which reduces the time it takes to join a room, but this can be a bit difficult if you are not used to the headset controllers.

A strange thing I observed in the VR headset (when we were using FrameVR) was that sometimes textures do not appear the same as they do on the desktop. This may be due to how I export my textures in Blender. Oftentimes it can be hard to troubleshoot these tools as there is often limited documentation, which can make it hard to resolve these issues.

*LiDAR Cafe Scan inside FrameVR, viewed on desktop.

*LiDAR Cafe scan inside FrameVR, viewed in VR.

Where should you start?

I recommend Polycam and FrameVR as a starting point for hosting meetings in your own metaverse. While FrameVR requires a bit of knowledge of Blender in order to scale the scan you have made in Polycam and then create a navigation mesh for import, this can easily be picked up in a day or two. This is a really straightforward, easy-to-learn pipeline, that in my opinion is the most intuitive for beginners.

*Geometry Cafe scan inside FrameVR, viewed in VR.

 

What is it like to meet in a VR headset?

Katie’s Thoughts:

While it can be difficult to initially set up the headset, once you get the hang of the controls it is an incredibly immersive experience when it works. Although our scan of the office is not perfect, it still feels as if you are walking around in our Cafe, and you can see the potential for meeting here if people are unable to make in-person meetings. It’s a shame that our meeting in the LiDAR Cafe scan was unsuccessful in VR, as I feel this would have been the most immersive experience in VR. This thankfully doesn’t happen all the time. For example, here is another office scan that can be viewed VR.

*LiDAR meeting room scan inside FrameVR, viewed in VR.

Hopefully, I will be able to address this as I learn more about Blender and FrameVR. Despite this, it was also very cool to visit other spaces within the meeting software, including the museum, which makes room for using this technology for recreational or educational purposes. This enables you to explore new places from the comfort of your own home, which is exciting.

Andrew’s Thoughts:

Meeting in a 3D scan can be fun! There is an exciting novelty to being able to “visit” a new or familiar space. Being able to have a virtual whiteboard or drop in 3D assets on the fly gives the meeting a nice level of interaction and opportunity for shenanigans. It's something that can’t really be replicated in 2D. I feel the issues with meeting in VR headsets are the same issues that plague the rest of the VR experience. Even the meeting platforms described as “easy to use” are still much more complicated than Zoom or Teams. While the hardware of the Oculus Quest is fantastic, the software needs development - every time you turn the headset on you are usually faced with a roadblock - maybe there is an update, or you need to go to the app for some reason, or your account has been locked. It's never a slick experience. The lack of high-quality pass-through (being able to see the outside world) leaves me feeling slightly disoriented and isolated. Then, in the meeting itself, talking to the current generation of avatars (while high quality), gives me the same feeling I get when talking to someone on Zoom when they have their camera turned off. Overall, a lot of these things will improve as the software and hardware mature. When it all works though it is a unique and immersive experience. Overall, I’m optimistic - meeting in a 3D-scanned version of your office can be really fun!

To summarise, even the best VR meeting software has its limitations, so it’s important to consider this when attempting to meet in a custom environment you have scanned. However, many of these are still in their infancy, so there is hope these limitations will be addressed in time, and more complex models and interactions can be used in the future. For now, a lot of trial and error is needed, but in most cases, this does not take away from the fun and unique experience of meeting in VR.