How to create an immersive 3D scene for your Zoom webcam on macOS Last Updated 31 August 2022 by Michael Forrest
I saw Nick Jushchyshyn’s set-up on a recent Office Hours Global stream. He has a very interesting setup for his Zoom calls, as you might expect from a Program Director of Virtual Reality & Immersive Media.
Video
Here’s a video showing how to make a 3D virtual environment for Zoom.
More background
What I like about it is that it is not “style over substance”, but rather it lets us see the reactions of other people on the call as he speaks while maintaining a hierarchy of information that puts him in the centre, which might not be possible using Zoom’s built-in tile layouts.
You can have this
It’s actually not that complicated to set something like this up. Today I’ll show you how to do it on your Mac, using the free multimedia software Vuo. You won’t even need a real green screen if you use my camera app.
What is Vuo?
Vuo is an application to help people develop interactive experiences and live performances. It gives you a node-based visual coding environment making it simple to connect live video and audio feeds, with advanced compositing and 3D rendering possibilities.
Scene breakdown
To me it seems we’re looking at screen captures laid out in a 3D environment. We can use chroma key to layer virtual screens in front and behind him.
The screens are translucent planes floating in 3D space.
- Bring in your webcam, send to window (for now)
- Create screen capture (brittle, need to decide on your screen layout, or we could use ScreenCaptureSyphon?)
- Create 3D foreground layers
- Composite layers
- Chroma key: switch to Shoot, set up the shader, start with a solid colour background
- Create background layer because no depth of field camera (we could probably get a shader but let’s keep it simple)
- About Syphon, install Syphon Virtual Camera
- Send Syphon from Vuo
- Test the audio delay using the Squares TV delay calculator.
- BONUS: add some other 3D models and flashing lights, because why not?
Tell me if you want me to demystify anything else like this and I’ll see what I can do.
