Virtual Production with VENICE 2 in Dubai

November 8, 2022

Alister Chapman reports from ambitious workshop

To promote VENICE 2, Sony MEA’s Omar Abuaisha wanted an eye-catching workshop and soon decided on shooting a short film with virtual production. The Pixojam XR stage in Dubai provided an excellent facility and I was keen to help run the workshop – virtual production is something I’ve been following for some time.

Virtual production uses a large LED screen or video wall to display the background to a scene. This could be the inside of a building or an outdoor location, it could be anywhere in the world – and beyond!

Alister Chapman shooting with VENICE 2 on virtual production set.

Unreal Engine’s Virtual Background

Typically, we think of the screen being your entire background, but it can just be a small element such as the view through a window. You then film your foreground action in front of the screen, so unlike green screen there is no need to combine the foreground and background in post-production.

What makes virtual production different to other uses of large screens is that the background isn’t real, it is created using technology borrowed from the computer games industry. Powerful computers running special software such as Unreal Engine generate the background using a virtual 3D model in real time as you shoot.

When filming big screens, the refresh rate of the camera is critical: if it is too slow you may have artefacts such as light and dark bands across the screen. VENICE 2’s 8K sensor has one of the fastest readout speeds in the industry and this makes it an excellent choice for hassle-free virtual production. I have to say that everyone at the workshop was impressed at just how easily the camera integrated into the virtual production environment.

Virtual production rendering workstations.
Virtual production shoot in Dubai showing live monitoring of virtual set.

Virtual Sets

For the Dubai event, our first virtual set was a large garage. In the foreground, a model would walk over to her motorbike, pull on her riding leathers and then drive off into our second virtual set: a desert highway.

One of the things that makes a virtual production come to life is camera movement. A motion tracker relays the camera’s position and movement so that the computer-generated backdrop can be kept in synch.

By moving the camera on a dolly as we shot, we were able to demonstrate this feature.  Objects in the virtual garage could be seen to pass in front of the back wall of the garage. To further enhance the sense of realism, we built a simple foreground set using a tool chest and other items that you would typically find in a workshop and placed these so they would pass through the foreground of the shot as the camera was moved.

VENICE 2 with motorbike on Dubai virtual production set.

No need for post-production

Unlike traditional green screen, in virtual production the foreground and background are shot and combined live. You can see exactly how the shot will look instantly and this was particularly impressive for our watching audience.

There is often no need to do any further post-production. But if you do need to make any further adjustments in post, the high resolution that comes from VENICE 2’s 8K sensor allows you to crop and adjust the framing as needed. VENICE 2’s very wide colour gamut also minimises colour errors between the screen and camera. To close the workshop, we applied a slight grade to the footage and used a laptop computer to demonstrate just how quick and easy it is to work on the surprisingly compact 8K X-OCN files.

While virtual production will never replace every type of production, I expect it will become a widely used way of shooting scenes that may be difficult or expensive to shoot traditionally. For example, vehicle tracking shots or shots representing remote or futuristic locations.

There can considerable savings and shooting desert scenes on an air-conditioned stage certainly emphasised the convenience factor!