Sony provides the solution to photo-realistic digital assets in contemporary games
Malmo, Sweden and London, UK-based Sharkmob uses a synchronised array of 144 Sony cameras in a spherical frame to capture realistic images of people and objects. Software converts the photographs into a 3D form, allowing gamers to see the objects photorealistically from any angle. The reliability and high resolution of Sony’s cameras give Sharkmob access to incredibly high-quality 3D assets and the ability to produce them day after day without issue.
We want to make games that are great for players. We want real-world objects that are really lifelike. That’s why we use photogrammetric scanning.
Anders Holmquist, CEO and Co-Founder of Sharkmob
Embracing the power of modern gaming technology
Computer game technology has always pushed technological boundaries, constantly seeking ways to increase resolution, detail, and frame rates. Games engines like Unreal and Unity can generate photorealistic objects and entire worlds from pre-made models in real time. Sharkmob’s photogrammetry team focuses on one aspect of gaming that is definitely not real-time: creating 3D assets from real-world objects. This calls for a subtle mix of skills, from artistic to technical, where the complex shapes of real-world objects are modelled using first a point cloud and then polygons—which are compatible with game engines. It is laborious, especially when modelling real people (their faces, for example) and the almost infinite complexity of both the natural and manufactured worlds.
Turning real-world objects into high-resolution 3D assets
According to Ian Pearce, Lead photogrammetry artist at Sharkmob, a traditional artist could take one or two weeks to craft a single, everyday item like a fire extinguisher. Familiar, mundane objects are arguably as important for establishing credibility in a 3D world as exotica like dragons and alien spacecraft. With resolutions and detail increasing all the time, thanks to GPU and display technology, Sharkmob has turned to an alternative approach for certain critical assets: digital photography and a technique called Photogrammetry.
Still-image cameras can capture incredible detail, but only from one position at a time. To model a complex 3D object that is viewable from all angles, it must be photographed from many positions, preferably simultaneously. Solving that technical challenge can involve over a hundred cameras – Sharkmob uses 144 Sony α6400 cameras – each mounted at precisely the optimum location. The cameras are accompanied by the appropriate lighting and coordinated remotely.
Sony cameras chosen for image quality, compactness and remote controllability
Sharkmob has chosen to make Photogrammetry a large part of its production effort. To achieve this, it uses Esper photogrammetry rig that support over a hundred cameras and their lighting and power connections. Forming an approximate sphere around the object (or person) to be digitised, everything is optimised to capture the image data needed to create an accurate 3D asset.
We use Sony cameras in the Esper rig because they are compact and because we can control them via software, using the Sony Camera Remote Command. Critically, the images are great—you can’t go wrong with them.
Ian Pearce, Lead photogrammetry artist at Sharkmob
Keeping to a budget
Price was a factor for Sharkmob, too, because it typically buys its cameras—most recently α6400—one hundred forty-four at a time. Sony’s cameras have high resolution and colour accuracy, yet they’re cost-effective and reliable, especially for exceptionally demanding applications like Sharkmob’s photogrammetry.
Sony’s Camera Remote Command at the core of the operation
The Sony Camera Remote Command plays a critical role, as it triggers the cameras simultaneously and coordinates all settings and controls across the rig. It’s a precision operation. Capturing a face requires eight versions, each with a different expression. That’s over a thousand images per face with a hundred and forty-four cameras. So automation is essential for an efficient operation, especially given the industry trend towards faster releases and product cycles.
Fifty times improvement
Creating realistic 3D assets with Photogrammetry can be up to fifty times faster than the traditional method. “It’s all plug and play once the Esper rig and Sony cameras are set up”, says Pearce. “It just totally works. You capture one image and just keep going!”
Thanks to Sony’s Camera Remote Command and its ability to integrate tightly with asset management systems, Sharkmob is able to work quickly and efficiently despite the huge volume of image files generated by the process.
What happens post-capture?
High-resolution capture with Sony cameras is just the start of a processing-intensive process. Once the images are captured and stored, specialist software applications create the final 3D asset. The first stage is to create a “point cloud”, where the software builds a cloud of dots in virtual space that hint at the solid 3D object they represent. The cloud is generated by the software finding points that match across multiple camera views. The point cloud is then processed to create a 3D version made from polygons complete with albedo and surface normals – simple shapes that are easy to compute but which, taken together, build a more easily rendered model of the original object. The final stage is to reduce the number of polygons for easier computation without severely reducing the perceived quality of the final asset.
All in a day’s work
This type of Photogrammetry, based on the Esper rig and with Sony cameras and the Sony Camera Remote Command at the core, is already a growth area for computer game companies and related industries. It can create stunningly realistic 3D assets tens of times faster than any other method. According to Ian Pearce, “Sharkmob can do photography in the morning and have a hyper-realistic, playable character by the evening with its cutting-edge photogrammetry setup.”
The Sony cameras have performed really well; they’ve never let us down—not even once! You would expect that with 144 cameras, some would fail, but it’s never happened to us, and I’m super happy that we have this partnership with Sony. Sharkmob can do photography in the morning and have a hyper-realistic, playable character by the evening with its cutting-edge photogrammetry setup.
Ian Pearce, Lead photogrammetry artist at Sharkmob