The full power of the Sony F65 and Super-Resolution Demosaicing, (SRDM), processing, has been fully exploited to produce a remarkable installation, in Tokyo.
Nest+Visual, Inc. has successfully met the challenge as leader of a project to produce 16K ultra-wide (16320×2304-pixel) content, to be presented on a 36-screen display installed at Tokyo International Airport, Haneda. Particularly significant was their decision to avoid use of multiple cameras and shoot the 16K-wide by 2K-high content, with a single Sony F65 CineAlta camera, equipped with an anamorphic lens. Because the camera itself shoots at 8K×4K resolution, it was necessary to first demosaic the image data to get to 16K×4K, and then crop the result to bring the height to 2K. This marks the first attempt to use a single camera for this type of display, an approach that eliminates the need for setting up and co-ordinating multiple cameras and then stitching the results—allowing all data to be easily captured on site and processed, later. Indy Associates, Ltd. was responsible for planning and physical production; while TYO Technical Ranch, Inc. and Sony Corporation provided technical support.
The content was produced for display on a 12×3 array of monitors covering a total width of 16.8 meters (55 ft) at a height of 2.3 meters (7½ ft). The full display, in other words, is 16,320 pixels across (16K) by 2,304 pixels vertically (2K – see photo.) To achieve 16K native resolution, the producers used a single Sony F65 camera equipped with an anamorphic lens. The conventional approach to shooting at resolutions or aspect ratios above 4K is to stitch the images obtained from multiple cameras. This was the method used when the first 16K production for this giant array opened at Haneda on 30th March, 2014 — which began running immediately when the expanded terminal opened. The conventional method of using multiple cameras, however, does require careful set-up and a precise co-ordination of cameras and lenses, and an often time-consuming stitching process. The advantage of the single F65-Sony F65 approach, used here, means that the production can be completed with much less equipment, meaning equipment costs and post-processing costs are both significantly lower.
Shooting in Frankfurt, Munich, London, and Paris
This film shoot was the first attempt ever to shoot 16K content using a single F65, together with an anamorphic lens. The Sony F65 camera utilizes a single-chip 8K CMOS image sensor carrying about 20 million pixels. A super-resolution demosaicing (SRDM) processor was applied to the camera’s raw data to further boost the resolution. This unique SRDM processor utilizes a newly developed algorithm that boosts the resolution to 8K, by referring to databases optimized to match the F65’s optical system and camera characteristics. Specifically, it can convert the raw data from a single F65 into DPX images with resolutions of up to 8K × 4K (8192×4320 pixels). To further extend this resolution processing, the producers also used an Arri Master anamorphic lens, (a cinemascope-type lens), to squeeze the image horizontally. Specifically, the lens compresses the camera’s raw data laterally to about 1/2 the wide-screen’s dimensions. The SRDM processor then expands (demosaics) the data (8K × 4K) back to its correct proportions, producing a 16K×4K image. This image is then cropped to achieve the 16×2 aspect ratio for 16K ultra-wide view.
The content was produced by Yoshiya Okoyama (photo), from wise, Inc. Here, he explains the benefits of shooting with a single camera, “If you use multiple cameras to get data for 16K resolution, then you’re going to have to consume a lot of work time stitching it all together. This time, we avoided that completely by using a single camera. We were able to work efficiently on site, and the post-processing work was much easier to handle.” Kōji Onomichi and Hajime Tanaka and Hajime Tanaka from TYO Technical Ranch’s Crank division joined the team, with Mr Onomichi responsible for the cinematography. Mr. Onomichi looks back over the decision to go with one camera, and how things worked out as a result:
\”When we started the project, the prevailing idea was that we would shoot with two cameras and use stitching. But there are many problems with this approach, similar to those when shooting 3D: it’s difficult to set up the rigs, difficult to adjust the cross-points, and so on. Even if we used two identical cameras, there would still be differences at the hardware level – and the shooting difficulties get worse, obviously, as the resolution goes up. So, while we were running camera tests, we came up with the idea of using a single F65, shooting at 8K with an anamorphic lens. So we shot various scenes using both approaches, and did rigorous dot-by-dot comparisons of the resulting image quality. We confirmed that the single-camera approach would give us all the quality we needed, and so we decided to go down that road instead. So of course, this made it much easier to set up the camera at our locations. And it hugely reduced the amount of equipment we had to lug around. And, when you’re using only one camera, you have more freedom to move it around and to operate it the way you want to. For example, we have a church scene where we pan from a horizontal interior view up to the ceiling—this would be very difficult to accomplish when using multiple connected cameras. We were also able to do handheld shots; and shots of people moving back and forth across the entire 7:1 width.\” Aoki Takashi, Business Planner in the Solutions Business Development department of Sony’s of Sony’s Professional Solutions Group, comments: \”The start of broadcasting from Channel 4K (Japan’s first 4K-dedicated channel), and the anticipation of 8K broadcasts from the 2020 Olympics, have drawn considerable attention to resolutions beyond 4K. Not only for video but, also, for large images, for signage, and more. The SRDM processor used in this project generates high-resolution images in freely selectable aspect ratios and resolutions—freeing up the use of ultra-high resolutions with genres and display devices that were previously off limits. It opens the way to new types of expression, and new types of video experiences.\” Sales of this SRDM processor are expected to begin in the final quarter of fiscal year 2014.
On location, (across Germany, Great Britain, and France), shooting ran from September 27th to October 12th, 2014. Numerous tests were run for each scene to ensure sufficient mobility and quality. Technical producer Yoshiaki Ishigaya (TYO Technical Ranch), R&D technical supervisor Yūsuke Yamazaki (TYO Technical Ranch / PPC), and colorist Ai Hirata (also TYO Technical Ranch / PPC), worked together to devise camera setups best suited to the indoor and outdoor settings and the subject movements within each scene – successfully achieving optimal camera performance under relatively restrictive shooting conditions. According to Mr. Ishigaya, “The idea was to shoot these European settings in completely natural lighting, so we really had to understand the camera and work out the best approaches.” The filmmakers shot 70 hours of footage, in F65 RAW/SQ mode at 60p. One terabyte of SR memory can store 24 minutes of such footage. DIT Hajime Tanaka had to use considerable ingenuity when confronting data management for a total of 12TB of HDD storage.
The squeezed image data from the anamorphic lens was sent through a mixer, cropped to 7:1 size, and checked on the output monitor. According to Mr. Onomichi, “This was a great experience for me as a cameraman, because it’s quite hard to get the right angles with a 7:1 ratio, and to get the right balance between people and scenery. And when you’re shooting 16K, you have to be extremely precise about the focus. So I was using a magnifying glass when I was checking the focus on the monitor.” As described above, an SRDM processor was used to convert the raw data into 8K resolution, using S-Log2 gamma and S-Gamut color space and converting the result into DPX. Data was then converted to 2K/ProRes for offline editing with Final Cut Pro. TYO Technical Ranch’s PPC grading room did the grading on finished cuts. Nest+Visual did the finishing internally, using Flame Assist. Ai Hirata served as the colorist, and Kōei Ishizaki was the editor. According to Mr. Okoyama, “We spent about two months in R&D, communicating very closely among ourselves, and as a result we succeeded in putting together a new and entirely original workflow. This was an entirely new challenge, and I think the technical team responded with considerable creativity.” This production was the second of a series of films to be presented in 16K ultra-wide resolution on Haneda’s 36-monitor display. With the common theme of, \”From Haneda to the World,\” the films show travel scenes of the different cities served by the airport. The first work in the series showcased World Heritage sites within Asia, whereas this second production looks at the Arts in four European cities — in particular the works and landscapes associated with Goethe (Frankfurt), Ludwig II (Munich), Edward Elgar (London), and Debussy (Paris). The film began showing at Haneda Airport in February 2015.
Indy Associates: Keiichi Okada, Miori Hirabayashi, Chiaki Takahashi. Nest+Visual: Yoshiya Okoyama, Yōhei Sako, Kentarō Wada Kōji Onomichi, Yuki Numao, Toshiaki Imaoka, Hajime Tanaka, Ai Hirata, Kōei Ishizaki, Yoshiaki Ishigaya, Yūsuke Yamazaki, Shinji Kobayashi, Yūta Shimakawa. Image: Sitting: Kōei Ishizaki Standing (left to right): Kōsei Ueyama (Nest+Visual), Yoshiya Okoyama (wise), Kōji Onomichi (Crank), Yūta Shimakawa (Nest+Visual), Yoshiaki Ishigaya (TYO Technical Ranch), Aoki Takashi (Sony).