The extension of the H.264 standard coincide with rapid developments in high-resolution/high-frame rate imaging technologies (sensors, displays), and in high speed storage technology. A high performance yet efficient compression technology such as H.264 plays a critical role in encoding the vast amount of imaging data generated by modern sensors into a modest file size, so that those images can be recorded on affordable memory cards, and edited/viewed on computers/editing software packages that are readily available.

Figure 1 shows the progress in CMOS imager technology, where Sony has developed and commercialized an imaging sensor with data transfer rates exceeding 30Gbps. It is expected that high resolution, high frame rates, and high bit rate imaging tools will become more common in the coming years. Figure 2) shows the progress of SxS and XQD memory card technology. Note that the latest cards guarantee real-time recording beyond 1Gbps, and the recording capacity has significantly increased over the years. And not just maintaining the same compact form factor but to make the size even smaller.

This document illustrates the attributes and benefits of the XAVC format. It also explains how XAVC fits into the current production workflow, along with well-established compression formats such as MPEG2, MPEG4 SStP, and various camera RAW files.

Sony has developed and commercialised an imaging sensor with data transfer rates exceeding 30Gbps. It is expected that high resolution, high frame rates, and high bit rate imaging tools will become more common in the coming years.

Figure 2 shows the progress of SxS and XQD memory card technology. Note that the latest cards guarantee real-time recording beyond 1Gbps, and the recording capacity has significantly increased over the years. And not just maintaining the same compact form factor but to make the size even smaller.

This document illustrates the attributes and benefits of the XAVC format. It also explains how XAVC fits into the current production workflow, along with well-established compression formats such as MPEG2, MPEG4 SStP, and various camera RAW files.

Due to the wide array of operating points that the XAVC format offers, 4K imaging bandwidth can be reduced to under 100Mbps depending on GOP structure, frame rate, and colour sampling. Such efficient selection of operating points is expected to significantly augment the home entertainment experience and applied to certain B2B applications where high resolution imaging is of prime interest.

The active pixel count of most 4K home displays will be restricted to 3840 x 2160, a quadruple of 1920×1080. (Quad HD or QFHD), which is different from the Cinema presentation standard that has 4096 pixels across the image plane. The XAVC format covers both 4096 and 3840 horizontal sampling formats, allowing the XAVC production tools to be used in both cinema and television applications.

Sony’s PMW-F55 camera records 4K XAVC Intra frames at operating points between 240Mbps (@24P) to 600Mbps (@60P) within the camera. PWS-4400 server supports these operating points as well to be suitable for 4K Live production platform. Bitrate for 4K was the major discussion point throughout the development. 100Mbps for 1080i is regarded as a reasonable bitrate as this has been widely used in the broadcast industry. Theoretically, to suffice required bit rate for 4K 60p, four times in resolution and double the frame rate would require eight times of data. This sums up to 800Mbps, nevertheless, 25% of the data can be reduced based on encoding efficiency for progressive frame compared to interlace.

In order to securely record such high data rate by a cost efficient yet compact media, Sony have developed the SxS Pro+ memory card family. The SxS Pro+ memory cards are compatible with all devices that have an SxS card slot, and achieve a sustained recording data rate up to 1.3Gbps. On a single SxS Pro+ 128GB memory card, the PMW-F55 records up to 50 minutes in 4K/24P or approx. 20 minutes in 4K/60P.

While the PMW-F5/F55 cameras are recording camera RAW files via the AXS-R5 on-board recorder, the in-camera SxS card slots can simultaneously record XAVC files at HD resolution, which perfectly matches the RAW files in respect to recording In/Out points, audio, timecode, and other metadata including file names. The XAVC files complement the RAW files as editorial files, which allow the editing process to start immediately when the SxS cards are removed from of the cameras.

In order to colour grade and edit images derived from camera RAW files, an extra image processing step (typically referred as De-Bayering, or De-mosaicking) is required. Although the camera RAW files offer the maximum creative freedom which is imperative for sophisticated post production work, there can be time/budget constraints. 4K XAVC files can be considered as a cost efficient alternative to camera RAW files. As shown on Figure 5 the 4K XAVC file size is similar to those of HD resolution files that are commonly used today. It is expected that 4K XAVC files will be one of the major driving force in expanding 4K production.

Today, the vast majority of broadcast and reality-TV industry is operating on the MPEG2 HD Long GOP format (50Mbps or 35Mbps) for HDTV production due to its small file size, high picture quality, and economy in computational requirements. From breaking news to reality shows, and prime sports events, the data rate
of 35 to 50Mbps is the sweet spot to operate a file based HDTV infrastructure.

Figure 6 shows how different compressed video streams can be decoded on a given computer platform, without resorting to any hardware accelerators or GPU`s. The horizontal scale represents Frames-per-Second, and it is obvious that MPEG2 50Mbps is the fastest (or most efficient) amongst all of the contenders.

Recently several broadcasters have started showing interest in adopting H.264 format as their main in-house format due to the following reasons.

• Consolidate all program files, from prime time programming to news, into one single codec contained in a common, industry-standard wrapper
• 10 bit sampling as opposed to 8bit on MPEG-2
• MPEG-2 50Mbps image quality perceived as not enough to replace current tape formats such as HDCAM
• Storage space, network bandwidth, and processing power are becoming less of an issue to handle multiple high bit rate streams

For 50p/60p HDTV operation, XAVC Intra supports up to 440Mbps and can be considered as the mezzanine level format that fills the gap between the mastering quality format (MPEG4 SStP, or HDCAM-SR), and MPEG2.

XAVC also plays a critical role in enabling a portable camcorder with extreme high-frame rate capability in HDTV resolution. The PMW-F55 records full resolution 1920×1080 10bit 4:2:2 images up to 180 Frames-per-second onto the internal SxS Pro+ memory cards. By filling the two memory card slots with 128GB cards, continuous recoding time is extended to approx. 40 minutes at 180Fps.

Demand for keeping the file size small enough to run through the current 35-50Mbps pipeline, even in the 1080-50p/60p operation workflow will still remain. Utilising Long GOP is the preferable technique for this demand, which makes file size smaller without affecting the picture quality. As shown in figure 6 below, decoding performance is quite equivalent between Long GOP 50Mbps and Intra 100Mbps, although more arithmetic is required.