Sustainability of Digital Formats: Planning for Library of Congress Collections |
|
Introduction | Sustainability Factors | Content Categories | Format Descriptions | Contact |
Full name | Apple ProRes 4444 Codec Family |
---|---|
Description |
The Apple ProRes 4444 Codec Family, comprising Apple ProRes 4444 and Apple ProRes 4444 XQ, are proprietary, lossy compressed, high quality video intermediate codecs designed to support high-dynamic-range imagery and motion graphics. These codecs, along with Apple_ProRes_422_Codec_Family and Apple ProRes RAW were developed for high-quality, high-performance editing in Apple's Final Cut Pro post-production and editing software and are widely supported in a variety of licensed third party products and workflows. The four digits in the name represent the three-part ratio describing the chroma subsampling values plus the optional alpha channel data which creates transparency in compositing or layering workflows by defining how much of the RGB or Y’CbCr pixels of a foreground image should be blended with the pixels in the background image. As explained in the Wikipedia article Chroma subsampling, the ratio describes the number of luma and chroma samples "in a conceptual region that is J pixels wide, and 2 pixels high." In the case of 4:4:4:4, all of the color components have the same sampling rate. The four parts of the ratio are as follows:
In the ProRes White Paper (dated January 2020 as of this writing but updated frequently), Apple states that "the Apple ProRes 4444 XQ and Apple ProRes 4444 codecs can be considered “visually lossless” for encoding the Y’CbCr or RGB pixel values intended for viewing, but “mathematically lossless” for encoding the alpha values that specify compositing." See Notes for more information about visually and mathematically lossless compression. According to Larry Jordan's Pick The Right Version of ProRes, "the difference between the 422 family and the 4444 family is how they deal with color. Image resolution is the same between the two. 422 color sampling creates one color value for every two pixels. 4444 color sampling provides a color sample for each pixel." Features shared by both the Apple ProRes 4444 and 422 ProRes codec families are:
Capabilities unique to the Apple ProRes 4444 family are:
In addition, Apple states that "Apple ProRes 4444 XQ preserves dynamic ranges several times greater than the dynamic range of Rec. 709 imagery—even against the rigors of extreme visual effects processing, in which tone-scale blacks or highlights are stretched significantly. Like standard Apple ProRes 4444, this codec supports up to 12 bits per image channel and up to 16 bits for the alpha channel". Larry Jordan in his What Is ProRes 4444 XQ? (updated) article emphasizes that "increased data rate of ProRes 4444 XQ is beneficial for capturing the superior tonal range of HDR material like Log C [wide gamut color space]". ProRes 4444 XQ records color data with to a finer level of accuracy which allows for more flexibility in editing at the end. Target data rate for Apple ProRes 4444 is 330 Mbps and 495 Mbps for Apple ProRes 4444 XQ for 4:4:4 sources (meaning without the alpha channel) at 1920 x 1080 and 29.97 fps. Using those parameters, typical file size for Apple ProRes 4444 is 146 GB/hour and for Apple ProRes 4444 XQ, approximately 223 GB/hour. ProRes codecs were usually contained within the QuickTime "mov" wrapper but starting with Final Cut Pro 10.3 released on October 27, 2016, the option for using ProRes in the MXF Generic Container was added as an option for broadcast delivery. Previously, MXF import and export was only available through third-party plug-ins starting with version 10.0.6 released in October 23, 2012. See Notes for more information. While detailed technical information on the ProRes codecs is not freely available directly from Apple, Apple did author a Registered Disclosure Document (RDD) from SMPTE (Society of Motion Picture and Television Engineers) in 2015. RDDs are not engineering documents or standards but rather "are documents within the scope of interest of the Society, submitted by any entity (the “Sponsor” [Apple Inc in this case]) that wishes to make a disclosure available to the public and to use the Society’s publication and distribution mechanisms for this purpose." SMPTE RDD 36-2015: Apple ProRes Bitstream Syntax and Decoding Process (available for a fee from IEEE) defines the Apple ProRes bitstream syntax, the bitstream element semantics, and the decoding process used to produce decompressed images for the 422 and 4444 ProRes codec families. According to this document, each video frame begins with a frame container atom which details the frame_size and frame_identifier which is always icpf for "image codec prores frame". After the frame container comes the frame header which includes the chroma_format declaring the sampling format of the frame (code value 2 for 4:2:2 and code value 3 for 4:4:4) and the alpha_channel_type field declaring the type of alpha channel data encoded in the bitstream, if any in Table 7. Those coded values are 0 = No encoded alpha data present in bitstream; 1 = 8 bits/sample integral alpha; 2 = 16 bits/sample integral alpha; with values 3-15 reserved for future use. Image data within individual frames starts off with a picture_header containing information about the picture size followed by any number of slices, each with its own slice_header. Figure 2 – ProRes bitstream syntax structure hierarchy on p. 8 gives a helpful visual diagram of this tiered structure. Picture data "are divided into macroblocks and then further into blocks. Macroblocks are 16×16 arrays of pixels, and blocks are 8×8 arrays of video component samples. For 4:2:2 sampling, each macroblock thus consists of four Y′ (luma) blocks, two Cb blocks, and two Cr blocks; for 4:4:4 sampling, each consists of four Y′ blocks, four Cb blocks, and four Cr blocks. ... Finally, the macroblocks of a picture are grouped into slices. Each slice consists of 1, 2, 4, or 8 contiguous macroblocks all in the same macroblock row. ... Each slice can be encoded and decoded independently, which facilitates parallel encoding and decoding." |
Production phase | Production (initial state) and post production (middle state). |
Relationship to other formats | |
Affinity to | Apple_ProRes_422_Codec_Family, Apple ProRes 422 Codec Family |
Affinity to | ProRes_RAW_Family, Apple ProRes RAW Codec Family |
Used by | QTV_AppleProRes4444, QuickTime Video, Apple ProRes 4444 Codec Family |
Used by | MXF Generic Container Mapped to Apple ProRes 4444 Codec Family. Not described at this time |
LC experience or existing holdings | The Library of Congress has a small number of Apple ProRes 4444 Codec Family files in various collections. In addition, the National Audio Visual Conservation Center can create Apple ProRes 4444 Codec Family files for duplication orders. |
---|---|
LC preference | The Library of Congress Recommended Formats Statement (RFS) lists Apple ProRes 4444, 4444 XQ and 422 HQ in a QuickTime wrapper as a Preferred format for Video - File-Based and Physical Media. |
Disclosure | Partially documented. Proprietary format developed by Apple, Inc. |
---|---|
Documentation | Apple provides high level information through its frequently updated ProRes White Paper. More detailed information about the Apple ProRes bitstream syntax, the bitstream element semantics, and the decoding process used to produce decompressed images is available in SMPTE RDD 36-2015: Apple ProRes Bitstream Syntax and Decoding Process which is available for a fee. Multimedia Wiki is another source but this site is not updated regularly. |
Adoption |
The Apple ProRes codecs were developed for high-quality, high-performance editing in Apple's Final Cut Pro post-production and editing software and also are widely supported in a variety of licensed third party products and workflows. Apple maintains the full list of licensed products although this list does not distinguish between the 422 and 4444 codec families. Key among the these is ARRI which, in 2014, was the first to introduce encoding capability for ProRes4444 XQ in its ALEXA XR/XT cameras. In Apple ProRes in-camera recording, ARRI details its history of ProRes recording capabilities including "In 2010 ALEXA was the first digital motion picture camera on the market offering in-camera recording of Apple QuickTime/ProRes files onto SxS PRO cards, providing the full range of codecs from Apple ProRes 422 Proxy to Apple ProRes 4444 and later 4444 XQ." Support for MXF-wrapped ProRes was introduced in 2019 with the ALEXA Mini LF. While the ALEXA Mini LF will only support ProRes in MXF, ARRI cameras that currently support QuickTime/Apple ProRes will continue to use QuickTime. The Blackmagic URSA Broadcast camera also supports Apple ProRes 4444 XQ and Apple ProRes 4444 in QuickTime. Starting in 2015, Microsoft's Azure Media Services supports all the Apple 422 ProRes and Apple ProRes 4444 Family codecs in QuickTime in its Media Encoder Standard transcoder. Other major implementors include Adobe Adobe Creative Cloud video apps Premiere Pro, After Effects and Adobe Media Encoder which supports ProRes 4444 and ProRes 422 export on macOS and Windows 10 starting in December 10, 2018. AJA Video Systems released an update to Desktop Software v15.2 in 2019 for Apple ProRes family capture/playback across Windows, Linux and macOS in AJA Control Room. Apple ProRes 4444 XQ is supported on OS X Mountain Lion v10.8 or later. Apple ProRes 4444 was added with Final Cut Pro 7 (2009) and Apple ProRes 4444 XQ was added to Final Cut Pro X (2012). Although Apple complains that unauthorized implementations provided by other third-party software and hardware products including FFmpeg "might result in decoding errors, performance degradation, incompatibility, and instability," this support is a further indicator of adoption. |
Licensing and patents |
Apple licenses and certifies specific third party products and workflows. The full list of licensed products is available. Licenses are not required for content creators. |
Transparency | Proprietary binary format, requires tools to write and read. |
Self-documentation |
For AppleProRes4444 in QuickTime implementations, technical metadata is in the QuickTime frame container atoms and frame headers which declare the codec type ("icpf" for ProRes), frame size and color information (stored in src_pix_fmt). The Multimedia Wiki includes a list of src_pix_fmt values but these values do not include needed options for ProRes 4444 pixel values since ProRes 4444 can use either RGB or YUV pixel formats. Accessibility Features The Apple ProRes 4444 Codec Family offers moderate accessibility support in addition to that described for the QuickTime container format. Per Apple specifications, caption data may be added within the alpha channel. Typically, accessibility features such as captions and subtitles are carried in containers and wrappers, not in encoded video data. QuickTime is a common wrapper for ProRes family codecs, though others may be used. See QuickTime for web accessibility information. See W3C's Making Audio and Video Media Accessible for more general information about accessible sound and moving image media. |
External dependencies | None. |
Technical protection considerations | None. Comments welcome. |
Moving Image | |
---|---|
Normal rendering | Supported |
Clarity (high image resolution) |
The Apple ProRes4444 codecs support mathematically lossless compression in the alpha channel values only of any bit depth up to and including 16 bits where it is essential to maintain exact color values rather than approximate color values. This is a change from the RGB or Y’CbCr pixel values, where approximate encoding is acceptable (aka visually lossless compression) as long as differences from the original are not visible to the viewer and do not affect processing. |
Functionality beyond normal rendering |
The performance of Apple ProRes codecs scales in multiprocessor environments so that the decoding time per frame goes down as the number of processors increases. When the system spends less time decoding each frame, it has time for more real-time effects processing. In addition, its scalar quantization allows for the possibility to decode a video at different partial resolutions (½ or ¼ etc size of the full frame size) which lowers the data throughput while maintaining high image quality. This functionality allows the user advantages in playback of additional tracks of video and higher quality previews. Because the presence of an optional alpha channel in an Apple ProRes 4444 stream contains data about the object outline for compositing workflows (as opposed to data about the entire object contained within the outline), the presence of an alpha channel typically reduces decoding and encoding data rate performance by only about 10 percent or less according to the ProRes white paper. |
Tag | Value | Note |
---|---|---|
Filename extension | See note. | The Apple ProRes 4444 Codecs may be contained within the QuickTime "mov" wrapper. |
Filename extension | See note. | The Apple ProRes 4444 Codecs may be contained within the Material Exchange Format "mxf" wrapper in the broadcast environment. |
Other | ASCII: icpf Hex: 0x69637066 |
Abbreviation for 'image codec prores frame' according to SMPTE RDD 36:2015 which lists this frame type ID in ProRes frame container. This value does not distinguish between ProRes 422 and ProRes 4444 codec families |
FOURCC | See note. | See QuickTime Video, Apple ProRes 4444 Codec Family for the FOURCC code used to indicate ProRes 4444 in the QuickTime container |
Pronom PUID | fmt/797 |
See http://www.nationalarchives.gov.uk/PRONOM/fmt/797. However, the entry for Apple ProRes does not distinguish between ProRes 422 and ProRes 4444 codecs or subtypes. |
Wikidata Title ID | See note. | No corresponding entry as of May 2024. |
General |
The FADGI Glossary defines lossless compression in this way: “Data compressed using a lossless compression technique will allow the decompressed data to be exactly the same as the original data before compression, bit for bit. The compression of data is achieved by coding redundant data in a more efficient manner than in the uncompressed format. The compression ratios that can be achieved with lossless compression are generally much lower than those that can be achieved using lossy compression techniques. Data compressed using a lossy compression technique results in the loss of information. The decompressed data will not be identical to the original uncompressed data. Conservative lossless compression can result in a form of lossy compression referred to as visually lossless compression.” Lossless compression works by removing redundant information that can be recreated from the remaining data. Visually lossless compression is "a form or manner of lossy compression where the data that is lost after the file is compressed and decompressed is not detectable to the eye; the compressed data appearing identical to the uncompressed data." In mathematically lossless compression, there is no loss of quality when the file is encoded and decoded and the compression is completely reversible. So in visually lossless compression, the compressed data looks identical to the uncompressed data and with mathematically lossless compression, the compressed data is identical to the uncompressed data. Apple's ProRes White Paper uses peak signal to noise ratio (PSNR) to indicate image quality, stating that PSNR measures "how closely a compressed image (after being decompressed) matches the original image handed to the encoder. The higher the PSNR value, the more closely the encoded image matches the original." Higher PSNR values also indicates more headroom in the file so that image sequences can be decoded and re-encoded over multiple generations to good effect. In the January 2020 version of the White Paper, a table on page 17 compares four ProRes 422 and 4444 subtypes with this comment, "the qualitative description for Apple ProRes 4444 (without an alpha channel) is identical to that for Apple ProRes 422 HQ. This is because Apple ProRes 4444, though its target bit rate is 50 percent higher than that of Apple ProRes 422 HQ, uses extra bits to encode the greater number of chroma samples in 4:4:4 at the same high quality headroom ensured by Apple ProRes 422 HQ for 4:2:2 sources." |
---|---|
History | While most ProRes 422 codecs use the QuickTime wrapper, the higher technical capabilities of the ProRes4444 codecs lead themselves to application in the broadcast community including MXF users. To facilitate this, Apple authored SMPTE RDD 44:2017-11: SMPTE Registered Disclosure Document - Material Exchange Format — Mapping and Application of Apple ProRes (available for a fee from IEEE) which maps ProRes bitstreams into a subset of the MXF Generic Container (SMPTE ST 379-2). MXF support was added to Final Cut Pro 10.3 in October, 2016. |
|