Sustainability of Digital Formats: Planning for Library of Congress Collections

JPEG XL is a standard for a compression codec for raster graphic images designed to support lossy and lossless compression, particularly for display on the web and the needs for the web environment on a multitude of devices. JPEG XL is being developed to outperform other popular web image formats such as PNG, JPEG 2000, GIF, and WebP with higher quality images and improved compression ratios. The codec improves on previous JPEG codecs and file formats, particularly as existing JPEG files can be losslessly encoded as JPEG XL files and restored to the initial JPEG file, ensuring backwards compatibility. The "jbrd" box, the JPEG Bitstream Reconstruction Data, contains the information needed to reconstruct the bit-identical file. The JPEG XL bitstream reconstruction data uses the image data along with any other metadata present, such as Exif, XMP, or JUMBF, to reconstruct the original JPEG file. This inherent compatibility allows for an efficient transition from JPEG formats to JPEG XL without the need to store two files.

The Overview of JPEG XL highlights three primary criteria:

• High fidelity to source image. JPEG XL was designed with high-fidelity image encoding capabilities. The JPEG XL encoder can automatically perform high-fidelity encoding, and compresses images as much as possible with no visible differences. The JPEG XL reference encoder is configured based on a perceptual fidelity target rather than technical parameters like traditional image encoders. See Cloudinary's blog post on image compression and fidelity.
• Encoding and decoding speed. Testing has shown that JPEG XL with 4 threads outperformed both libjpeg and libjpeg-turbo, two JPEG codec implementations. Additional testing showed that the default JPEG XL speed, "squirrel" is 3 times as fast as HEVC-x265, while the JPEG XL "cheetah" speed is 13 times as fast. The JPEG XL decoder is 9 times faster than FFmpeg. Published results and experiment descriptions can be found in Benchmarking JPEG XL image compression paper from the Multimedia Signal Processing Group. The JPEG XL codec has an encoding speed of 50 MP/s and decoding speeds of 132 MP/s. The responsiveness functionality of JPEG XL codec allows for the support of progressively decode images which other formats based on video codecs, such as WebP, HEIC, and AVIF, do not support.
• Compression ratios. The JPEG XL compression improves upon previous JPEG encodings. Studies and testing of compression, encoding, and decoding speeds JPEG XL developers highlight the vast improvement in encoding and decoding speeds. Testing concluded that JPEG XL and HEVC were indistinguishable from the original at bitrates near 1 but JPEG XL reached higher perceptual quality compared to other codecs when using objective quality metrics. According to the XL overview on Gitlab, lossless compression reduces JPEG XL files to "60-75%" of PNG files and smaller than WebP files for photos.

Some of the key features for the JPEG XL codec as described by the JPEG group’s white paper are:

• improved functionality and efficiency compared to traditional image formats (e.g. JPEG, GIF and PNG)
• progressive decoding (by resolution and precision)
• lossless JPEG transcoding
• support for both photographic and synthetic imagery
• graceful quality degradation across a large range of bitrates
• perceptually optimized reference encoder
• support for wide color gamut and high dynamic range (HDR)
• support for animated content
• efficient encoding and decoding without requiring specialized hardware

The JPEG XL codec utilizes a number of coding tools. See the JPEG XL white paper for descriptions of all coding tools.

The codestream contains one or more frames which can be looped (infinitely or number of times) in the case of animated images. Zero-duration frames are also possible and represent different image layers. By default, the decoder will blend and coalesce frames, producing only a single output frame where all multiple zero-duration frames and all output frames are of the same size (the size of the image canvas). All output frames can have either no duration or non-zero duration. Each frame contains pixel data in one of two modes:

• VarDCT mode, where variable-sized DCT transforms are applied and the image data is encoded in the form of DCT coefficients. This mode is always lossy, but can also be used to losslessly represent an existing JPEG image. Find an example of the JPEG XL VarDCT block size selection here.
• The other mode used is modular mode, where only integer arithmetic is used, enabling lossless compression.

Both modes can separately encode supplementary "image features" that are rendered on top of the decoded image. You can read more about those image features such as patches, splines, and noise, on JPEG XL's GitLab.

One of the benefits to the JPEG XL codec and its improved compression ratios is the resulting impact on server storage. Figure 1. (on page 1) of the JPEG XL white paper highlights a usage scenario of a JPEG image encoded as a JPEG XL and the decoding process to reduce server costs. Figure 3. of the white paper, as well as the Wikipedia entry for JPEG XL, illustrates the architecture of the codec. An input image can be encoded as a compressed JPEG XL codestream in either lossy or lossless methods. See Cloudinary’s blog post about JPEG XL which illustrates visual comparisons to other image files formats such as JPEG, HEIC, and WebP.

The "X" of the JPEG XL file name is based in part on the JPEG group's naming convention for many of its new standards since 2000 including, JPEG XT, JPEG XR, and JPEG XS. The authors used the "L" as an indication of the hopeful longevity of the file format as it will replace the legacy JPEG. Comments welcome.

As of March 2022, this standard is in published status. ISO/IEC 18181-2 documented format specifications are broken into four parts. Part 1 of the format specifications was fully published in March 2022, while Part 2 was published in October 2021. Parts 3 and 4 remain in development.

Part 1 carries the running title Part 1: Core coding system. As stated on the JPEG group's own description of the standards, Part 1: "Defines the JPEG XL codestream and decoder, which can be used for lossy encoding, lossless encoding, and lossless re-compression of existing JPEG images."

Part 2 carries the running title Part 2: File Format. Part 2, "specifies an extensible box-based file format which adds support for metadata (e.g. Exif and JUMBF) and legacy JPEG bistream reconstruction data."

Part 3 carries the running title: Conformance testing. As stated by the JPEG group, this part, "provides test material and procedures to validate proprietary solutions against the standard specification."

Part 4 carries the running title Part 4: Reference software. Part 4 "provides a free and open source, royalty-free JPEG XL reference implementation."

Adoption of JPEG XL appears limited at this time. Few active web browsers support JPEG XL file display with a currently monitored list found on Github. This list includes various types of JPEG XL Software Support and its stated purpose is to, "point end-users to software that can read/write jxl and keep track of the adoption status of jxl."

An unofficial list of image viewers and image libraries that can open and render JPEG XL files are:

• ImageMagick
• libvips
• Imlib2
• lmlib2
• ImageGlass
• GIMP (with associated plugin)
• XnView
• KolourPaint
• KPhotoAlbum
• VookiImageViewer
• For additional software, see this JPEG XL overview slide deck. Other image viewers and online tools can also be found on the JPEG XL Github

jpegxl.io tutorials also provides a list of software tools that support JPEG XL images. This list consists of tutorials for software or browser configurations necessary to render JPEG XL images. Some browsers included in this tutorial include:

• Chrome Chrome developers and Google engineers announced the removal of JPEG XL browser support in October 2022. (See History in the Notes below)
• Edge
• Safari

The tutorial list includes over 30 additional software applications with various levels of support for JPEG XL encoded images. The tutorial list also provides designations of support such as "partial support" and "full support."

The JPEG Use Case and Requirements paper details the need for an improved image coding standard which may shed light on how JPEG XL files will be used including:

• social media applications
• media web sites
• animated image applications
• AR (augmented reality) and VR (virtual reality)
• applications
• Mobile apps and games

Sample JPEG XL files and conformance testing can be found here.

Comments welcome.

Per the JPEG group’s whitepaper, "to enable compression to smaller file sizes, the jpeg xl codec reducing header and metadata overhead, including compression of colour profiles and Exif/XMP metadata.”

Per Jon Sneyer's JPEG XL documentation on GitLab, there is a clear separation between metadata and image data. Whatever is needed to correctly display the image is considered to be image data including elements such as ICC profiles and Exif orientation. The developers goal is to reduce the ambiguity and potential for incorrect implementations by different applications. Remaining metadata, such as Exif or XMP, can be stored in the container format but has no impact on image rendering. Exif orientation, for example, is a field ignored by applications since the orientation defined in the codestream takes precedence. This metadata can be done without affecting the image display.

Good support for wide color gamut is a touted main feature of the new encoding. One of the JPEG XL codec’s coding tools is the creation of the XYB color space, inspired by the human eye’s color perception. It uses a gamma of 3 for compute-efficient decoding. More information about the XYB color space can be found in the JPEG XL white paper or the SPIE 2019 conference paper on JPEG XL. The XYB color space was specifically designed for high-precision and perceptually modeled color which supports a high fidelity image encoding, one of the key features of JPEG XL. See this Cloudinary blog post on image compression and fidelity.

The JPEG XL Software Gitlab provides a summary of the color management codestream features of the JPEG XL encoding.

JPEG XL images have a fully defined colorspaces with two options:

• Pixel data is in a specified (non-XYB) colorspace, and the decoder creates a pixel buffer in this colorspace along with an ICC profile that describes that colorspace.
• Pixel data is in the XYB colorspace. The decoder can produce a pixel buffer directly in a desired display space such as sRGB, Display-P3 or Rec.2100 PQ.

Colorspaces can be signaled in two ways in the JPEG XL codec:

• CICP-style Enum value, which is a very compact representation that covers most or all of the common colorspaces. The decoder can convert XYB to any colorspace without requiring an external color management library.
• ICC profiles, ICC profiles can be used including CMYK ones. External color management software has to be used for color conversions as the ICC profile data gets compressed.

The Free Lossless Image Format (FLIF) was a lossless image format first released in September 2016 that claimed to outperform other image formats for web presentation including, PNG, BPG, and JPEG 2000. Based on compression experiments conducted by the formats developers, FLIF files bypassed JPEG 2000 files to be 53% smaller, 43% smaller than PNG files, and 22% smaller than lossless BPG files. FLIF was essentially superseded by the Free Universal Image Format (FUIF) one of the components of the JPEG XL format. In combination with the FUIF, Google’s PIK file format, also designed for high quality and fast decoding, served as a precursor to JPEG XL. Both FLIF and PIK were submitted to the JPEG XL Call for Proposals (see below) in 2018 respectively. Neither the FLIF or FUIF formats are being developed as they have been superseded by JPEG XL.

The JPEG Committee launched the Next-Generation Image Compression activity, formally known as JPEG XL in 2018. As stated by the JPEG group, "This activity aims to develop a standard for image compression that offers substantially better compression efficiency than existing image formats (e.g. >60% over JPEG), along with features desirable for web distribution and efficient compression of high-quality images."

This Next Generation Image Compression activity initiated a final call for Proposals in April 2018 to further the development of JPEG XL. Important dates from the proposed timeline include:

• 06/11/2018 - JPEG XL AHG meeting (Brussels) for anchor generation and final selection of content, bitrates and metrics.
• 07/07/2018-07/13/2018 - WG1 meeting (Berlin): review of anchor evaluation results and agreement on final test set and evaluation procedures. Issue amendment to Call for Proposals regarding final content, bitrates and metrics.
• 09/01/2018 - Deadline for submission of binaries, algorithm description and design, and encoded-decoded test material.
• 10/13/2018-10/19/2018 - WG1 meeting (Vancouver). Assessment of technical proposals and objective/subjective evaluation results.

The file format bistream was frozen in December 2020 to ensure that it would be decodable by future releases.

In January 2021, Part 1 (codestream) was FDIS submitted while Part 2 (file format) was FDIS submitted with a Draft Amendment to Part 1 in April 2021. See this JPEG-XL slide deck for a broader history of JPEG XL and next steps for the file format.

Part 2 was published in October 2021

Part 1 was published in March 2022.

While Google Chrome was previously listed as a web browser that supports JPEG XL, in October 2022, Google Chrome developers announced their decision to remove support for the JPEG XL encoding and the related file format. Google Chrome initially offered JPEG-XL support via a feature flag (chrome://flags/#enable-jxl) since Chrome 91. With the upcoming Chrome 110 release, Google announced that they will be deprecating the format. Per Chromium’s bug tracker for JPEG XL decoding support, Google cited these reasons for not pursuing support for JPEG XL:

• "Experimental flags and code should not remain indefinitely
• There is not enough interest from the entire ecosystem to continue experimenting with JPEG XL
• The new image format does not bring sufficient incremental benefits over existing formats to warrant enabling it by default
• By removing the flag and the code in M110, it reduces the maintenance burden and allows us to focus on improving existing formats in Chrome"