Sustainability of Digital Formats: Planning for Library of Congress Collections

VP9 is an open and royalty-free video coding compression format. According to VP9 Wikipedia, “VP9 is the last official iteration of the TrueMotion series of video formats that Google bought in 2010...with the company On2 Technologies that created it.” VP9 is the successor to VP8 video compression format, for the WebM Project open video codec in 2013. VP9 Video Codec is documented in the Draft VP9 Bitstream and Decoding Process Specification, last modified March 2017, as well as the related document from the Internet Engineering Task Force (IEFT), RTP Payload Format for VP9 Video. WebMProject.org’s FAQs page describes VP9 as a highly efficient audio compression technology, VP9 delivers “high-quality video while efficiently adapting to varying processing and bandwidth conditions across a broad range of devices...VP9 makes it easy to integrate into existing environments and requires comparatively little manual tuning in the encoder to produce high-quality results.”

Next-Generation VP9 Bit-stream:

According to A Technical Overview of VP9—the Latest Open-Source Video Codec, 2015, “In late 2011, Google started working towards a next-generation video codec, aka VP9, with the explicit goal of creating a format that produces a much more compact bit-stream than predecessor VP8 (fdd000578), especially for high-definition content, with only a modest increase in decoding complexity.” The article also describes the VP9’s Coding Tools and Bit-Stream Features, referenced below.

Coding Tools:

• Prediction Block-sizes – introduces super-blocks (64x64).
• Prediction Modes – supports set of 10 Intra prediction modes.
• INTER Modes – supports set of 4 Inter prediction modes.
• Subpel Interpolation – supports family of three 8-tap filters and Bilinear filter.
• Transform Types – uses Discrete Cosine Transform (DCT), Asymmetric Discrete Sine Transform (ADST), and Walsh-Handamard Transform (WHT).
• Entropy Coding and Adaptation – bit-stream is Arithmetic Encoded; allows a combination of frame-level forward and backward updates.
• Loop Filter – designed to eliminate blocking artifacts on boundary types, loop filter operates in raster scan order.
• Segmentation – supports framework with flexible tools to be used in an application-specific way to improve perceptual coded video quality.

VP9 Bit-Stream Features:

Note: Features are “designed to support specific use-cases that are important to Internet video delivery and consumption...Exploiting these features appropriately needs specialized encoders and/or decoders to be created.”

• Error-Resilience – coding mode where decoding can continue without errors even if some frames are lost, allows frames to be decoded independently of previous frames.
• Frame-Parallelism – frame level flag, enables multi-threaded decoder implementations for smoother playback performance.
• Tiling – VP9 introduces tiles, independently coded and decodable sub-units of video frame, enabling significant speed-ups without introducing additional latency.
• Alternated Resolution Reference Frames – flexible features, such as 8 reference frame buffers and four different entropy coding contexts, enable encode/decoders to implement various forms of coarse-grained scalability.

According to Naveed Altaf A in VP9 Codec – What is VP9?, May 2022, like VP8 but improved, VP9 uses block-based motion estimation for higher compression performance, has improved rate control, and supports High Dynamic Range (HDR) and 10-bit color depth. “The VP9 codec is similar to the H.265 (HEVC_family) process in terms of its working and supports parallel processing. It reduces the bitrate to half of the original without compromising on the quality...The VP9 codec works by compressing the initial raw video file using an algorithm into half the original size which makes it fit for seamless transmission over the internet.” Wikidata has a helpful comparison of compression efficiency between HEVC/H.265 and VP9 based on subjective assessments.

The WebM Project’s VP9 Coding Profiles page lists the VP9 4 variants, or coding profiles, that define the tools used during encoding. Profile 0 is the basic variant (default), the least demanding from hardware implementation.

• Profile 0
  • Color Depth: 8-bit/sample; Chroma Sampling: 4:2:0
• Profile 1
  • Color Depth: 8-bit; Chroma Sampling: 4:2:2, 4:4:4
• Profile 2
  • Color Depth: 10 or 12-bit; Chroma Sampling: 4:2:0
• Profile 3
  • Color Depth: 10 or 12-bit; Chroma Sampling: 4:2:2, 4:4:4

The WebM Project’s VP9 Levels and Decoder Testing page defines 14 levels, limiting various encoding configuration options, for VP9 Profile 0 (8-bit) and Profile 2 (10-bit). Layers are coded with resolutions, frame rates, and bitstreams.

Levels listed: Max Luma Sample Rate, Max Luma Picture Size, Max Bitrate, Max CPB Size, Min Compression Ratio, Max Number of Tiles, Min Alt-Ref Distance, Max Reference Frames, Max Width and Height for Luma Picture, and Example Frame Size and Display Rate.

Google Groups Conversations user, James Zern, describes profiles and levels, stating “Profiles and levels are just a description of the properties of the particular bitstream, they don’t affect the decode process, they’re used to constrain the bitstream at encode time. In VP9’s case this information is transmitted out of band, e.g., via the container or dash manifest.”

Uses of VP9:

According to James Konik in The VP9 Codec: How Does It Stack Up?, June 2021, in the BitMovin.com blog post, “VP9 outperforms HEVC at resolutions above HD. That makes it a great choice if you’re working with video at different sizes, or if you’re focusing specifically on HD...Thanks to widespread hardware support, it gives smoother playback on PCs in most scenarios...Safari has added support for the codec, making it the clear choice for high-resolution video in the macOS version of browser.”

In the TuBeast.com’s article, Stream Faster and More Efficiently with VP9 – YouTube's Codec, January 2023, Tommy L. writes, VP9 is “primarily used for streaming video content on YouTube, and is designed to improve video streaming performance, quality, and reduce bandwidth usage. VP9 is also referred to as “YouTube’s Codec” because of its exclusive use on the platform.”

Open source, royalty free format.

The WebM Project states, “VP9, the WebM Project's next-generation open video codec, became available on June 17, 2013.”
“Google does not demand royalty payments for the use of codecs WebM supports (VP8/VP9 video).”

WebMProject.org VP9 Video Code - summarizes VP9 topics relating to WebM Project.

Draft VP9 Bitstream and Decoding Process Specification Version 0.6, Last Modified 03- 27-2017. Note: Specification is not final.

Related Documents: RTP Payload Format for VP9 Video

• VP Codec ISO Media File Format Binding
• VP9 Profiles and Levels
  • VP9 Levels and Decoder Testing
  • VP9 Coding Profiles
• HDR10+ Metadata Handling
• Acquiring VP9 (libvpx)
• WebM Wiki - User-Contributed Pages
  • VP9 Tips
  • VP9 Known Issues

Jan Ozer of Wowza.com, in VP9 Codec: Google’s Open-Source Technology Explained, 2021, wrote, “In terms of usage, by far the largest distributor of VP9-encoded content was YouTube, though YouTube still encodes the vast bulk of its files with H.264. In 2016, Netflix announced that it was using VP9 for video downloads, while JW Player distributes on-demand video in its online video platform using H.264 and VP9.”

Wikipedia endorses multimedia formats that are open and royalty free, as such since 2016 Wikipedia accepts VP8 and VP9 video formats.

As stated at VP9 Wikipedia, VP9 is supported in the following browsers: Chromium, Google Chrome, Opera, Firefox, Microsoft Edge, and Safari. “In March 2016 an estimated 65 to 75% of browsers in use on desktop and notebook systems were able to play VP9 videos in HTML5 webpages.”

Up-to-date browser support for VP9, “AV1 Video Format, meant to succeed its predecessor VP9” on CanIUse.com.

Operating systems Microsoft Windows, macOS, Linux, Android OS, and iOS support the VP9 codec.

Media Player software support, including VLC, MPlayer, Kodi, MythTV, and FFplay.

Comments welcome.

According to VP9 Wikipedia, “March 2019, Luxembourg-based Sisvel announced the formation of patent pools for VP9 and AV1.”

The WebM Project and Google are aware of the Sisvel Video Coding Licensing Program patent pools, stating “While open source, royalty free and/or no charge technologies are widely offered, there is no way to prevent third parties from demanding licensing fees on these types of technologies."

According to Phil Cluff in Did Sisvel AOM with Their Patents Down?, 2019, “Sisvel International S.A. announced two new patent pools, one for the VP9 video codec and one for the AV1 video codec...This pool seems to cover any device which houses a hardware decoder for either VP9 and AV1...Both AV1 and VP9 are marketed as ‘Royalty Free’ codecs by the Alliance for Open Media (AOM) and Google respectively, but it sees, that Sisvel doesn’t agree with statement...Their video specifically flags AV1 and VP9 as "FREE?" codecs, saying they still contain approaches based on patent encumbered technologies.”

Sisvel Supporting Links:

• About VP9
• VP9 License Terms
• VP9 Patent List

As stated by Jan Ozer in VP9 Codec: Google’s Open-Source Technology Explained, 2021, “Sisvel launched a patent pool for VP9 (and AV1) in March 2019 seeking royalties on consumer devices but not VP-encoded content...Since Sisvel isn’t seeking royalties on content, the pool should have minimal impact on streaming producers considering publishing with the VP9 codec.”

Comments welcome.

Depends upon algorithms and tools to read.

WebM Editors per Digiarty Software, Inc.:

• VideoProc Converter – simple interface with instant editing, supports both VP8 and VP9 codecs.
• OpenShot Video Editor – free, open-source editor, supports WebM VP9.

Encoders:

• The WebM Project VP9 Codec Implementation – libvpx
• Intel SVT VP9 Encoder
• Two Orioles Eve-VP9 Encoder

Decoders:

• The WebM Project VP9 Codec Implementation – libvpx
• FFmpeg – multimedia framework

System-on-a-Chip (SoC) manufacturers supporting VP9 list.

Comments welcome.

Supports the inclusion of metadata.

Per WebMProject.org HDR10+ Metadata Handling, “HDR10+ metadata can be specified in the form of ITU-T T.35 terminal codes. See the BlockAddID element in the WebM Container Guidelines. ATSC 2094-40/CTA-861.4 define one of the possible ways to specify HDR10+ metadata.”

• HDR10+ - adds dynamic metadata to HDR10 source files.
• BlockAddID – identifies BlockAdditional level, “For VP9, 0x01 is reserved and 0x04 indicates ITU T.35 metadata as defined by ITU-T T.35 terminal codes.”
• ITU-T T.35 Terminal Codes – three types: country code, terminal provider code or manufacturer code), and terminal provider oriented code. Per T.35, country codes are defined per recommendation and the terminal provider oriented codes are defined by the manufacturer.
• ATSC 2094-40/CTA-861.4 - describes technology from SMPTE ST 2094-40, Dynamic Metadata for Color Volume Transform – Application #4, the technology used for dynamic metadata for HDR content.

Elecard.com’s page, HDR in Depth, April 2022, provides an overview of Standard Dynamic Range (SDR) and High Dynamic Range (HDR), the main HDR standards, and validation for video formats, VP9, among others.

• The HDR10 standard “employs static metadata that is applied to entire video sequence...Metadata includes mastering display colour volume and content light level information...VP9 carries data at the media container level: MKV/WebM: SmDm (Mastering Display Metadata) and CoLL (Content Light Level); MP4: mdcv and clli boxes.”
• The HDR10+ standard “uses dynamic metadata which allows more efficient editing...Backward compatibility with HDR10...In VP9, metadata are specified in BlockAddID of the WebM container.”

For more information, visit Telestream’s HDR10 Metadata and HDR10+ Metadata document pages.

Comments welcome.

None beyond availability of supporting software/hardware.

Comments welcome.

WebMProject.orgVP Codec ISO Media File Format Binding provides information on “ISO Base Media Track for video encoded with Video Partition structured video codecs ("VP"),” listing ISO/IEC 23001-7—Common encryption in ISO base media file format files as a Normative Reference. Common Encryption for VP video streams is specified, “If the VP9 data is encrypted, the Protection Scheme Info box ('sinf') SHALL be present, and SHALL contain a Scheme Type ('schm') box. The scheme_type field of the 'schm' box SHALL be 'cenc', indicating that AES-CTR encryption is used when samples are encrypted.”

IANA.org, VP9 Security Considerations: See section 8 of RFC-IETF-Payload-VP9-16.

Per RFC-IETF-Payload-VP9-16, Section 8, Security Considerations,

“RTP packets using the payload format defined in this specification are subject to the security considerations discussed in the RTP specification [RFC3550]...It is not an RTP payload format's responsibility to discuss or mandate what solutions are used to meet the basic security goals like confidentiality, integrity and source authenticity for RTP in general. This responsibility lays on anyone using RTP in an application... Applications SHOULD use one or more appropriate strong security mechanisms... Implementations of this RTP payload format need to take appropriate security considerations into account.”

Comments welcome.

Good support. WebMProject.org, “VP9 uses 16 bits for width and height, for a maximum resolution for 65536 x 65536 pixels. Neither VP9 nor VP8 place any restriction on framerate or datarate.”

FFmpeg and VP9 Encoding Guide, last updated 6 months ago, “VP9 offer different rate control modes, which determine the quality and file size...libvpx-vp9 offers a simple variable bitrate (VBR) mode by default.” VBR mode tries to reach the specified bit rate on average, higher bit rate, better quality.

According to Sreejata Basu in Muvi.com’s blog, Decoding the VP9 Codec, October 2021, “VP9 also has widespread hardware support ensuring smoother playback on PCs in most scenarios.”

Comments welcome.

Good support. Also stated by Sreejata Basu in the blog, Decoding the VP9 Codec, October 2021, “VP9 is a great choice for delivering Over-The-Top (OTT) video on desktop as it works smoothly on PCs at high resolutions. It can also save a good amount of bandwidth while delivering the same or better quality...The better support for HD+ and 360-degree videos also make it a great choice for delivering high quality video.”

Per Developers.Google.com, Recommended Setting for VOD, “VP9 supports a range of frame sizes, from tine resolutions through 4K. Larger frame sizes give higher quality but they require more bandwidth to deliver, and more processing power to decode.”

Comments welcome.

Some support, scalability. According to the RTP Payload Format for VP9 Video, “The VP9 codec can maintain up to eight reference frames. It also allows a frame to use another frame of a different resolution as a reference frame, allowing internal resolution changes without requiring the use of key frames. Those features together enable VP9 to implement various forms of coarse-grained scalability, including temporal, spatial, and quality scalability modes.”

Comments welcome.

Naming Conventions/Considerations (2 different Codec Names found):

According to WebMProject.org, VP Codec ISO Media File Format Binding, the document “Specifies a general ISO Base Media track and sample format for video encoded with Video Partition structured video codecs ('VP'), such as MPEG VCB (MPEG-4 Part 31), VP8, VP9, and others”

Developer.Mozilla.org document, Web Video Codec Guide:

• Codec Name: VP8 (short); Full Codec Name – Video Processor 8; Container Support – 3GP, Ogg, WebM.
• Codec Name: VP9 (short); Full Codec Name – Video Processor 9; Container Support – MP4, Ogg, WebM.

External Links:

• 3GPP file format (3GP). See (https://portal.3gpp.org/#/).
• Off Vorbis Documentation. See (https://www.xiph.org/vorbis/doc/).
• WebM. See (https://www.webmproject.org/).
• MP4. See (https://www.iso.org/standard/39259.html).

Relationship to Opus Audio Codec:
Windows Blog, WebM, VP9 and Opus Support in Microsoft Edge, 2016, “Opus is the primary audio codec paired with VP9 in the WebM specification.”

VP9 was created by On2Technologies as TrueMotion VP9. It is the last official iteration of the TrueMotion series of video formats. According to Wikipedia, On2 Technologies was a small, publicly traded company that designed video codec technology. It was acquired by Google in February of 2010. On2’s VP8 technology became the core of Google’s WebM video file format. Google developed VP9 and it was released on June 17, 2013. According to Calvin Wankhede of androidauthority.com, Google developed VP9 as a free and open codec to avoid dealing with H.265's confusing royalty situation.