Sustainability of Digital Formats: Planning for Library of Congress Collections

The Library of Congress Recommended Formats Statement (RFS) lists MPEG-4 as an Acceptable viewing proxy format for Video - File-Based and Physical Media.

Open standard in that it is fully documented and disclosed. As with any ISO-sponsored project, any updates to the specification are done through the ISO process for such which includes funneling feedback through national members, such as ANSI in the case of the USA. This process is transparent in its procedure but because membership in national bodies is limited (for example, individuals are not eligible to join ANSI as members), it is not considered an open format. Moreover, the specification documents are paywalled.

Developed through ISO technical program ISO/IEC JTC 1/SC 29 for coding of audio, picture, multimedia and hypermedia information. The working group WG11 (for coding of moving pictures and audio) is also known as the Moving Picture Experts Group (MPEG). See the ISO Standards Catalogue for the list of standards published by ISO/IEC JTC1/SC29. See http://mpeg.chiariglione.org/standards/mpeg-4 for information specific to MPEG-4 and its many parts.

ISO/IEC 14496-14:2003. Information technology -- Coding of audio-visual objects -- Part 14: MP4 File Format.

The total documentation package for ISO/IEC 14496 is extensive; 17 parts have been published from 1998 to 2004, with more to come. See complete list of documents in Format specifications below.

MPEG-4 Visual, Systems, and Advanced Video Coding licensing is managed by MPEG LA LLC (https://www.mpegla.com/).  These licenses cover the manufacture and sale of devices or software and, for some content disseminators, levy fees according to number of end users or the extent of content delivered.  The arrangements are updated periodically; for example, in January 2005, MPEG LA announced that the patent portfolio had been expanded to cover the FRExt (Fidelity Range Extensions) associated with MPEG-4_AVC and ITU H.264.

MPEG-4 Audio licensing is managed by Via Licensing Corporation (http://www.via-corp.com/licensing/index.html, link available through Internet Archive), an independent subsidiary of Dolby Laboratories. MPEG-4 Audio licensing appears to be limited to the manufacture of devices or software.

The inclusion of metadata of various types is a key element in MPEG-4. As indicated in the notes below, object and scene descriptions are required in order for MPEG-4 content to be presented.

Semantic description is carried by Object Content Information (OCI) descriptors and streams; the standard also permits the inclusion of MPEG-7 data, a separately standardized structure for metadata to support discovery and other purposes.

Depends upon encoding; the encodings used are generally lossy and provide moderate to very good fidelity. See, for example, AAC_MP4, considered to be superior to MP3 (MPEG-2 layer 3 audio) at a given bit rate.

The MPEG-4 standard also provides support for other "natural" sound encodings, e.g., parametric coding (HILN or Harmonic and Individual Lines plus Noise) and CELP (Code Excited Linear Prediction) and other encodings for speech.  The standard also supports the synthesis of audio, and for what is called Synthetic-Natural Hybrid Coding (SNHC).  The presentation of these elements depends upon the use of AudioBIFs (Audio BInary Format for Scenes).  In 2005, the MPEG committee announced two additional audio capabilities: Audio Lossless coding (ALS; lossless compression of multi-channel sound using time-domain prediction and entropy coding) and Scalable to Lossless coding (SLS; a scalable enhancement layer is added to a lossy bitstream that extends the representation to lossless but which can be truncated at delivery time).  The compilers of this document do not know the degree to which any of these various elements may be implemented in practice.

The AAC_MP4 audio structure provides a capability of up to 48 main audio channels, 16 LFE (Low Frequency Encoding or Effects) channels, 16 overdub/multilingual channels, and 16 data streams.

SNHC [and other note-based or synthetic?] sound can be spatially presented using extensions of the concepts initially implemented in Virtual Reality Modeling Language (VRML).

Paraphrased from the former www.m4a.com site: MP4 can be used for MPEG 4 video files, combined video and audio files, or just plain MPEG 4 audio. M4A files contain only MPEG 4 Audio. Apple started using M4A to identify files unprotected by digital rights management; note that protected QTA_AAC files carry the M4P and and M4B (for bookmarkable files) extensions. Apple felt that MP4 was too general (video, video/audio, or audio) and might confuse some media players. Until recently, encoder and player software like Nero and Compact used .mp4 for audio files while WinAmp 5.02, Apple iTunes, and others used .m4a. Today, most audio software developers allow you to choose the file extension you prefer.

The Wikipedia article Apple Lossless (consulted November 2, 2012) reports that the m4a extension is used for files containing either AAC_MP4 or the Apple Lossless encoding, wrapped in the MPEG4_FF_2 (MPEG-4, version 2) file format.

The four file formats associated with the ISO/IEC 14496 family of specifications are:

• MP4_FF_1, "version 1" from Part 1 (2001)
• MP4_FF_2, "version 2," this document, from Part 14
• MP4_FF_AVCE, for Advanced Video Coding extensions, from Part 15
• MP4_XMT, "textual format" from Part 11

Version 2 is very similar to its predecessor MP4_FF_1 as both owe a debt to the QuickTime file format that preceded them. This lineage is shared with the supertype for MP4_FF_2, ISO_BMFF, defined in Parts 12 of both the MPEG-4 and JPEG 2000 standards.

The object-based design of MPEG-4 is characterized as follows in Fernando Pereira and Touradj Ebrahimi's The MPEG-4 Book (Upper Saddle River, NJ: IMSC Press, 2002): "MPEG-4 is an ISO/IES standard developed by MPEG for communicating interactive audiovisual scenes. The standard defines a set of tools that provide binary coded representation of individual audiovisual objects, text, graphics, and synthetic objects. The interactive behaviors of these objects and the way they are composed in space and time to form an MPEG-4 scene are dependent on the scene description, which is coded in a binary format known as binary format for scenes (BIFS) . . . . The audiovisual streams are defined as elementary streams (ESs) and managed according to the object descriptor (OD) framework . . . . In addition, the OD framework defines additional streams for object content information (OCI), MPEG-J [Java APIs], and intellectual property management and protection (IPMP)." (p. 188)

BIFS owes a debt to the Virtual Reality Modeling Language (VRML), even as it extends VRML's capabilities and employs binary encoding. Timing of elements in MPEG-4 is managed by a Synchronization Layer (SL). The delivery of MPEG-4 content is supported by the Delivery Multimedia Framework or DMIF and its application interface.

MPEG-J is described in Part 1 of the standard (ISO/IEC 14496-1:2004). This API for the interoperation of MPEG-4 media players with Java code is  contrasted with a conventional parametric system.  "By combining MPEG-4 media and safe executable code, content creators may embed complex control and data processing mechanisms with their media data to intelligently manage the operation of the audio-visual session. The parametric MPEG-4 System forms the Presentation Engine while the MPEG-J subsystem controlling the Presentation Engine forms the Application Engine.  The Java application is delivered as a separate elementary stream to the MPEG-4 terminal. There it will be directed to the MPEG-J run time environment, from where the MPEG-J program will have access to the various components and required data of the MPEG-4 player to control it." (p. xii)