Format Description Categories
Browse Alphabetical List
ARC_IA, Internet Archive ARC file format
Format Description Properties
- ID: fdd000235
- Short name: ARC_IA
- Content categories:
- Format Category:
- Other facets:
- Last significant FDD update:
- Draft status: Partial
Identification and description
||ARC_IA, Internet Archive ARC file format.
||Specifies a method for combining multiple digital resources into an aggregate archival file together with related information, used since 1996 by the Internet Archive to store 'web crawls' as sequences of content blocks harvested from the World Wide Web.
||Used for web-accessible content in archived state, representing the final form disseminated in final state over the web to a user agent (web browser).
|Relationship to other formats
||Data of various types, for example, HTML pages, images as GIF, JPEG, etc.
Has later version
Web ARChive file format
|LC experience or existing holdings
||LC has substantial volumes of captured web sites in the ARC_IA format. See http://www.loc.gov/webarchiving/technical.html
LC's preferred format for harvested Web sites harvested in bulk is now WARC. The Library of Congress Recommended Formats Statement (RFS) 2016-2017 includes ARC as an acceptable format for websites.
||Developed by the Internet Archive (Brewster Kahle). Documentation and tools to use files in the format freely available.
||Described at http://www.archive.org/web/researcher/ArcFileFormat.php
||The file format developed for the Heritrix web crawler, supported by the International Internet Preservation Consortium.
| Licensing and patents
||The wrapper is transparent; contained data varies.
||In the ARC files containing the actual archived "documents" (html, gif, jpeg, ps, etc.) each document is preceded by some header information about the document: the document file format, the document size, outward links that the document contains, etc. At the Internet Archive, each ARC file has a corresponding DAT file that contains only the header information.
||User access depends on large-scale indexing of a corpus of ARC files or a separate copy of the record headers (e.g. Internet Archive DAT files). Indexing the DAT files can support user access by URL and date, as in the Wayback Machine.
|Technical protection considerations
Quality and functionality factors
||Supported through Internet Archive's Wayback Machine or equivalent tool.
|Documentation of harvesting context
||Allows for basic information about the time of harvesting, the IP address of the harvesting machine, Internet Media Type (MIME type) and response code for the harvest transaction, etc.
|Efficiency at scale
||Excellent for efficient bulk harvesting and efficient indexing for access by URL and date. The use of coordinated ARC and DAT files is one way to support efficient indexing for such access.
|Support for stewardship.
||The capabilities in ARC that support long-term management of a corpus of web archive files is basic. WARC was developed as an extension to ARC, in part to provide better capabilities for managing Web archives for the long term. See
Web Sites and Pages: Quality and Functionality Factors.
File type signifiers and format identifiers
||ARC files are not typically transmitted to users or used in ways that depend on recognition by file type.
|Wikidata Title ID
Last Updated: Tuesday, 28-Feb-2017 14:16:21 EST