Sustainability of Digital Formats: Planning for Library of Congress Collections |
|
Introduction | Sustainability Factors | Content Categories | Format Descriptions | Contact |
Full name | Flexible Image Transport System (FITS), Version 3.0 |
---|---|
Description |
Flexible Image Transport System (FITS) was initially developed by astronomers in the USA and Europe in the late 1970s to serve the interchange of data between observatories and was brought under the auspices of the International Astronomical Union in 1982. In 2012, FITS is still in widespread use as a data interchange and archiving format by astronomers. FITS is a file format designed to store, transmit, and manipulate scientific images and associated data. The term "image" in the standard's name is loosely applied and FITS files often contain only non-image data. Astronomers view their images as data for analysis rather than simply as pictures to look at. From its beginning, FITS was seen as a transport format for more than a still image intended for printing or viewing in two dimensions. FITS was designed to facilitate the unambiguous transmission of n-dimensional regularly spaced data arrays, an n-cube. These multi-dimensional arrays may be 1-D spectra, 2-D images or data cubes of three or more dimensions. Two-dimensional tables containing rows and columns of data can also be stored in a FITS file. The substantial use of FITS for data that even astronomers would not class as image data led to consideration to change the full name for the FITS format, but the proposal was rejected. For the same reason, in this resource, FITS is categorized primarily as a dataset format, with use for image data as secondary. When FITS is used for straightforward image data, it is often (but not always) in one of two forms: two-dimensional (X, Y) with single values at each point, perhaps some measurement of intensity in a particular spectral band (comparable to grayscale); or three-dimensional (X, Y, color), with the third axis representing different points on a spectral axis, as measured by wavelength, frequency, or some other appropriate scale. There is no assumption in astronomical analysis or in FITS of a defined colorspace (such as RGB). Astronomy images are often rendered in false color in order to make infra-red and ultra-violet radiation visible [see Notes below]. A dimension of the n-cube may also represent time steps, or indeed be used to represent other data characteristics. FITS is also often used to store non-image data, such as spectra, photon lists, data cubes, or even structured data such as multi-table databases. FITS includes many provisions for describing photometric and spatial calibration details, together with image origin metadata. FITS was designed with an eye towards long-term archival use, and the maxim "once FITS, forever FITS" (or "once FITS, always FITS") has guided the IAU to ensure that the format is backwards compatible as new features are added. Version 3 (published in 2012 and distinguished in this format resource as FITS_3) documented several features approved since the previous edition of the full specification: support for 64-bit integers (from an earlier maximum of 32 bits) in images and tables; tables with variable-length arrays; and adopted new World Coordinate Systems (WCS) conventions to map an element in a data array to standard physical coordinates on the sky. WCS conventions also exist to define mappings for a spectral coordinate into a standard frame of reference, allowing spectral shifts due to the Doppler effect and/or cosmic expansion to be accounted for. The WCS conventions are implemented through an extensive set of reserved keywords and values to be used in headers associated with image arrays. A WCS convention for specifying time and relative dimensions in space is in preparation as of early 2013. A FITS file consists of one or more Header + Data Units (HDUs), where the first HDU is called the primary HDU, or primary array. The primary array may be empty or contain an N-dimensional array of pixels, such as a 1-D spectrum, a 2-D image, or a 3-D data cube. Data types supported in the primary array in FITS_3 are: unsigned 8-bit bytes, 16, 32, and 64-bit signed integers, and 32 and 64-bit single or double precision floating point reals (using the ANSI/IEEE-754 standard, approved by IEEE in 1985). FITS_3 documented support for 16 and 32-bit unsigned integers. Any number of additional HDUs may follow the primary array; these additional HDUs are called FITS extensions. A FITS file may have an empty primary array and comprise only extensions. Three standard types of extensions have been included in the FITS Standard since the mid-1990s:
For more detail on the form of HDUs, see Notes, below. FITS is maintained by the International Astronomical Union (IAU) FITS Working Group. Version 3 of the specification document incorporates incremental changes made in earlier years, was approved in 2008, and published formally in 2010. Note that the "once FITS, forever FITS" maxim has led the working group to avoid explicit identification of a version within a FITS file; thus versioning is not considered "strong." |
Production phase | Used for interchange between observatories (middle phase) and archiving (final phase). |
Relationship to other formats | |
Has earlier version | Several earlier versions, starting from 1993, not described separately on this website. |
Has later version | FITS 4.0 was adopted in July 2016, and formally released after editing for language in August 2018. Not described separately on this website. |
LC experience or existing holdings | The Library of Congress has a small amount of FITS files in its collections. |
---|---|
LC preference | The Library of Congress has not collected astronomical images in formats intended for analysis. For derived images converted to a visible colorspace and intended for display or printing, the Library prefers the TIFF_UNC or TIFF_LZW formats for raster images. See the Recommended Formats Statement for the Library of Congress format preferences for Still Image Works. |
Disclosure | Openly documented, non-proprietary data format, developed and maintained under the auspices of the International Astronomical Union (IAU), by the IAU FITS Working Group. |
---|---|
Documentation | Available from https://fits.gsfc.nasa.gov/fits_standard.html through the FITS Support Office hosted at NASA's Goddard Space Flight Center. |
Adoption |
The FITS file format is a well-established format for exchange and archiving of astronomical data. It is supported by the International Astronomical Union, NASA, and other national and international bodies engaged in astronomy and related fields. Data from major international programs, such as the Hubble Telescope, the Sloan Digital Sky Survey, and the orbiting Chandra X-ray Observatory are released in the FITS format. So are records from land-based observatories, such as the National Optical Astronomy Observatory (NOAO), the National Radio Astronomy Observatory (NRAO), and the Keck Observatory. In Europe, the land-based European Southern Observatory (ESO) and recent missions of the European Space Agency (ESA) release their data in the format. Software libraries for reading and writing FITS files exists in many programming languages. See https://fits.gsfc.nasa.gov/fits_libraries.html. An important example is CFITSIO, a library of subroutines in C and Fortran maintained by NASA's High Energy Astrophysics Science Archive Research Center (HEASARC). Many other tools build on this library. The GDAL (Geospatial Data Abstraction Library) uses the CFITSIO library to support conversion among raster image formats, including FITS. Image viewers and format converters that support FITS are listed at https://fits.gsfc.nasa.gov/fits_viewer.html. Utilities also exist to convert FITS images to image formats for publication and display. Examples are FITS Liberator and STIFF. In March 2010, the Vatican Library announced that historical scans of ancient books and manuscripts in the Vatican's collections would be stored in the FITS format. A December 2011 press release from the European Space Agency similarly reported the Vatican Library's plan to take advantage of the long-lived format developed for astronomy. |
Licensing and patents | No issues. |
Transparency | Header information is human-readable ASCII. Includes dimensions, etc. for data units. |
Self-documentation | Optional keyword/value records in headers can be used to provide metadata. The FITS standard specifies some keywords and allows for user-provided keywords. COMMENT and HISTORY records can also be used to describe the data unit and its provenance. The standard provides no facility for embedding metadata using a recognized markup such as an XML Schema. |
External dependencies | None. Although it is convenient to use existing software libraries, the specification has enough detail for new software to be developed. |
Technical protection considerations | The FITS specification does not support encryption of data. |
Still Image | |
---|---|
Normal rendering |
The FITS format is primarily for use with astronomy images and/or data; it is not a format for raster images in general. Many FITS files have no image data, containing, for example a catalog of celestial objects. A useful description of FITS for an audience interested in the applicability of FITS for storing scanned images of cultural heritage materials, such as manuscripts, was given by Lucio Chiappetti at the 2012 European Week of Astronomy and Space Science [https://www.vatlib.it/moduli/Ammenti_EWASS2012.pdf] at a special session on long-term preservation of digital cultural resources. Astronomy images in the FITS format are typically not "normal" images in one important sense; they are ready for analysis, not ready for human viewing. In particular, the treatment of color, or rather the equivalent of color extended outside the visible spectrum, is very different from that in the raster image formats used for digital photography, such as TIFF. In the most natural "sky" images, the X,Y dimensions are celestial coordinates and the Z dimension is related to intensity (e.g., flux, or number of photons). In FITS there is no default colorspace or standard way to identify that one of the conventional colorspaces (such as sRGB) has been used. See Notes below for a discussion on false color for astronomy images. Utilities exist to convert FITS images into TIFF_6, BigTIFF, or other image formats for publication and display. Examples are FITS Liberator and STIFF. A list of FITS Image Viewers and Format Converters is made available by the FITS Support Office. In addition, the simplest FITS image files can be imported into a few general graphics applications (such as ImageMagick, GIMP, or Graphic Converter) and support panning, zooming, image manipulation, etc. in the same way as other raster image formats. |
Clarity (high image resolution) | FITS can be used to store the source sensor data or derived data at the fullest resolution available. |
Color maintenance | The color management mechanisms used for digital photographs and for printer support are not applicable to astronomy images. A short tutorial on astronomical image processing focusing on color is at https://esahubble.org/projects/fits_liberator/improc/. |
Support for vector graphics, including graphic effects and typography | No support is defined within the standard for vector graphics. |
Support for multispectral bands | Support for multispectral imagery is a basic function of the FITS format. |
Functionality beyond normal rendering | In addition to storing image data, FITS can store multidimensional data, tables of ASCII or binary data, and application-defined data units that follow certain rules. FITS_3 documented the format's support for variable-length arrays. |
Dataset | |
Normal functionality |
Data in a FITS file is required to be stored in big-endian byte order. Numeric data types supported within FITS_3 are: signed (two's complement) integers of 16, 32, and 64 bits; and unsigned 8-bit integers; floating point (32-bit) and double-length (64-bit) binary numbers in IEEE-754 (1988) Basic formats. Unsigned integers of 16, 32, or 64 bits (and 8-bit signed integers) can be stored by shifting into the range of the corresponding signed integer and using the BZERO keyword to indicate the shift that must be applied to restore the original value. Character data is stored in one-byte characters and limited to 7-bit ASCII. Compared to a general dataset format, such as DBF, DBase Table File Format, this set of supported data types is limited. FITS supports three basic data models, as exemplified in the specifications for the three extension types:
|
Support for software interfaces (APIs, etc.) | Several software libraries exist for reading, writing, and manipulating FITS files, in particular the CFITSIO Software Library, maintained by NASA's High Energy Astrophysics Science Archive Research Center (HEASARC). However, in contrast to NetCDF and HDF, the software libraries are not considered integral to the FITS standard. |
Data documentation (quality, provenance, etc.) | Understanding data quality and provenance is essential for use of astronomy data and astronomers and astrophysicists put effort into describing their data fully. This is achieved through exploiting the simple keyword/value form of header records, including a considerable set of reserved keywords and controlled lists of values, and the ability to add project-specific keywords and values. Community practice has also developed for making extensive use of the HISTORY and COMMENT records. |
Beyond normal functionality | In comparison to a generic dataset format, FITS has many features specifically designed for astronomy and astrophysics. These include many reserved keywords, with associated controlled value lists or ranges appropriate for astronomy data, and support for celestial and spectral coordinate frames of reference. |
Tag | Value | Note |
---|---|---|
Filename extension | fits |
This .fits extension is a matter of convention rather than a requirement. Other extensions have been used. |
Internet Media Type | image/fits |
A FITS file described with the media type image/fits should be principally intended to communicate the single data array in the Primary HDU. |
Internet Media Type | application/fits |
The application/fits media type is for use for all FITS files that cannot be usefully treated as simple images. A FITS file that could validly be given the image/fits media type may be categorized as application/fits |
Magic numbers | HEX: 53 49 4d 50 4c 45 ASCII: SIMPLE |
From File magic numbers originally at http://www.astro.keele.ac.uk/oldusers/rno/Computing/File_magic.html (now linked via Internet Archive). This magic number is technically less appropriate than the longer pattern registered with IANA (see below), since it would also identify files marked "SIMPLE=F", a convention sometimes used for files deviating deliberately from the standard and used locally for internal purposes. |
Magic numbers | HEX: 53 49 4d 50 4c 45 20 20 3d {20 bytes of Hex 20} 54 ASCII: SIMPLE {2 spaces} = {20 spaces} T |
As registered with IANA in RFC 4047. The PRONOM entry for FITS uses a longer signature based on more header records. |
Pronom PUID | x-fmt/383 |
See https://www.nationalarchives.gov.uk/PRONOM/x-fmt/383. |
Wikidata Title ID | Q1428303 |
See https://www.wikidata.org/wiki/Q1428303. |
General |
Form of FITS HDUs: The FITS format was first standardized in 1981 and terminology in the early specifications make clear its origin in an era of punched card images stored on magnetic tape. The standard has evolved gradually since then, with proposed changes published as journal articles with opportunity for feedback over an extensive period. However, 80-byte header records and other features of the format still reflect its early origin. Every HDU consists of a human-readable ASCII header unit followed by an optional data unit. Each header or data unit must be an exact multiple of 2880 bytes long. Unused space at the end is padded with zeroes or blanks as appropriate. Headers consist of 80-character keyword records which have the general form: KEYNAME = value / comment. Each header unit begins with a series of required keywords that specify the size and format of the following data unit. The required keywords may be followed by other optional keywords to describe various aspects of the data, such as the date and time of the observation. COMMENT or HISTORY keywords are also frequently added to further document the contents of the data file. The data unit, if present, immediately follows the last 2880-byte block in the header unit. False color: In astronomy, the data value(s) associated with a "pixel" are measurements of intensity in a spectral frame of reference that is usually very different from the conventional colorspaces used for printing or screen display. Depending on sensors and equipment design, an image may represent intensity in a single spectral band (which may be narrow or wide), or intensities in separate spectral bands that may vary from radio waves (infra-red) to gamma radiation (ultra-violet). To be viewed, the images must be transformed by manipulating (by stretching, squeezing, cropping, etc.) the spectral histogram into one in the visible range. The result is known as "false color." See the tutorial Chandra Images and False Color for a good explanation of why "false" color is needed and how astronomers use different color approaches to convey different aspects of what has been observed. [Note: Chandra is an orbiting X-ray observatory operated by the Smithsonian's Astrophysical Observatory at Harvard.] Application-specific conventions and extensibility: User communities, for example for sub-disciplines, have developed generic and application-specific conventions that refine the FITS standard for increased interoperability in specific contexts or for specific categories of astronomical data. The standard also defines a mechanism for adding conforming extensions to the file structure. |
---|---|
History |
The FITS data format was developed in the late 1970s for interchange of astronomical image data between observatories. In 1981, the FITS specification was published in Astronomy and Astrophysics Supplement, Vol. 44, pp. 363-370. By then, FITS had become the de facto standard interchange format for astronomy data. In the 1990s several versions of the specification were published by NOST (NASA Office of Science and Technology): NOST 100-1.0 (June 1993); NOST 100-1.1 (September 29 1995); NOST 100-1.2 (April 1998); and NOST 100-2.0 (March 1999). FITS Version 2.1b was published in December 2005. Version 3.0 was approved in July 2008, and formally published in 2010. Version 4.0 was adopted in July 2016, and formally released in August 2018 after editing for language, not substance. A complete summary of the FITS 4.0 language editing is also available from the FITS Standard Document site. All versions since 1993 are available at https://fits.gsfc.nasa.gov/fits_standard.html. The description of the specification for FITS Version 4.0 at https://fits.gsfc.nasa.gov/fits_standard.html indicates, "The main difference from the previous Version 3 of the document is that a number of FITS conventions are now officially recognized, including conventions for representing Time world coordinates, and conventions for compressing FITS images and tables. A complete list of all the changes can be found in appendices H.3 and H.4 of the document." See https://fits.gsfc.nasa.gov/fits_overview.html for more details of the early history. |
|