Sustainability of Digital Formats: Planning for Library of Congress Collections

Introduction | Sustainability Factors | Content Categories | Format Descriptions | Contact
Format Description Categories >> Browse Alphabetical List

HDF-EOS, Hierarchical Data Format-Earth Observing System (based on HDF4)

>> Back
Table of Contents
Format Description Properties Explanation of format description terms

Identification and description Explanation of format description terms

Full name Hierarchical Data Format - Earth Observing System (HDF-EOS), based on HDF Version 4
Description

HDF-EOS is a self-describing file format for transfer of various types of data between different machines based upon HDF4 (Hierarchical Data Format Version 4). The HDF-EOS data format is standard HDF with added conventions, data types, and metadata elements as specified for the Core System of the Earth Observing System Data and Information System (ECS for short). HDF-EOS adds three geolocation data types (point, grid, and swath) which allow the file contents to be queried by earth coordinates and time. An HDF-EOS file also supports the ECS core metadata standard which is essential for ECS search services. An HDF-EOS file can be read by any tool that processes standard HDF4 files, although HDF-EOS geolocation or temporal information generally is not accessible using standard HDF4 calls. A data product that includes ECS metadata is a valid HDF-EOS file, whether or not it employs any of the grid, point, or swath models.

HDF-EOS is an extension of HDF4 from The HDF Group (THG). Both HDF4 and HDF-EOS are essentially defined by the API calls available through a software library. As of the August 2010 release of HDF-EOS (HDF-EOS2.6), the programming languages supported are C (C and C++) and FORTRAN (F77 and F90). The library code is written in C, with an interface which allows the FORTRAN user to call the HDF-EOS routines via automatic conversion to C. The August 2010 release of HDF-EOS software uses version 4.2r5 of HDF4 software. HDF-EOS has ordinarily been updated for compatibility with new releases of HDF4.

In some contexts, the format and software tools are called HDF-EOS2, to distinguish from the later version HDF-EOS5 based on HDF5.

Production phase HDF-EOS is the official format for storing data collected from EOS satellites: Terra, Aqua and Aura.
Relationship to other formats
    Modification of HDF4, Hierarchical Data Format (HDF) version 4
    Has later version HDF-EOS5, based on HDF5, as opposed to HDF4. At this time, HDF-EOS5 is not separately described on this website.

Local use Explanation of format description terms

LC experience or existing holdings  
LC preference  

Sustainability factors Explanation of format description terms

Disclosure Fully documented. HDF-EOS was developed and is maintained by Raytheon under contract to EOSDIS Core Systems (ECS). Version 2.17 was released in August 2010.
    Documentation The HDF-EOS User's Guide for the ECS Project is the documentation for the format. It is available online in two parts: Volume 1: Overview and Examples (http://newsroom.gsfc.nasa.gov/sdptoolkit/docs/HDF-EOS_UG.pdf) and Volume 2: Function Reference Guide (http://newsroom.gsfc.nasa.gov/sdptoolkit/docs/HDF-EOS_REF.pdf). Documentation associated with software releases is available from ftp://edhs1.gsfc.nasa.gov/edhs/hdfeos/.
Adoption

HDF is the scientific data format standard selected by NASA as the baseline standard for EOS. HDF-EOS is the official data format for EOSDIS data production and archive. HDF-EOS is the official format to store data collected from EOS satellites: Terra, Aqua and Aura. The ASTER and MODIS data products archived at the Land Processes Distributed Active Archive Center (LP DAAC) are in HDF-EOS format.

An increasing number of software programs for data viewing and analysis can use files in HDF-EOS format. See, for example, Applications that Read HDF-EOS from The HDF Group. The OPenDAP project supports HDF-EOS data access through the HDF4 handler in its Hyrax software. Matlab has routines to read HDF4 and HDF-EOS files, providing higher level functions than the native API. GDAL has a driver to import HDF4 and HDF-EOS files.

    Licensing and patents None
Transparency To quote from an April 2011 presentation on long-term preservation of HDF data, "The complex byte layout of HDF files makes long-term readability of HDF data dependent on long-term availability of HDF software." The authors suggest as a solution, the development of a mapping tool that can create a map of an HDF dataset in a simple structure that could be used independently to facilitate data access. See h4mapwriter.
Self-documentation Designed as a self-describing format.
External dependencies See Transparency, above.
Technical protection considerations  

Quality and functionality factors Explanation of format description terms


File type signifiers and format identifiers Explanation of format description terms

Tag Value Note
Filename extension hdf
hdfeos
he2
he4
These extensions are recognized by the NCAR Command Language for HDF-EOS files. See NCL page for HDF. Only .hdf is found in the usual online databases of file extensions.

Notes Explanation of format description terms

General  
History

Two earlier browsers for HDF-EOS, HE5View and EOSview, have not been supported since 2006. See HDFView HDF-EOS Plug-in Goals. Their functionality, with enhancements, has been put into an HDF-EOS plugin for the more general HDFView browser.


Format specifications Explanation of format description terms


Useful references

URLs


Last Updated: Wednesday, 22-Feb-2017 13:45:58 EST