Sustainability of Digital Formats: Planning for Library of Congress Collections

Introduction | Sustainability Factors | Content Categories | Format Descriptions | Contact
Format Description Categories >> Browse Alphabetical List

Reflectance Transformation Imaging (RTI) File Format

>> Back
Table of Contents
Format Description Properties Explanation of format description terms

Identification and description Explanation of format description terms

Full name Reflectance Transformation Imaging (RTI) File Format
Description

The RTI image file format is a file format used for output from a process of Reflectance Transformation Imaging, a method of computational photography. The format was developed under an IMLS National Leadership grant (LG-25-06-0107-06) to the University of Southern California (USC) and Cultural Heritage Imaging (CHI). The project was titled "Developing Advanced Technologies for the Imaging of Cultural Heritage Objects." The Visual Computing Lab of ISTI-CNR [VCL ISTI-CNR] in Pisa, Italy was a sub-contractor responsible for developing the RTI file format and an associated viewer. The RTI file format was designed as a more general successor format to PTM, which was named for its first application (polynomial texture mapping).

RTI and PTM image files are both created from multiple digital photographs of a subject shot from a stationary camera position. In each photograph, light is projected from a different known, or knowable, direction. See Notes below for two popular methods for generating a controlled set of captures. The result is a series of images of the same subject with varying highlights and shadows. Lighting information from the images is mathematically manipulated to fit a model of the surface to the data in the images. A primary objective is to enable a user to (virtually) re-light the RTI or PTM image interactively and examine its surface on a screen. In addition, the image files can be used to create enhanced visualizations. The PTM format (with file extension .ptm) is based on fitting a binomial quadratic equation to the light data for each pixel. The RTI format (with file extension .rti) indicates support for a variety of additional fitting methods, currently including Spherical Harmonics (SH), Hemi-Spherical Harmonics (HSH), and Adaptive Basis PTM. All these methods derive a per-pixel reflectance model represented by a set of coefficients that define the fitted function that takes into account the direction of the lighting. The stated concept motivating the RTI specification is that it can be "universal" and support any similar basis for encoding the "reflectance properties of the depicted surface." However, as of early 2018, RTI Builder appears only to support PTM and HSH. The curious or mathematically inclined will find comparisons of PTM and HSH and links related to the underlying mathematics for HSH in Notes and Useful References, below.

The specification for the RTI File format, available from the CHI Forum, is titled "RTI Format - Draft 0.9." The authors are listed as Massimiliano Corsini and Prabath Gunawardane of the Visual Computing Lab of ISTI-CNR [VCL ISTI-CNR] in Pisa, Italy. It is dated December 2008, with an update in December 2010 by Carla Schroer of CHI. An RTI file has three main components:

  • Header: This component is in plain text lines, terminated with CR LF (0x0D 0x0a). Comment lines, beginning with #, may precede the primary content of the header. The primary content of the header consists of three lines with numeric values that function as structural metadata that will permit an application to interpret the Data component appropriately. A sample header is

    #HSH1.2
    3
    1908 3148 3
    4 2 1

    • First in the example is a single comment indicating the fitting model (HSH) and version (1.2).
    • Line 1 contains a single integer, a code indicating the fitting model used for the RTI image (RTI_Type). The specification lists four values: 1 (PTM); 2 (SH); 3 (HSH); and 4 (Adaptive Basis).
    • Line 2 contains three integers defining: image width in pixels; image height in pixels; and number of color channels (e.g., 3 for RGB, 1 for grayscale). This last item is called color_dimensions.
    • Line 3 contains three integers, defining: number of coefficients (basis_terms) in the fitting function (e.g., 6 for PTM, 9 for a second order HSH); code indicating whether coefficients are calculated for a single composite luminance value (1) or independently for each color channel (2); data element size in bytes, typically 1, 2, or 4.

    If RTI_Type is "Adaptive Basis," there will be additional lines, one per color channel. The compilers of this resource have not encountered explicit use of this feature. Comments welcome.

  • Data: All image data is stored in little-endian binary format as a contiguous block without separators. The main data is preceded by floating point values for scale and bias for each coefficient (basis term). Assuming a value of N for basis_terms, there will be N scale values followed by N bias values. The actual value for a coefficient is calculated from the stored coefficient by Cactual = Cstored*scale + bias. This technique is commonly used in graphics applications to optimize computational precision for pixel data given a constrained size for data elements, and is also used in the PTM format. The pixel data is stored in row order, with basis_terms*color-dimensions elements for each pixel.
  • Metadata: The last component is a single XMP packet. The specification suggests element sets for basic and extended metadata, but also has a note indicating that the metadata section of the specification needs to be modified to take into account the entities/properties of the CIDOC Conceptual Reference Model."

Specialized viewing applications, such as RTIViewer or the ISF Standalone Viewer (developed for the InscriptiFact project at USC), offer the capability to change the virtual light source or rotate the virtual object, revealing more detail than possible using traditional photography.

Production phase An RTI image file is created from multiple still images as a file designed for the end user to view interactively with the capability to adjust the virtual lighting source, and to apply filters that permit more detailed visualization and non-destructive analysis than working with the original object. Not intended as an archival format. Cultural Heritage Imaging, a non-profit organization providing support and training for various computational photography techniques, states "Photogrammetry is archive friendly. Strictly speaking, all of the 3D information required to build a scaled, virtual, textured 3D representation is contained in the 2D photos present in a well-designed photogrammetric capture set." They recommend retaining the images in the capture set together with information about the capture configuration. This statement and approach applies equally well to Polynomial Texture Mapping or Reflectance Transformation Imaging. In all cases, as software improves and computational power available increases, even better images or models should be derivable from the same source images.
Relationship to other formats
    Affinity to PTM, Polynomial Texture Map (PTM) File Format

Local use Explanation of format description terms

LC experience or existing holdings No direct experience.
LC preference The Library of Congress Recommended Formats Statement (RFS) includes RTI as an acceptable format for Scanned 3D Objects (output from photogrammetry scanning.

Sustainability factors Explanation of format description terms

Disclosure A specification for the RTI format has been developed under the auspices of Cultural Heritage Imaging (CHI), a non-profit corporation that describes itself as "dedicated to advancing the state of the art of digital capture and documentation of the world’s cultural, historic, and artistic treasures." This specification was originally written by members of Visual Computing Lab (VCL) of CNR-ISTI. VCL developed an viewer for RTI files with support from CHI and the University of Southern California from a 2006 grant from the US Institute of Museum and Library Services (IMLS) National Leadership Grant Program (Award Number LG-25-06-010706).
    Documentation

A link to a draft specification (last updated in 2010) for the RTI Format is posted in the CHI forum under Where can I find the file format specifications for RTI and PTM?. The link to RTI Format - Draft 0.9 will yield a request to sign in or register. Registration is free.

An explanatory note in the posting says that it is "marked 'draft' because it contemplates a .xmp structure embedded in the file structure, and that part was still under discussion when this document was created. The format for the RTI data is final, and this is what is used by the RTIViewer and other viewers that support this format."

Adoption

The RTI format is supported by the same organizations and tools that support the PTM format. The widely used RTIBuilder software produces PTM format files if the PTM fitter is used and RTI format files if the HSH fitter is used. The format is used in the same cultural heritage domains as PTM, especially those for which the HSH fitting model has demonstrated a particular advantage.

Examples of domains for which HSH has advantages include:

In the abstract for his 2016 tutorial on Reflectance Transformation Imaging (RTI) for Cultural Heritage Artefacts, Konstantinos Papadopoulos stated, "The method has been extensively used on a wide range of cultural objects, such as inscriptions, manuscripts, rock art, paintings, numismatics and any possible material including metal, stone, leather, paper, wax, bone, and clay." Resources that provide examples of the variety of cultural heritage applications of RTI, include: videos and lists of presentations from Illumination of Material Culture: A Symposium on Computational Photography and Reflectance Transformation Imaging (RTI)", a two-day symposium hosted by the Metropolitan Museum of Art in March 2017; and Highlight-Reflectance Transformation Imaging (H-RTI) for Heritage Applications (2016) from the Colour and Space in Cultural Heritage (COSCH) project (2012-16)

An important development leading to widespread adoption of the RTI imaging method and file format in cultural heritage institutions is the introduction of open-source tools to deliver effective interactive viewing of re-lighting, etc. over the web. For more on webRTI, see Notes, below.

See also Adoption for PTM.

    Licensing and patents No concerns. Although CHI does not have a copyright statement on the specification, it encourages use of the format. Software for building and viewing RTI images is available from CHI as open source under the GNU public license (version 3). See Why Create and Use Open Source Software? Reflections from an imaging nonprofit, a blog post by Carla Schroer of CHI from April 2016.
Transparency The simple format is relatively transparent, with easily recognized divisions between the ASCII Header, the binary Data block, and an optional XMP metadata block. The ASCII Header contains information needed to parse the data block into its numerical elements. A simple program could present the numerical data elements in human-readable form.
Self-documentation The RTI file format is designed to include an XMP packet with metadata at the end of the file. CHI has developed a suite of tools known as the Digital Lab Notebook (DLN). To quote from CHI, "Digital Lab Notebook (DLN) is a term coined by CHI for a digital record of the means and circumstances used to generate a digital representation of an imaging subject. The downloadable DLN tools listed below simplify the collection of scientific, standards-based metadata." The DLN:CaptureContext tool helps the user record and group metadata related to the locations, institutions, imaging subjects, image rights, people, and equipment associated with a photographic data acquisition session. See also A Context Metadata Collection and Management Tool for Computational Photography Projects.
External dependencies None beyond software that can import and render this format. Not supported by regular raster image viewers or editors. Requires a specialized viewer.
Technical protection considerations The RTI file format has no internal support for encryption or other means of technical protection.

Quality and functionality factors Explanation of format description terms

Still Image
Normal rendering The RTI image format is not intended as a simple two-dimensional raster image. However, it is raster-based and can support panning and zooming. Simple 2D images can be derived for printing, to display in browsers, or to render or manipulate with widely used raster image tools.
Clarity (high image resolution) Good
Color maintenance No support for ICC color profiles.
Support for vector graphics, including graphic effects and typography No support for vector graphics.
Support for multispectral bands Some imaging applications using RTI have experimented with use of light outside the visible spectrum. See, for example, the Jubilees Palimpsest Project.
Functionality beyond normal rendering RTI images are intended to capture surface properties and support viewer control with varying virtual light sources. Characteristics that can be captured include approximately correct diffuse lighting for small-scale features and global effects like self-shadowing. The images are sometimes described as 2D+.

File type signifiers and format identifiers Explanation of format description terms

Tag Value Note
Filename extension rti
Used conventionally for files in this format. Another file format using the same extension is mentioned in several databases of file extensions: Real Tracker Instrument file. See, for example, .rti from file-extensions.org. According to http://fileformats.archiveteam.org/wiki/Real_Tracker_module, the Real Tracker application was an MS-DOS application and its website disappeared in 1999. Comments welcome.
Internet Media Type Not found.  Comments welcome.   
Magic numbers See note.  The header for an RTI file provides no opportunity for a magic number. The file may optionally start with a comment beginning with the # character. Files beginning with "#HSH" have been observed. Comments welcome.
Pronom PUID See note.  As of June 2018, PRONOM has no record for the RTI format.
Wikidata Title ID Q55020067
For Reflectance Transformation Imaging file format. See https://www.wikidata.org/wiki/Q55020067.

Notes Explanation of format description terms

General

RTI image capture methods: An RTI file is created from a sequence of digital images of the subject with a specific set of lighting angles. Two commonly used methods to produce the image sequence are described below. Either method allows specialized software, such as RTIBuilder, to construct an RTI file.

  • Highlight Method; An image-capture technique for Reflectance Transformation Image (RTI), in which you include an image of at least two reflective spheres in each shot. The reflection of the light source on the spheres enables the processing software to calculate the lighting angle for that image. This light source information is used during the generation of an RTI or algorithmic rendering image. The highlight method allows great flexibility in subject size and location. It involves manually moving a light source to different angles relative to an object (at a constant distance), and using a shiny reflective ball to determine the lighting angle. While this non-automated technique doesn't require much in the way of special equipment, it can be slow and often requires two people to perform. For an in-depth discussion, see the RTI Guide to Highlight Image Capture. For examples of application in different contexts and for different types of object, see Highlight-Reflectance Transformation Imaging (H-RTI) for Heritage Applications (2016) from the Colour and Space in Cultural Heritage (COSCH) project.
  • Light Array or Dome Method: An apparatus (typically dome-shaped) that has a set of lights at fixed positions. Control software flashes each light in a fixed sequence in order to produce the image sequence needed to produce an RTI file. Because each light is at a known position, the software can associate each image with the lighting angle used for it. The technique referred to as Dome-RTI involves putting the object of interest under a dome with multiple light sources on the inside, and a camera looking down through a hole at the object. An electronic controller turns lights on and off in sequence, and fires the camera shutter in sync with the lights. Because this can be fully automated, and because the lighting angles are always the same, obtaining the required set of images can be faster and easier than with the Highlight-RTI technique. See Affordable Reflectance Transformation Imaging Dome for instructions and specifications for a particular kit using the Dome-RTI approach.

Common objective for HSH and PTM image fitting models: The PTM and HSH models are two examples of Reflectance Transformation Imaging fitting methods, in which the distribution of light reflected from an object surface is modeled as functions of angles of lighting and viewing and of spectrum as captured by a device such as a camera. The term bidirectional reflectance distribution function (BRDF) is used to describe the way light from a single light source reflects off an opaque surface and is observed by a viewer or camera. The motivation behind both PTM and HSM was to provide a compact representation of an approximation to the actual BRDF for an object surface that would support real-time interpolation to simulate relighting the object surface from different points. In both cases, the model can be used to generate synthetic computer graphics or to build a digital reproduction of a real object surface from a set of images captured using methods such as those described above. PTM and HSH are both widely used in cultural heritage contexts.

Comparisons of HSH and PTM image fitting models/algorithms: Comparisons located by the compilers of this resource are primarily based on use of RTIBuilder software from CHI. See Reflectance Transformation Imaging: Guide to Highlight Image Processing Version 1.4 (manual for RTIBuilder version 2.0.2). RTIBuilder offers two "fitters" (PTM and HSH) and provides output in the PTM (.ptm) file format for the former and in the RTI (.rti) file format for the latter. RTIBuilder offers a choice of LRGB and RGB for the PTM fitter; see the description PTM for a discussion of these variants, which both use 6 coefficients for the fitted polynomials. RTIBuilder options for the HSH fitter are for polynomials of order 1, 2, and 3, with 4, 9, and 16 coefficients respectively. More coefficients provide more accuracy in fitting the surface geometry, but require more space for storage and more computational power for building and for viewing.

In HSH or PTM - How to choose the best fitter from the Cultural Heritage Imaging Forums, Carla Schroer of CHI stated in 2012, "we have found that the HSH approach is superior in terms of rendering more sculptural objects that have self shadowing - where in the ptm in specular enhancement mode you might get just a black area where there is a lot of shadowing, an HSH will have better data. Also HSH does a better job of rendering shiny material as shiny. PTMs are always matte. Sometimes a matte surface is easier to see what you are looking for - so like most things it is a trade-off. Finally, I will note that HSH files are generally bigger than lrgb ptms. PTMs use 6 coefficients of a bi-quadratic polynomial to describe the surface normal. Tom Malzbender wanted a compact format for this. In HSH, this number is determined by the "order" that you choose when you create it. a first order HSH uses 4, a 2nd order 9 and a third order 16. So this affects the file size of the result, but it's also why the HSH can render shiny surfaces more accurately." Later in the thread, she says of HSH, "its main disadvantage is that we don't have all the rendering modes in the viewer implemented for it." The RTIViewer software available for download in early 2018 still has fewer rendering modes for RTI files based on HSH.

RTI Files from Reflectance Transformation Imaging For Lithics echoes Schroer's assessment, "There are two kinds of RTI data files. *.ptm files are a fitting of a binomial quadratic equation to the light curve, while *.rti use a 2nd or 3rd order Legendre polynomial for the fit. *.rti files are a better fit, but there are more visualization modes available with *.ptm files."

In Measuring Changes in Cultural Heritage Objects with Reflectance Transform Imaging (2013), Marcello Manfredi et al, states, "There are two main ways to describe the reflectance function: using polynomial texture map (PTM) and using Hemispherical Harmonics (HSH). The HSH approach is superior in term of 3D rendering because PTM uses only 6 coefficients of a biquadratic polynomial to describe the surface normal while a third order HSH uses 16 coefficients." Their experience led them to state, "The aim of this research was the development of a reliable RTI imaging method for monitoring changes in cultural heritage objects. RTI provides detailed information on the geometry of the object surface. We showed that RTI can be reproducible and quantified to the level where it can be used as practical method for measuring change in objects. We found that computing surface normals with HSH is more reproducible than using the older PTM method, and is therefore more sensitive to changes."

In A New Quantitative Method for the Non-Invasive Documentation of Morphological Damage in Paintings Using RTI Surface Normals (2014), Manfredi et al. propose and demonstrate an RTI method for the non-invasive detection of color changes and morphological changes (cracks, holes, bends, etc.) in paintings, arguing that the HSH approach is superior to PTM for this application.

Links to the mathematics underlying the HSH fitting approach: The HSH methodology for rendering surfaces was first published as A Novel Hemispherical Basis for Accurate and Efficient Rendering in 2004 by Pascal Gautron et al. As part of work under a grant to CHI in 2007-2009, PhD students Prabath Gunawardane and Oliver Wang working with James Davis at University of California, Santa Cruz took the HSH method and developed a software module to fit the HSH model to image data from an RTI capture. There is no publication based on that particular activity (although the source code is available from CHI on request). The CHI Forum had a thread in 2014 with topic Scientific Method HSH vs. PTM in 2014. It included the question, "Do you have any papers regarding the hsh?" There were several expert responses with useful information, but no citation to a paper. The compilers of this resource have also found no official documentation for the underlying model and fitting algorithm employed in the HSH fitter in RTIBuilder. Comments welcome.

The compilers of this resource have identified some articles that include useful discussion of the mathematics of surface modelling and the HSH fitting method, including explanatory diagrams. Lindsay MacDonald's 2014 article, Colour and Directionality in Surface Reflectance, has a discussion of PTM and HSH, with a straightforward summary of the formulae for the coefficients used for HSH of order 1, 2 , and 3. Representation of Cultural Objects by Image Sets with Directional Illumination (2015), also by MacDonald, has a fuller discussion. Another author providing helpful explanations of reflectance modeling in general and HSH in particular is Shireen Elhabian. See her article On the Use of Hemispherical Harmonics for Modeling Images of Objects under Unknown Distant Illumination (2011) and a discussion of her research interests in Appearance Modeling. Many resources that discuss the issues of surface rendering and introduce the terminology are available online, for example The Phong Model and the concepts of Illumination Models and BRDF from Scratchapixel.

WebRTI: An important contribution to the adoption of RTI is the development of WebRTIViewer, a viewer requiring HTML5 and building on Web-GL, for high resolution RTI images based on PTM or HSH fitting models. For the web visualization, an RTI image must be preprocessed with the command line tool webGLRTIMaker, which creates a tiled multi-resolution format to mount on a web server. The tools also work with high-resolution images in widely used raster formats (JPEG, PNG, and TIFF).

The compilers of this resource have been unable to locate a specification document for the multi-resolution format produced by WebGLRTIMaker. Comments welcome. The webGLRTIMaker source code is available on github. An example with the webRTIviewer source code has a number of separate JPEG images for 15 tiles of the same pixel dimensions (1_1.jpg, 1_2.jpg, 1_3.jpg, 2_1.jpg, etc.) and a file named info.xml that holds structural metadata, including indication of tile arrangement and positions. In Storytelling of a Coin Collection by Means of RTI Images: the Case of the Simoneschi Collection in Palazzo Blu, Gianpaolo Palma, the main developer of webRTIviewer, states that webGLRTImaker "subdivides the RTI image in nine layers, one layer for each HSH coefficient, where the i-th layer contains the i-th coefficient of the three RGB colour channels. Then for each layer it creates a multiresolution quad-tree and produces a tile for each node of the tree. Finally, it saves each tile in a different JPG image. With this format to visualize a specific pixel, we need to load the nine JPG images that contain its HSH coefficients. The main advantage is the out-of-core loading of the node of the quad-tree making immediately available the interaction with the coin without awaiting the complete loading of the data. This means that at the beginning the user interacts with a low-resolution version of the coin, which is progressively refined as soon as the higher resolution data are loaded. The loading of the tiles at the different resolutions is guided by the zoom and pan operation of the user."

History

The widely used "dome" technique for constructing RTI images was developed by Tom Malzbender and Dan Gelb, research scientists at Hewlett-Packard Labs. A paper describing the first tools and methods, using was presented at SigGraph 2001: Polynomial Texture Maps. See also Polynomial Texture Mapping (PTM). In March 2017, Malzbender spoke at a symposium organized by CHI and the Metropolitan Museum of Art, "Illumination of Material Culture: A Symposium on Computational Photography and Reflectance Transformation Imaging (RTI)". His talk gave a brief history of RTI and presented some new possibilities. See video of Day 2: Afternoon Session at time 44.09.

The first distributed draft of the RTI format was dated August 27, 2008 as an output of the 2006 IMLS grant (Award Number LG-25-06-010706). Draft 0.9 was published in December 18, 2008 and updated in minor ways in December 2010 by CHI.


Format specifications Explanation of format description terms


Useful references

URLs


Last Updated: 08/10/2021