Adding an NDLP collection to the digital library

Outline of process -- seven phases

Notes on outline:

• In early 1996, the digital archive is implemented using a UNIX file-system. Over the next two years, prototype systems will be developed for a digital archive that supports features needed in the long term. These include control of access to an item according to relevant permissions and restrictions, support for managing and delivering items that incorporate other digital items, and the ability to generate archival digital copies for preservation of the digital collections. Notes in italics indicate where aspects of the process described here are not yet implemented but will apply in the longer term.
• The "outputs" listed here for a phase are primarily those that affect later steps in this process (Adding a collection to the digital library) and are common to all collections. For some collections there may be other outputs, relating to preservation or other forms of access to LC digital materials (e.g. through specialized systems in reading rooms of custodial divisions or on CD-ROM).

1 -- Select

The selection of collections for digital conversion for the National Digital Library Program takes into account the value of the collection to the nation, the current state of technology, and the readiness of the collection for scanning. A collection will be selected for inclusion in the digital conversion workstream depending on the interplay of these factors and a consensus for inclusion among the custodial division, the NDLP team, and the Library administration.

Output:
Collection selected for digitization, including a preliminary description of scope and approach.

2 -- Plan

Planning involves two general aspects:

• preparing a production workplan for applying preservation treatment and producing the digital reproductions of the selected items, including the choice of digital formats for archiving and presentation;
• designing the framework for presenting the collection and providing user access, selecting the type of access aid (i.e. bibliographic records, an online finding aid document with active links to individual items, or other aid), and determining the level of detail for the access aid (e.g. item-level, group-level, or a mix).

Output:

Production workplan, including:

• Preservation treatment plan
• Methods selected for digitizing original items
• Choice of digital formats for capture, archiving and presentation,
• Estimate of storage space required for digital collection,
• Description of and plan to produce access aid and other framework elements,
• Scheme for assigning "logical" names to items, for use in the access aid and repository storage, including reservation of URNs [once URNs are in use],
• Situation report on restrictions relating to copyright, publicity and privacy, and terms of gift to the Library.

3 -- Produce

The production phase involves:

• digital conversion of the items in the collection selected for digitization; typically this includes several steps:
  1. initial capture (often, this will consist of scanning to create an image set); typically executed onsite at the Library, generally by a specialist contractor,
  2. post-processing to create additional digital reproductions to serve access or archiving goals, typically executed offsite by specialist contractor, and
  3. initial quality review by contractor (or Library scanning crew) to ensure completeness and quality of capture;
• preparation of the access aids, such as MARC bibliographic records (typically skeletal group- or item-level records) or a finding aid marked up in SGML according to the proposed standard for Encoded Archival Descriptions.

Depending on the workplan, these two aspects of production may be performed in parallel or in sequence.

Output:

• Digital reproductions of collection items, named according to scheme established in planning phase; these will include both files that are used for end-user access and files that are archived for future needs.
• Access aid: either a set of MARC bibliographic records and/or an SGML finding aid. The finished access aid incorporates the logical names for reproductions to support active links to the items.
• Information required for digital archive, including minimal metadata and restriction information. [Content and form to be specified as part of design for digital archive.]
• Unless incorporated in the SGML finding aid, HTML-format files for the collection framework, such as:
  • a scope and content note, i.e. a description of the collection and its provenance;
  • special presentations highlighting certain aspects of the collection or displaying a sampling from its content;
  • structured browsing tools (beyond the principal access aid) specific to a collection.

4 -- Store in digital archive

This phase could be a routine procedure once the production phase is complete, or accomplished in batches as intermediate steps are completed for some parts of the collection. The approach taken will probably vary with the format of the original items (sound recordings, video recordings, photographs, maps, etc.), the proportion of production that is done by outside contractors, and the convenience of the custodial division(s). Tasks include:

• Store files for items in collection in directories and files as specified by naming scheme.
• Register items in URN handle-server. [Once in use]
• Deposit items in digital repository. [Once in use]

Output:
By the end of this phase, all digital reproductions of items in the collection will be stored ready for production access.

5 -- Enable access

This phase will involve a selection from the following tasks:

• Store files for access aids in directories and files as specified by naming scheme.
• Register document-style access aids in URN handle-server. [Once in use]
• Deposit document-style access aids in digital repository. [Once in use]
• Generate indexes for related MARC records
• Generate indexes for textual items within the collection
• Prepare any customized scripts associated with searching collection indexes and displaying results for the user
• Mount HTML pages for the WWW framework on the LCWEB server for test access
• Add relevant viewers to the supported configuration for workstations providing public World Wide Web access in reading rooms.
• Prepare collection-level MARC record for future inclusion in MUMS.

Output:
By the end of this phase, all aspects of the access framework, including indexes and document-style access aids, will be ready for testing.

6 -- Test and refine

This phase should involve as much participation as possible, from test users, both expert and novice, from subject specialists from the relevant collection divisions, and from specialists assigned to specific tasks such as proofreading, testing links, checking for consistency of style and terminology, and so on.

Output:
By the end of this phase, approval for release of the collection for public access will have been given by all relevant parties.

7 -- Release to public

Final technical tasks are:

• Move HTML pages for the WWW framework to the production area of the LCWEB service.
• Provide links to new collection from appropriate points in LCWEB structure.
• Add collection-level MARC record with an appropriate pointer in field 856 (or a local field) to the LC's MUMS catalog.

Continue for more detail on each phase:

• Select a collection for digital conversion
• Plan the approach to digitization and framework for primary access
• Produce the digital collection and access aids -- in preparation
• Store the collection in the digital archive -- in preparation
• Enable access through chosen framework -- in preparation
• Test and refine access framework -- in preparation
• Release to public -- in preparation

Delve deeper for further discussion of:

• Steps in the Digitization Process (last updated by Carl Fleischhauer in January 1996) lists the steps in preparing a collection for digitization in more detail, emphasizing preparation of the materials, management of a digitization contract, and quality control, all very important aspects of the process. Since many of these steps do not relate directly to the technical framework on which this documentation focuses, they have not all been detailed in this outline. The steps fall mainly into the planning and production phases.
• Structure of the Digital Archive outlines requirements and the planned architecture for LC's long-term digital archive.

Intro -- Index -- Glossary -- Feedback

Adding a collection --
(2/2/96)