Clay tablets bearing the laws of Sumeria and oracle bones inscribed with the names of kings and officials of the Xia Dynasty remain as evidence of two ancient civilizations. Yet much of the digital-only record of contemporary history is vanishing, leaving no artifactual record at all.
- Digital content born on the Internet more than doubles every year, but much is lost as soon as it appears.
- The average life span of a Web page is only 44 days; 44 percent of all the Web sites available in 1998 had disappeared within a year.
- Although the Library is capturing Web sites from the 106th Congress forward, all of the Web pages of members of Congress created before the 106th session are gone–erased from the Internet–and with them a vital chapter in U.S. history.
These are some of the statistics that Librarian James H. Billington and Deputy Librarian Donald L. Scott marshaled during press and staff briefings in February to illustrate the need for a national digital preservation program, which was launched with Congress' Dec. 3 acceptance of a plan for the National Digital Information Infrastructure and Preservation Program (NDIIPP).
Associate Librarian for Strategic Initiatives Laura Campbell heads the NDIIPP initiative. - Charlynn Spencer Pyne
"Much of what has been created [on the Web] of late is no longer accessible, and much of what disappears is important one-of-a-kind material that can never be recovered," the Librarian said on Feb. 14, when he and Laura Campbell, associate librarian for strategic initiatives, released the preliminary program plan at a national press briefing.
"We are in danger of losing history itself because the artifacts that historians have relied upon for centuries may not always be available when they are increasingly only obtainable in the more fragile, evanescent digital world," Billington said.
He emphasized that collection and preservation of materials in the digital realm are entirely consistent with the Library's historic mission "to sustain and preserve a universal collection of knowledge and creativity for future generations."
NDIIPP Vision and Goal
Campbell, who will oversee the NDIIPP, said that the master national digital preservation plan builds on reviews of existing programs and state-of-the-art efforts in preservation, librarianship, and related technical fields, as well as 18 months of consultations with more than 200 individuals and organizations in the information technology industries; commercial, newspaper, and scholarly publishing; entertainment; broadcasting; libraries, archives and museums; and nonprofit organizations and philanthropic foundations.
The result of a "lengthy process of listening and learning," she said, the NDIIPP aspires to "ensure long-term access to a rich body of digital content through the establishment of a network of partners, collaborating in a digital preservation architecture with defined roles and responsibilities."
Campbell described the overall goal as the development of a nationwide collection and preservation strategy for digital materials in cooperation with the information and technology industry, concerned federal agencies, libraries, research institutions, and not-for-profit entities.
"The plan recognizes that the purpose of an infrastructure is to enable disparate communities and entities to cooperate in a vast, coordinated yet coherent system. It has two major components: (1) a network of cooperating institutions and partners (its human face); and (2) the technical framework, the communications networks, services, and applications that support the human face and allow individual institutions to undertake their local activities that are compatible with a larger system."
The 66-page plan, together with 257 pages of appendixes, is available in digital format on the Library's Web site at www.digitalpreservation.gov.
Background
Addressing a Library-wide audience in the Coolidge Auditorium on Feb. 24, Campbell recalled Billington's early vision (dating back to his first year in office in 1987) of using new information technology to make the Library's collections accessible more widely; he called it "getting the champagne out of the bottle."
"Many of us hung our hat on his vision," she said, citing the Library's pioneering efforts during the 1990s to build the National Digital Library, which includes the flagship American Memory project that now contains nearly 8 million historical items, which the Library and 36 collaborating institutions make available freely to millions of Library Web-site visitors.
Commending staff for their creation of American Memory and the premier legislative information site, THOMAS, Campbell said, "Many of you here took remarkable steps" into the unknown digital world of the early 1990s.
From 1998 to 2000, it became apparent from the phenomenal increase of information available only in electronic formats, including that on the Internet, that a plan was needed for a nationwide effort to select, collect, describe and preserve for future generations some of the digital-only information before it disappeared forever. Billington asked the National Academy of Sciences (NAS) to assess the Library's role.
On July 26, 2000, NAS issued its report, "LC21: A Digital Strategy for the Library of Congress." The report recommended that the Library not only frame its own strategy for acquiring, describing, and preserving digital materials in the 21st century, but also lead a nationwide, collaborative effort to preserve select digital materials that would have historical value in the future.
"The message was that the Library of Congress should take the challenge seriously, that the Library should take a leadership role, together with researchers, scholars, institutions and other stakeholders," Campbell said. "The message was that the Library needed to get outside the institution, that we had been insular."
The Library stepped forward quickly into that national leadership role with a request to Congress to support the need to meet the challenges laid out by "LC21." In December 2000, Congress approved legislation (PL 106-554) to establish the National Digital Information Infrastructure and Preservation Program. The law provided potential funding of up to $175 million, as follows:
- nearly $5 million to craft this strategic national preservation plan, including fact-finding research and reports on which to base the plan (Campbell recommended that staff read some of these major reports, prepared by national experts, contained in the appendixes);
- $20 million, subject to congressional approval of the plan (granted on Dec. 3, 2002), to launch the program and meet program goals, which call for specific actions;
- up to $75 million in appropriated funds, contingent upon raising matching funds, which could be in the form of in-kind contributions, such as expertise and hardware and software.
(A government-wide recission of .22 percent in late December 2000 reduced this special $100 million appropriation to $99.8 million.)
"Some people think it's all about money; it's not all about money," Campbell said, emphasizing that these funds, "which might not be available in today's climate," are to be used for the nationwide digital preservation program. These funds are separate from other congressional appropriations to the Library for a "Digital Futures" budget, which supports the Library's internal efforts to select, describe and archive its growing digital content.
The Stakeholders
In its 2002 authorizing legislation, Congress instructed the Library to work jointly with:
- key government agencies–the Department of Commerce, White House Office of Science and Technology Policy, and National Archives and Records Administration;
- people with expertise in the collection and archiving of digital materials from the National Library of Medicine, National Agricultural Library, National Institute of Standards and Technology, Research Libraries Group, OCLC Online Computer Library Center and Council on Library and Information Resources; and
- a broad representation of private-sector institutions working in digital formats.
All are represented on a 26-member National Digital Strategy Advisory Board (see box) that Campbell organized. "Every one of the 26 has gotten involved," she said.
Operating from "major lessons learned," Campbell said the Library "tried not to take a bold leadership role," but to facilitate listening and talking. "There is contention about the handling of digital materials, and we had an opportunity to bring people together," she said.
An additional 180 participants representing a cross section of interests (motion pictures, commercial and noncommercial radio, broadcast and cable television, publishing, philanthropic foundations, government agencies, research institutions) were invited to attend workshops and meetings during the planning period.
The purpose of three workshops held in the fall of 2001 was to bring together experts in the fields of digital content creation and distribution. From these sessions, attended by more than 90 industry representatives, the national digital preservation planners agreed on a number of points:
- There is a need for a distributed, coordinated solution to digital preservation.
- There is no ready commercial answer for the migration of digital content from the Web to accessible archives.
- Technology is not necessarily the most important part of the solution; participants wanted to know how the nationwide project would be organized and who would be involved, and who would have what responsibilities.
- Intellectual property rights are critical; rights holders are fearful of free access to digital materials and wonder how their rights can be protected. There is confusion about copyrights and what one can and cannot do under the law.
- There is a need to define what digital material will be collected, who will collect it, who will store and maintain it, and how access will be provided.
The Research
NDIIPP plan appendixes include the results of national surveys, interviews of experts, workshop findings and other research. In particular, Campbell recommended that Library staff read Appendix 6, "Copyright Issues Relevant to the Creation of a Digital Archive: A Preliminary Assessment," written by June M. Besek, director of studies at the Kernochan Center for Law, Media and the Arts, Columbia Law School.
She also recommended Appendix 3, "Digital Preservation in the United States: Survey of Current Research, Practice, and Common Understanding," compiled by Daniel Greenstein, director of the Digital Library Federation, and Abby Smith, director of programs for the Council on Library and Information Resources.
Also on Campbell's reading list is Appendix 7, "It's About Time: Research Challenges in Digital Archiving and Long-term Preservation," a preliminary draft of findings of 30 computer scientists and 15 federal agency experts meeting in two days of workshops led by Margaret Hedstrom of the University of Michigan.
Hedstrom's report issues a call for research through the National Science Foundation, the Library of Congress and others. "This was the beginning of our effort to leverage federal dollars to address similar needs and develop the tools to serve many of us," Campbell said. Library staffer Carl Fleischhauer contributed to this report.
Samuel Brylawski of the Library's Recorded Sound Section contributed one of several papers to Appendix 2, titled "Building a National Strategy for Digital Preservation: Issues in Digital Media Archiving." He examined the issues related to the "Preservation of Digitally Recorded Sound."
National Digital Information Infrastructure Preservation Program (NDIIPP) Advisory Council
Jim Barksdale
President and CEO
Barksdale Management Corp.
James H. Billington
Librarian of Congress
Library of Congress
Lynne Brindley
Chief Executive
The British Library
Laura Campbell
Associate Librarian for Strategic Initiatives
Library of Congress
John W. Carlin
Archivist of the United States
National Archives & Records Administration (NARA)
Nancy L. Eaton
Dean of University Libraries
Pennsylvania State University
Donald L. Evans
Secretary
U.S. Department of Commerce
James Gray, Manager
BARC (Bay Area Research Center) Microsoft Corporation
Margaret Hedstrom
Associate Professor
University of Michigan School of Information
Larry Irving
President
Irving Information Group
Glenn R. Jones
President and Chief Executive Officer
Jones International Ltd.
Brewster Kahle
Co-Founder, President and CEOAlexa Internet
Michael E. Lesk
Information and Intelligence Systems
National Science Foundation
Donald A.B. Lindberg, M.D. Director
National Library of Medicine
Clifford Lynch
Executive Director
Coalition for Networked Information (CNI)
Carol Mandel
Dean of Libraries
New York University
Victor McCrary
Chief
Convergent Information Systems Division
National Institute of Standards and Technology (NIST)
Mario Morino
Chairman
Morino Institute
Charles E. Phelps
Provost
University of Rochester
Richard S. Rudick
Senior Vice President
John Wiley & Sons
Michael C. Ruettgers
Executive Chairman
EMC Corp.
Richard Russell
Chief of Staff
White House Office of Science and Technology Policy
John F. "Jack" Sandner
Special Policy Advisor & former Chairman
Chicago Mercantile Exchange
Richard Sarnoff
President
Random House New Media and Corporate Development
Donald J. Waters
Program Officer for Scholarly Communication
Andrew W. Mellon Foundation
William Wulf
President
National Academy of Engineering
