NAS: Library Needs a Digital Strategy

The Library needs to develop an overall strategy for acquiring, describing, and preserving electronic journals and books, Web sites and links, databases, and other digital creations.

If it is to be relevant in the 21st Century, the Library must not only acquire digital information for its own collections through the copyright deposit system, but also lead other libraries in a collaborative, strategic effort to select digital information that is worth keeping, determine how to catalog it, and decide how and where to archive it before it is lost forever.

That is the thrust of a National Academy of Sciences (NAS) committee report, LC21: A Digital Strategy for the Library of Congress, issued to the Library on July 26. Dr. Billington commissioned the study in 1998.

James J. O'Donnell, vice provost, Office of Information Systems and Computing, University of Pennsylvania in Philadelphia, chaired the 16-member Committee on an Information Technology Strategy for the Library of Congress. The committee project was approved by the governing board of the National Research Council, which is the chief operating arm of the National Academies of Sciences and Engineering.

O'Donnell and four other committee members commented on their 207-page study during a public session, July 26, at the Academy of Sciences in Washington, D.C.

O'Donnell summed up the committee's overarching conclusions: "The digital revolution has upended expectations, expanded possibilities, and posed breathtaking opportunities for libraries around the world.

"Our study of the Library of Congress and its strategic and tactical reactions to this moment in history concludes that the walls of that venerable place have not yet sufficiently been penetrated by the thrilling possibilities of the moment, and that if this moment is lost, the library risks subsiding into gray irrelevance," O'Donnell warned, adding that he and the committee "stand behind that fundamental conclusion with all our professional conviction."

Committee members and follow-up panelists, who also commented on the report, praised Dr. Billington's "bold decision" to ask the National Academy of Sciences for the study.

"I very much admire Dr. Billington's decision to request this study," said panelist Deanna Marcum, Council of Library and Information Resources. "While the staff might well have been able to write its own self-study, the fact is that there is nothing like an outside statement to focus an institution on making necessary changes."

Another panelist, Clifford Lynch, Coalition for Networked Information, said the Library could have hired a "routine consultant" to address some technology issues at LC.

"Instead, the Library went to the National Research Council, which has a well-established history of putting together independent, broad-ranging, thoughtful committees that have a notable degree of independence and rigor. . . . What we have is quite a remarkable report, which I think the Library of Congress needs to be applauded for asking for," Lynch said.

"This isn't just a narrow review of information technology. This is really about how the Library of Congress and the digital world connect up with our society, with the whole system of libraries, and the management of cultural and intellectual heritage in this country," Lynch said.

The NAS recommendations pose "one of the most interesting collective challenges in the history of the Library of Congress," the Librarian said. "The study addresses what we should be doing and how fast we should be doing it."

Addressing LC managers during a special session the day after the report was issued, Dr. Billington said he was "enormously energized" by the report.

"It's not going to be business as usual," the Librarian said, adding that he expects a large-scale response to the report, on the order of the Library's successful efforts to establish the National Digital Library Program in a short time and to plan and implement the Integrated Library System.

He said he would not have wanted a study like this "if I didn't think we had extraordinary commitment in this institution, if I didn't think we had in place extraordinary managers who are dedicated."

(Copies of the NAS study may be obtained from the National Research Council site, at http://www.nap.edu/books/0309071445/html/.)

Committee chair O'Donnell has been invited to brief the staff on the report at an all-staff forum, from 2 to 3 p.m. on Monday, Sept. 25, in the Coolidge Auditorium.

"Now we get down to work," Scott told the managers. He said he wants the Library's response to the report in hand by the end of the year and concrete budget requests ready to go. The NAS committee asked that the Library publish, by Jan. 1, 2001, its own review of the report and outline of the agenda the Library will pursue.

Alluding to the committee's emphasis on flexible personnel practices that will enable the Library to recruit, hire, retain, and train staff, Scott said, "The Library has to fix its personnel policies and hiring practices, library-wide. If it can't do that, it can't do anything."

He said the HR21 task force needs to accelerate the work it has been doing "and stop talking about it," adding "the time for pilots is past."

Scott said the Digital Futures Group has to "step up the pace, too, and get more people involved."

He also noted the urgency of securing the Library's computer systems.

"If we focus on these things, we will have a plan that is well thought out that we can implement," Scott said. "By the summer 2001, the library community will see changes that we have started to implement."

Associate Librarian for Library Services Winston Tabb said he was "gratified and relieved" to learn from the report that the "Library is on the right track, but needs to run faster."

"A major reason the committee could say, after 18 months of looking at us closely, that we are on the right track is that most of the major elements of what they believe we must do to lead this Library-and the nation's libraries-in the 21st century is contained in the five-year plan of the Digital Futures Group, which was prepared entirely by the staff of the Library of Congress," Tabb said.

That the thinking, vision, goals, and plan of the Digital Futures Group converges with the NAS report illustrates that the Library "has an exceptional staff who should make the nation proud," Tabb said.

The associate librarian said this is not the time for second-guessing past decisions or indecisions, endless chat sessions, or hand-wringing. "If we do not seize the offensive and begin immediately to prioritize and develop action plans for those recommendations with which we agree, we will have wasted a rare opportunity to lead, to renew this Library, and to make a lasting difference in the intellectual life of this nation," Tabb said.

The Digital Futures Group is eager to begin its assigned task of reviewing the NAS recommendation, he said, and to work closely, as assigned, with the Register of Copyrights "on perhaps the most critical challenge we face-accelerated development of a full CORDS production system that will enable copyright owners to deposit, and the Library to acquire, works in digital form."

In his July 26 statement, O'Donnell said three of the committee's conclusions-those relating to the Library's digital strategy, management structure, and human resources-deserved primary mention.

O'Donnell said the committee concluded that "the Library has made very slow progress in devising strategy and tactics for the acquisition, preservation, cataloging, and accessibility of material that is created and distributed primarily in digital format-material we call 'born digital.'

"Although there have been ad hoc projects and pilot programs around the Library, no overall strategy has been developed, and fundamental systems needed to make such a strategy feasible are not in place. This is not to say the Library has been inactive, but we do believe that the steps taken have been far too limited in scale," he said.

He noted that the National Digital Library Program-"a kind of internal Apollo Project"-brought together skilled and visionary people, found resources in creative ways, and delivered the product (5 million digital artifacts) "in an astonishingly short time."

"But turning the Library's paper treasures into digital artifacts is only one tiny part of the work of creating a true National Digital Library," O'Donnell said.

The committee reviewed the Library's history, noted the preeminence of its traditional collections and its landmark achievements, such as the creation and standardization of machine-readable cataloging (MARC), assessed the status of digital information and technology in LC's various units and curatorial divisions, and recommended some strategies for capturing, accessing, and preserving digital information.

The committee made a case for acquiring digital information. "At the heart of its recommendations is the committee's strong awareness of the role of digital information at the center of contemporary discourse. That role is a simple fact, unrelated to whatever e-zealots or bibliophiles might wish to be the case.

"For some important areas of human knowledge, the best new knowledge can be acquired in digital form," the committee said.

Formats include thousands of e-journals, many of them scholarly journals, including digital versions of printed journals and digital-only publications, most of which will be linked through indexing services and search engines; e-books, which vendors are beginning to distribute as the result of deals with publishers; databases; and some 3.6 million Web pages and linked resources.

"Probably the most active area of digital publishing today is taking place on the World Wide Web. . . . some of these digital works are as important as records of current research and creativity as were the journals and books of the eighteenth, nineteenth, and twentieth centuries," the committee said.

Librarians in the 21st century will have to change their thinking about collections, the committee said. "[C]ollection will consist necessarily of a range of activities, from the acquisition of traditional materials to something much more like what we do when we own stocks in companies whose printed share certificates we never see."

In deciding what digital information to acquire, the Library will have to think through how much it can afford to preserve. LC should develop acquisition and preservation strategies in concert with other libraries.

"The burden of preserving digital collections is enormous. The committee believes that this burden must necessarily be shared among a variety of archiving institutions," the committee said. "To ensure that all important research materials are preserved for future generations, it is important that archiving institutions understand the scope of each other's stewardship roles, as is the case with hard-copy publications."

The committee recommended that the Library:

• explicitly define the sets of digital resources for which it will assume long-term curatorial responsibility;
• work with other institutions to define appropriate levels of responsibility for archiving and accessing digital resources LC does not want to keep and maintain;
• selectively adopt a "portal model" for some selected program areas, creating links from LC's Web site to distant archives and arranging for users' access to licensed commercial resources.

Some of the toughest questions for all libraries relate to ownership of digital information and the high cost of acquiring, accessing, maintaining, and preserving it. Traditionally, libraries owned materials as the result of purchases, gifts, or exchanges, and permitted use as allowed by copyright or best practice. "Many electronic resources, on the other hand, are maintained by publishers, vendors, or others designated by them, and libraries or individuals can obtain rights of access through custom or mass-market licenses," the committee said.

"The Library of Congress has a unique and privileged position in the acquisition of cultural materials," the committee said. "While other libraries in the United States must find willing sellers and must have the financial wherewithal to become willing buyers in a freely made commercial transaction, the requirement of legal deposit with the Copyright Office gives LC a presumptive right to full ownership of a copy of each and every artifact published in the United States."

"The Library's role in registering copyright and enforcing the mandatory deposit law creates a unique opportunity for it to collect digital information that might otherwise vanish from the historical record," the committee said.

However, the committee also found: "The Library of Congress, as recipient of mandatory deposit copies of works published in the United States, lags significantly in receiving and archiving the born-digital product of the nation."

The committee questioned whether the Copyright Office should continue to develop the Copyright Office Electronic Registration, Recordation and Deposit System (CORDS) in its current form or use a different system for receiving deposits of digital publications, registering them, and adding them to the Library's collections. This project has been underway since 1993. The committee said it was "concerned about the scale and deployment" of CORDS, which is expected to handle 100,000 digital deposits (less than 15 percent of the projected total of 725,000 deposits) by the year 2004.

O'Donnell said it "is urgent that this stage of experimentation be brought to conclusion with the installation of a system capable of handling much larger quantities of digital information, and in particular, capable of allowing the deposit of the huge, creative output of the World Wide Web.

"We believe that this last requirement can only be achieved if the Copyright Office and other units of the Library begin to work together with an intensity that they have not hitherto been challenged to demonstrate," O'Donnell said.

The new copyright acquisition system needs to integrate well with other Library systems and make it easier for providers of information to register and deposit their works, and for the Library to enforce the deposit requirement, the committee said.

The committee recommended that the Copyright Office complete a statement of work for a production system in FY 2001, as planned, and "as soon as possible," by the end of calendar year 2000.

"To achieve this goal, the resources and attention of Library-wide senior management should be directed to the Copyright Office, perhaps comparable to the scale and visibility of the ILS implementation," the committee said.

The committee urged the Congress to support and fund the acquisition of a production system.

Finding that the mechanisms and policies for the deposit of digital works currently favor printouts or tangible forms, such as CD-ROM, over digital editions of digital works, the committee recommended that new standards be set for the appropriate formats for digital material acquired through copyright deposit, purchase, exchange, and donation.

In particular, the committee recommended that LC "aggressively pursue clarification of its right to collect copies of U.S.-based Web sites under the copyright deposit law. If questions about this right remain, then LC should seek legislation that changes the copyright law to ensure that it has this right."

The committee emphasized the importance of LC delineating its responsibilities for preserving digital information: as sole proprietor and custodian of some digital materials, as a "fail-safe" caretaker of collections created by others, and as a partner in preserving distributed digital collections.

"As LC identifies the areas in which it will assume the lead responsibility for digital preservation, other organizations can adjust the scope of their digital collections accordingly," the committee said.

"Just as the Library cannot ignore the problem of digital preservation, it cannot be expected to do it all," the committee said.

The committee noted three challenges. Digital materials are especially vulnerable to loss and destruction because they are stored on fragile magnetic and optical media that deteriorate rapidly and that can fail suddenly. They become unreadable and inaccessible if the playback devices or software necessary to retrieve information become obsolete. Legal questions surround libraries' rights to copy digital information for preservation or backup purposes, to reformat information so it remains accessible, and to provide public access.

The committee found that, although many national libraries, university research libraries, archives, bibliographic utilities, and organizations with large holdings of digital information are actively pursuing solutions to these problems of digital preservation, "LC has at best played only a minimal role in these initiatives.

"As a consequence, it has little awareness of potential solutions that are emerging from joint research and development projects and has not contributed much to this important national and international problem for the library community," the committee found.

The Library should join, and, where possible, lead or facilitate national and international research and development efforts in digital preservation, the committee recommended. Also, the Library should take an active role, including working with Congress if necessary, to rework intellectual property restraints to permit copying migrating digital information to a new format for preservation purposes.

Lacking an overarching strategy and long-range plan for digital preservation, the committee said, the Library should "immediately form a high-level planning group to coordinate digital preservation efforts and develop the policies, technical capacity, and expertise to preserve digital information. A digital preservation plan should be put in place and implemented as soon as possible.

The committee also recommended that high priority be given to filling the position of head of the Preservation Directorate with someone who is knowledgeable about digital preservation.

"One enduring role of libraries during the transition from physical to digital information will be the intellectual task of cataloging-imposing order on diverse resources with the goal of making those resources easier to discover and manage," the committee said.

Noting that LC probably has the largest cataloging operation in the world, and remarking upon LC's long and distinguished history of leadership in setting and coordinating cataloging standards (MARC, Anglo-American Cataloguing Rules, subject headings, name authority controls), the committee urged LC to assume a leadership role in the development of digital information "metadata," defined as "information that describes the structure or content of a document but is not part of the document."

"The metadata environment is evolving rapidly. This will have profound implications for libraries and other information providers generally and for the Library of Congress in particular," the committee said. "It is the responsibility of the Library, and indeed of the nation, to offer leadership here for the benefit of the national and worldwide communities of information providers and users."

The committee made two recommendations:

(1) The Library should treat the development of a richer but more complex metadata environment as a strategic issue, increasing dramatically its level of involvement and planning in this area, and it should be much more actively involved with the library and information community in advancing the evolution of metadata practices.

"This will require a dedication of resources, direct involvement by the Librarian in setting and adjusting expectations, and the strong commitment of a project leader assigned from the Executive Committee of the Library."

(2) "The Library should actively encourage and participate in effort to develop tools for automatically creating metadata. These tools should be integrated in the cataloging work flow."

"In the end, the success or failure of the Library in the digital age will be marked chiefly by its ability to rethink and reinvent (along with comparable institutions around the world) the way the collecting and cataloging are done, whether the artifacts are digital or analog. If this report has a single central message, it is this one," the committee said.

In its assessment of management issues, the committee said: "It is no longer possible, if it ever was, for senior management of large organizations to regard information technology as a black box to be controlled and managed by technologists. Certainly a library, the core of whose business is the storage and preservation of information, needs to integrate strategic and tactical thinking about information technology into every level of its management vision."

Responsibility for overall strategic planning should be the job of a second deputy librarian, the committee recommended. This deputy also would supervise Information Technology Services, chair a new information technology vision, strategy, research, and planning group (ITVSRP), and manage LC's relationship with a panel of outside experts on a new Technical Advisory Board.

Citing the Whole-Book Cataloging Pilot Program as an example of reengineering work processes, the committee also emphasized a need to examine Library-wide workflow processes "and rationalize them across unit boundaries before new information systems are designed and developed or acquired." The Copyright Office and its interface with Library Services is the place to start this assessment, the committee said.

"The ITVSRP group would be chartered to lead the Library of Congress, and the national and world libraries, into the digital age," the committee said. "It would also provide strategic technical thinking for the Library. Members of the ITVSRP group would have a good grasp of current technologies, be effective communicators and diplomats both within LC and without (to help build bridges to industry and academia), and have some grasp of how LC works today."

Every unit of the Library would have to get approval from this group for major information technology investments, the committee suggested.

The committee also recommended the creation of a Technical Advisory Board, which would be run by a distinguished outside chair and include librarians from the United States and abroad with a broad vision of the future of libraries, as well as technology specialists with expertise in relevant areas, the committee said.

This board would sit with the Executive Committee of the Library twice a year and inform executives of developments and directions in information initiatives and enterprises that the Library has in hand with the ITVSRP.

The committee recommended further that the technology awareness of LC be enhanced by creating a limited number of visiting research positions within LC for experts from around the country.

The committee also recommended that the Library not appoint a chief information officer at this time.

These recommendations flowed from the committee's findings of no strategic planning for information technology within LC, a lack of senior management review of priority setting within Information Technology Services (ITS), and attendant shadow and duplicate systems in the Library as units establish their own systems. "No one within the Library is explicitly looking at technology and technical trends, looking at where technology will be five years out. No one is bringing that vision to the major service units and current innovators and saying to them, 'This is what [Information Technology] could bring to you in five years; how would you use it?'" the committee said.

The committee noted that although ITS is chartered to provide a service to the rest of the Library, "it is not chartered or budgeted to generate ideas about the Library's future, even as that future relates to technology."

"The current ITS organization and staff are not well suited to provide technical vision, strategy, or technical leadership for the Library. These roles require an outward-looking organization that participates in national and international library initiatives," the committee said.

O'Donnell said the committee was "deeply concerned that human resource issues in the Library-some a legacy of past management problems, some intrinsically related to the challenges of managing information technology in the federal sector-pose the most threatening set of obstacles to improvement."

One problem is noncompetitive pay. "It is practically impossible for the Library to hire new college graduates in computer science," the committee said, noting that the Library is able to pay only about 60 percent of the going salaries for IT workers.

Another problem is a federal personnel system bogged down by Civil Service human resources regulations; several layers of court-ordered procedural steps added to all hiring and review of personnel as result of the Cook case; and outmoded management practices "that are redolent of old, assembly-line methods."

"In the short term, hiring has become slower and more cumbersome, just at a moment when technical staff particularly are pursued with increasing speed and agility by virtually every other sector of the economy," the committee said.

"The need for workforce flexibility in the digital age is in direct conflict with rigid human resource rules at the Library of Congress," the committee said.

Noting that the National Digital Library Program hired quickly not-to-exceed staff and contractors with special skills, the committee recommended a greater reliance on outsourcing and contract employees.

(Deputy Librarian Scott on Thursday, July 27, gave managers the green light to use personal service contracts to expedite hiring. "If a particular unit needs special hiring dispensation, you've got it," Scott said.)

Noting that some 40 percent of the LC staff will be eligible to retire by 2004, the committee recommended against the automatic hiring of replacements with similar skills. "Retirements should instead be viewed as opportunities to hire staff with the qualifications in librarianship and technology needed to meet the digital challenge, and reengineering should be rewarded when senior management allocates staff positions to units," the committee recommended.

The committee recommended that LC's Human Resources directorate review hiring practices to make sure job descriptions are written to appeal to applicants outside the government.

Recognizing the long tenure of employees, the committee recommended more training opportunities for staff. Professional development, outside technical training, and practice in using the training are crucial.

"Congress should be asked to increase the Library's training budget by a significant amount. This increase should be more than an incremental one-more on the order of a doubling or tripling of this year's amount in the next budget submitted to Congress," the committee recommended.

The committee also recommended that LC increase the number of junior and senior level staff involved in professional association activities, which are a source of learning, networking, and leadership training; that more internships be offered to graduate and undergraduate students; and that mentoring teams be formed to provide peer training in technology.

A formal assessment of lessons learned from the ILS installation should be completed by Jan. 1, 2001, the committee said.

Finally, the committee recommended that human resources staff "should become agents of change and business partners more rapidly than is foreseen in the HR21 plan."

Noting that the Library increasingly turns to outside sources for information technology needs, such as the Voyager software and Sun server for the new ILS, the committee said the Library will continue to need a strong in-house organization to perform some internal information technology development, training, support, and operations, and to monitor outside contracts.

The committee said Information Technology Services (ITS) basically does its job of supporting LC's computer and communication systems. It acquires, supports, and maintains the computer, networking, and telephone systems for LC; it does most of the in-house programming and much of the computer-systems training; and it monitors contracts with software and hardware vendors.

"ITS produces substantial and serviceable high-level architectural documents on topics such as storage and retrieval of digital content, centrally supported systems infrastructure, and telecommunications. Its server and storage architectures meet its customers' needs.

"Although it is not what in industry would be seen as an exceptionally responsive or cutting-edge technical organization, given the many restraints of its environment, ITS adequately provides basic services. The Library of Congress systems work and people get their jobs done, even though-as is true in other organizations-they want more services than the central organization can possibly deliver within budget constraints," the committee found.

Although mid-level management seems to be hard-working and competent, "ITS staff do not . . . have the technical depth to meet the current challenges. They cannot keep current in areas like networks, security, middleware, and databases . . . because the Library of Congress does not budget adequate time or money for their continuing education or attendance at professional conferences. Nor is there a budget for members of the IITS staff to spend time investigating new technology," the committee found.

Committee recommendations included investing more in the continuing education and training of ITS staff.

The committee suggested ITS be more businesslike in its operations by instituting a system of service-level agreements that include performance metrics and by instituting a cost-accounting system by which service units may determine what their information technology actually costs them.

The committee found two main infrastructure problems, the lack of information technology security and underpowered networks.

Finding that LC's computer and information security competence and policies are "seriously inadequate," the committee recommended that technical experts should be retained to recommend an information security plan, which should be implemented promptly. Congress should provide funding for a disaster recovery strategy and its implementation.

All local area networks need to upgraded to 100-megabit-per-second Ethernet circuits as soon as possible, and compatible Ethernet switches should replace ATM switches designed for digitized telephone traffic, the committee recommended.

Among the committee's other findings and recommendations are these:

• All LC employees should have easy access to universal e-mail.
• The Library should establish disk-based storage for online data and a remote online disaster recovery facility. The current system does not distinguish between storage for access and storage for preservation and is too costly.
• The Library should place a higher priority on implementing an appropriate digital information repository.