I have pleasure in presenting the report of the SMETE Library Workshop,
held at the National Science Foundation on July 21 to 23. The participants
looked at the educational opportunities that would be provided by a
comprehensive digital library for science, mathematics, engineering, and
technology education (SMETE), aimed at the needs of undergraduate students
and instructors. The workshop was able to build on an earlier study
organized by the National Research Council.

Our overwhelming recommendation is that a digital library of the kind that
has been proposed would be highly beneficial to undergraduate education.
We hope that the NSF will find the resources to support such an effort.

As chairman of the workshop, I would like to thank the members of the
steering committee, the workshop participants, and the staff of the NSF for
their energy in making this workshop a success.

This is the report of a workshop held at the National Science Foundation to examine the concept of a digital library to support undergraduate education in the fields supported by the NSF. The working name of the proposed library is "the SMETE Library", an acronym for Science, Mathematics, Engineering, and Technology Education. An earlier study examined the feasibility of such a library; this workshop emphasized the impact on education.

A broad consensus emerged that a comprehensive SMETE Library would be a major resource for undergraduate education and that the NSF should be urged to implement it. Moreover, the outline of an implementation plan emerged that is practical and cost-effective. The workshop agreed that, in order to be successful, the library must be comprehensive, it must be well managed, and it must be funded in a manner that emphasizes service to the educational community. Throughout the workshop there was a sense of urgency. The time to start is now, while digital libraries and the web are still in flux.

Section A of this report describes this vision; this section is a synthesis of the recommendations from the final session of the workshop. Section B is a review of the workshop, describing the membership, the agenda, and the process by which the conclusions were reached.

The report is available on-line at http://www.dlib.org/smete/public/report.html.

The opinions, findings, conclusions, and recommendations expressed in this report are those of the participants, and do not necessarily reflect the official views, opinions, or policy of the National Science Foundation.

The Internet has the potential to transform undergraduate education, but only a fraction of that potential is now being realized. Some of this gap lies in the maturation process that is part of any transition, but a larger part is the result of fragmentation. Resources of great value are not being used because students and faculty do not know about them, or do not know how to use them.

The NSF has funded the development of many of these resources. Some materials were developed explicitly for undergraduate education; other materials that were created for research purposes are also valuable for students. Additional materials are being created by the commercial sector, by private individuals, and by other government agencies.

While great efforts have been placed on creating materials, less attention has been given to organizing them, maintaining them in the long term, helping people find them, and training people how to use them. For example, a faculty member who is planning a course has only the most rudimentary tools to discover what materials are available or whether they have proved effective in other courses. A student who is researching a topic is forced to choose between general-purpose web search services and commercial databases designed for scientific and technical research. Neither faculty nor students can safely rely on resources that might be withdrawn without notice, or change subtly overnight.

This situation can be seen as a problem, but it is also an opportunity. The increasing maturity of the web and recent research into digital libraries, combined with the energy and enthusiasm with which collections and services are being made available over the Internet, have reached a critical mass. The resources exist. The critical missing link is a framework and infrastructure to maximize their impact on education.

To meet these needs, the workshop articulated a vision of a new type of library, the SMETE Library. This is envisaged as a comprehensive library of the digital resources and services that are available for undergraduate education in science, mathematics, engineering, and technology. The key word here is "comprehensive". Faculty are very specific in wanting a single place where they and their students can discover, use, and possibly contribute a wide range of materials.

"Comprehensive" does not imply uniform and most certainly does not imply centralization. The SMETE Library will be a federation of library services and collections that function together to create a digital learning community. Organizationally, the SMETE Library will consist of a small central operation with a wide range of partners. Some of the services and collections are already well organized; for these, the SMETE Library will be a gateway. Others exist but are poorly organized; for these, the SMETE Library will stimulate the creation of specific services. Some materials are fragmented, unorganized, or hard to find; in these cases the SMETE Library will build library services and may even manage specific collections. Across all these areas, the SMETE Library will provide tools to help faculty and students find and use materials, with services to assist them in evaluating quality and appropriateness.

The SMETE Library will take a broad view of science and technology, and of scientific education. The primary audience is faculty and undergraduate students, but there is no hard distinction between the needs of high school students, undergraduates, and graduate students, nor between students in formal programs, independent learners, and the general public. The SMETE Library could serve them all. The members of the workshop see a particular need to organize and provide access to materials that have been developed specifically for education, but undergraduates need access to other materials as well, especially research literature and the experimental data that is increasingly available over the Internet. The SMETE Library will strive to be a gateway for all these materials. Finally, the SMETE Library will have a variety of financial models for access to the materials; some content will be free of charge while other materials will be available on a commercial basis.

The SMETE Library will contain and provide access to a wide range of materials, some of which have not been collected in the past or have been difficult to discover. The range includes curricula and courseware materials, computer programs, the results of educational research, scientific research reported both formally in journals and informally in web sites, raw data for student activities, and multimedia image banks. Further, the SMETE Library will provide services for authors and instructors such as annotation, evaluation, and peer review of donated materials. For students and faculty, it will offer the capability to search for desired information by subject area, to have access to scientific data sets, and to interact with peers. Faculty, students, and other clients such as independent learners will be able to participate in forums. Interdisciplinary activities, lifelong learning, and the process of education will all benefit. In this way, the SMETE Library will be much more than the sum of its parts, and will promote change and innovation in scientific and technical education at all levels.

The workshop participants were enthusiastic about the educational benefits of a national, federated, virtual digital library that will support students, faculty and authors through high quality collections, tools and services. To achieve maximum impact, it should cover all of science, mathematics, engineering and technology, with varying ports of access (by discipline, by type of user, by access purpose). The goal is maximum technical, intellectual and financial flexibility for all users. Guiding principles for the SMETE Library are that it should:

be driven by educational and scientific needs,

facilitate educational innovations,

be stable, reliable and permanent,

be accessible to all (though not all materials will be free),

build on, and leverage past and current work in courseware libraries, digital library research, and successful commercial sites,

be adaptable to new technologies.

As the principal goal is to improve the quality of and increase access to undergraduate education, the primary users must be students and instructors. However, the materials and services that the library will provide should be of great value to everybody with a serious interest in science and technology, including the research community, K-12 teachers, the public at large, and everybody involved in continuing education.

The educational benefits and national impact of the SMETE Library will be both to improve the quality of education and to improve access to information. Rather than being limited by a single textbook, students and faculty will be able to draw on a much wider range of scientific and educational resources. A quality review system will allow the user to have an appropriate level of confidence in the material being used. In addition to basic course material and reports of research, students and faculty could also access:

simulation software that could either be downloaded or run on more powerful machines,

data sets for inquiry-based learning.

The Impact on Education

The SMETE Library will have an impact on every aspect of scientific and technical education. The only limits are the breadth of the collections, and the imagination of faculty and students. The collections and related services would allow faculty to learn about and incorporate diverse learning processes, including collaborative learning. Faculty could engage in professional training and learn about best practices through discussion groups and on-line workshops. Students could supplement their own learning with:

discussions on topics with students and researchers from other universities and institutions.

The SMETE Library will encourage the dissemination of research into educational methods. It will also facilitate the involvement of industry and government laboratories in the educational process. Whereas some universities benefit from guest speakers from industry or government in the classroom, not all schools are able to arrange such visits. The digital library, enabled by new information technologies, would provide a forum for real-time video or voice communication to a wider range of learners. These virtual lectures and discussions could be captured and then added to the library for later access.

The SMETE Library would also facilitate cross-institutional sharing of educational resources. The ultimate goal is the development of a community of scientific and technical educators who use the library for cross-disciplinary and cross-institutional collaboration. Access and discussions with authors and prior users would be possible, along with an archive of past reviews and discussion of materials in the SMETE Library. The collections could be annotated and linked to these discussions and reviews.

The digital library also opens the opportunity for students at different institutions to work on joint projects or experiments, perhaps sharing and adding to the same data set and its analysis. This would also promote physical resource sharing, as students and instructors may have varying access to high-end instrumentation, computational capabilities, data collections, and technology.

A Faculty Scenario

Professor Green teaches at a community college. He plans to refresh his course in calculus to include real world applications of differential equations. He has used the SMETE Library in the past and has set up his user profile, including the courses he has taught and his educational background. He performs a search for "calculus applications in engineering and physical sciences" and is given a list of possibilities, prioritized by his search goals and his profile. He is also given a list of discussion groups of faculty with similar interests. Some of the courseware listed has a large enough user base that they have their own discussion groups.

Professor Green is particularly interested in a multimedia courseware package that uses Matlab exercises to relate ordinary and partial differential equations to the control of a disk drive assembly. When he links to the web version of the case study, he finds that an industry expert is currently holding "office hours" with a class from another university. Professor Green joins the "office hours" and is able to observe and then participate in the class discussion, taking on both student and instructor roles. Based on the information in the library and his participation in the on-line office hours, he adapts the courseware for his class and arranges for similar interactions with the expert from industry. Later he develops his own exercises and adds them to the SMETE Library for others to use.

Access

For the SMETE Library to achieve its potential, it is critical that the library be accessible by all members of the SMETE community (especially undergraduate students, faculty, and authors). The library could have its greatest impact by improving the knowledge base and instructional resources of those undergraduate educational institutions that historically have lacked good library and computing facilities. The SMETE Library provides an opportunity to level the playing field for students and faculty whatever their institutional affiliations, but effort is needed to achieve this goal.

The workshop strongly recommends that accessibility should be an explicit part of the SMETE Library mandate. This requires a two-part strategy. The first is that the library should be realistic in its technical expectations. Since accessibility can be limited by a range of factors, including network bandwidth, availability of computers, and costs, the library must be designed to accommodate a wide range of users and be realistic about the technology that they use. In particular, to serve the off-campus needs of faculty and students, the library should provide most core services and material based on modem level communication. However, not all SMETE Library services need be limited to the lowest common denominator of the current capabilities of students and faculty. Technology is improving rapidly, and the library must grow with it. One approach is to provide multiple version of content that can be adapted to a range of bandwidth options.

The second part of the strategy is that the SMETE Library should work vigorously with concerned individuals and organizations, including the NSF and other federal and local agencies, to ensure that all students and faculty have good Internet access. Modern scientific and technical education requires that all faculty and students have computers and telecommunications, with the training to use them effectively.

The SMETE Library will be a team effort comprising four groups: the National Science Foundation, the SMETE Library central team, the SMETE Library partners, and associated research projects. Each group has a specific and distinct role.

The National Science Foundation

The leadership of the National Science Foundation is a vital component of the SMETE Library plan. The NSF brings prestige that will stimulate the community to share and work together. The foundation will provide oversight, serve as a catalyst for coordination among existing digital library initiatives, and encourage the fullest possible interaction and collaboration between the foundations directorates and other government agencies. The foundation through its funding and evaluation functions can ensure that the SMETE Library will be sustainable, reflect current best practices regarding standards, and be a cost-effective mechanism for enhancing quality education in science, mathematics, engineering and technology for undergraduates in the United States. Actions by the foundation can also serve to bridge the gap between rapidly-changing technology and much of the educational arena.

However, the workshop also urges a more proactive role for the NSF. Many of the resources that will form the SMETE Library are the products of work that the foundation funds. These resources include web sites on which research results are reported, curriculum materials, and scientific data sets. The foundation can request or require that research projects which create these resources meet certain standards in disseminating them. The two most important standards are commitments to maintain materials for specified periods of time, and the provision of basic metadata that can be used for information retrieval. The foundation can also work with other federal agencies to ensure that their products are structured in a manner that makes them easier to access and use by faculty and students.

The NSF can promote the success of the SMETE Library by structuring initiatives across directorates such that they foster dissemination of research and educational findings utilizing the SMETE Library. Using its broad leverage and the strength of the peer review process, the foundation can work to ensure that materials developed for the library will be recognized for quality and that such recognition will be accepted within all educational communities.

The SMETE Library Central Team

The SMETE Library has both central and distributed parts. The workshop participants envisage a small but active central part with both organizational and technical functions. Planning and implementation of the SMETE Library will be guided by a central team that will recruit other partners and manage a small set of central services. It will also ensure ongoing evaluation of the strengths of the service provided, the breadth of the audience, the impact of student use on the learning process, additional needs of faculty and students, and future partners that might strengthen the enterprise.

At the heart of the SMETE Library will be a central area, technically a small web site. This will be the place where users begin their search for information and learn about the tools and services that are available. The objective is that it will be so good that every faculty member and student will maintain a bookmark to it. The reference area will provide guides to the resources that are available: collections, services, and collaborative areas. Collectively, these resources will cover all fields that are relevant to undergraduate scientific and technical education. The central team will manage this reference area. All other resources will be managed by partners. Hence, key tasks for the central team are to establish guidelines and to identify, coordinate, and work with the correct set of partners.

During the workshop several suggestions were made about possible forms for the central team, but no strong consensus emerged.

Partners

The SMETE Library will be a federation. The site to which the NSF lends its name and towards which it directs its primary marketing will be considered the central site, but it is unlikely to be the biggest or most heavily used. Other sites on the Internet, that are independently managed but have formal relationships with the SMETE Library, will be chosen as partners. Potential partners in the federation include digital library projects funded by the NSF or other government agencies, such as the National Engineering Education Delivery System by UC Berkeley and its partners, the Physics e-Print Archive at Los Alamos National Laboratory, and the Netlib repository of mathematical software. The partners constitute the distributed component of the library.

Partners will have various levels of participation. Core partners will commit to providing an agreed upon set of services. They will operate within guidelines on indexing and metadata standards, quality control, and procedures on availability and archiving of materials. Initially, the SMETE Library might plan for a small number of core partners who take responsibility for major areas of science, technology and education. Some will be existing, high-quality services for whom affiliation with the library is mutually beneficial, but other partnerships will have to be established. For example, an important task is to organize, index, and archive curriculum materials. These might be managed by a set of discipline-based partners. Many of the SMETE Library features discussed in the workshop related to the services and collections that will be provided by the core partners.

The challenge is to establish mutually beneficial arrangements between the SMETE Library and the partners. Partners will provide services to the SMETE Library. In return, some partners will receive funding; others will receive less tangible benefits. Each partner will be different and success will depend upon effective leadership. Professional societies, commercial publishers, and software and Internet vendors are all potential partners. The SMETE Library must have an organizational structure that recognizes the members of the federation as true partners. There will a process for shared decision making.

Associated Research

The SMETE Library is not a research project, but it will be greatly strengthened by a parallel program of research. The workshop recommends a continuing program of research linked to the SMETE Library. In addition to its programs in digital libraries, the NSF would sponsor research in technology of particular relevance to education, the impact of technology on the learning process, the effectiveness of digital learning communities, and how computer scientists, the library community, and faculty, students, and high school teachers can work together to optimize learning opportunities. However, it is important that these research activities are clearly separated from the implementation of the library itself.

The first steps for this program are already underway. The NSF is the lead agency in the Digital Libraries Initiative, and the NSF has already provided substantial funding for research that will contribute to the aims of the SMETE Library.

This section describes some of the services and collections that are envisaged to be in the SMETE Library. It should be considered as indicative of the breadth of the library, but not as a detailed plan.

Collections

A primary goal of the SMETE Library is to provide integrated and effective access to a wide range of materials, including but going well beyond the types of materials found in a conventional library. SMETE should include a comprehensive collection of the materials that might be used for education. They include, but are not limited to:

To select materials for inclusion, the SMETE Library will build on and link to existing resources, such as professional societies, commercial publishers, research archives, and local web sites developed in conjunction with NSF-supported projects. The library will survey existing materials to identify gaps, and may commission new materials or encourage NSF to solicit proposals for the purpose of filling gaps. In addition, the SMETE Library will encourage researchers and practitioners to develop materials of the quality expected by the library, thus bringing contemporary research and practice into the educational mainstream.

Guidelines

Workshop participants recommended that priority be placed on establishing guidelines for the SMETE Library materials and services, so that content in the central and distributed sites will have established quality and other characteristics that will promote their use and sharing among users. An example of a guideline would be the metadata terms used to describe content, such as subject, title, discipline, concept, keywords, and prerequisites. By using a consistent set of terms for cataloging the content in the central site and distributed sites, searching catalogs would be easier and more efficient.

Workshop participants suggested the following areas for establishing guidelines:

Quality. Promote development of content with high scientific or educational impact.

Discovery. Enable users to find content in a collection easily and precisely.

Access. Simplify installation and presentation of content. This might include installation instructions with software and hardware dependency information if the content requires more than a standard web browser.

Archiving and stability. Commit to retaining stable versions of materials and not withdrawing them without warning.

Conformance

In order for the guidelines to be effective, they must be used by as many of the distributed sites of the SMETE Library as possible, but the need to be comprehensive is in tension with the desire for guidelines. The breadth and depth of commitment to the guidelines by a partner site will be one of the characteristics that determines the strength of the relationship between the organization hosting the site and the SMETE Library. For example, it might be decided that all core partners must be committed to the guidelines for search query interoperability, whereas, a partner that was simply linked to the SMETE Library central site might not follow these guidelines.

Users would benefit from a branding program, by being able to identify sites that include content that meet the SMETE Library guidelines. The simple identification of support for the SMETE Library guidelines at distributed sites may prove valuable in promoting the guidelines and the support of the distributed the SMETE Library concept. Some members of the workshop suggested the development of a logo and marketing terms, such as "A SMETE Library Core Collection".

Information Discovery and Quality Control

As many workshop participants noted, nearly every item proposed for the digital library already exists somewhere on the Internet, but the lack of organization makes finding materials and separating the wheat from the chaff daunting tasks. Tools and materials will have a major impact on education only when the typical faculty member or student, pressed for time, can quickly locate materials and assess their quality and applicability. The SMETE Library will meet this need by:

developing indexes and catalogs,

providing standards for the selection and review of library materials,

promoting standards and practices for authors of materials.

Indexes and Catalogs

The SMETE Library will need to provide some form of indexes or catalogs to supplement existing commercial services and the web search programs. The recommended approach to indexing is to associate metadata with each item in the collections. The metadata will follow specific guidelines. (For example, the Educause Instructional Management Systems Project is developing specifications that might be used for this purpose.) An indexing program can assemble this metadata and build searchable indexes for the library.

This procedure can be established for core collections, but not for all collections. Hopefully, most of the other collections will have some metadata that follows the emerging Internet standards in this area. However, it must be accepted that the SMETE Library may not have a comprehensive catalog, beyond the core collections.

Quality Control

Workshop participants expressed a wide range of views regarding whether, or to what extent, educational materials should be reviewed prior to library inclusion. All agreed that it is difficult with educational materials to predict which will be found useful by other faculty or students. The peer review process of professional journals is not a good model for much of the material likely to be considered for a digital library. Quality and ease of use have subjective aspects that may be promoted more effectively through establishment of a broader range of review processes and best practice guidelines, such as guidelines for instructional and user interface design informed by research in distributed learning.

With this in mind, the participants recommend that quality of materials be addressed by developing a dynamic and on-going quality assurance program of tagging certain materials with a record that might include: description and documentation of the materials, lists of users with comments and post-use reviews, formal reviews and other assessment data, and means of communication with the author. Materials that contain this information can be identified as SMETE compliant. In addition, it was suggested that the SMETE Library consider establishing a limited number of yearly awards for excellence. These recommendations will provide several levels of quality assurance, ranging from materials that are untested and unreviewed, up to fully reviewed and award-winning materials.

Tools and Practices for Creation of Material

The SMETE Library will provide authors and publishers with tools and guidelines that will encourage the creation of materials that fit within the SMETE Library framework. The objective is to make compliance with the guidelines so straightforward that it will become routine with authors, publishers, and others who manage collections.

These tools and practices will support standardized submission, review, and user annotation of materials. They might also include group and multimedia authoring tools, along with developer feedback and Internet link maintenance mechanisms.

Services

The SMETE Library must be more than just a repository of materials, with catalogs and indexes. To meet the needs of faculty and students of the twenty-first century, the workshop participants recommend that library materials be organized and linked to provide a dynamic, interactive environment rather than a static collection of isolated documents. They should emphasize interactive engagement over a passive learning environment, and allow for user modification where possible. Some participants even considered that environments for collaboration might emerge as the central theme of the SMETE Library.

The SMETE Library will provide an interconnected set of services that, together with system level services, will support the use of electronic modes of finding, accessing, and acquiring skills, information, and knowledge in all disciplines, both for faculty and students. Faculty services will provide registration, search engines, and training, to support the discovery of materials and the tracking of usage. These services will also ease the adoption and adaptation of materials accessible through the library and support the discussion and collaboration that fosters the sharing of information necessary for the effective and efficient use of SMETE Library identified materials. Student services will include the maintenance of a list of locations of appropriate applications tools, easy search facilities for finding review materials in introductory and remedial course content, and support for collaboration among students on large, inter-campus projects. Training is a theme that runs though many of these services.

Archiving and Stability

Stability and archiving are different aspects of the same topic. Teaching requires that materials be stable. Faculty must be confident that resources incorporated within a course do not change in the time period between planning a course and its delivery. The flexibility of the Internet leads to instability. Resources are often withdrawn with no notice. Worse, versions may change in ways that are incompatible with educational plans. We envisage the SMETE Library providing procedures, technical standards, and notification systems to address this problem.

The archiving question is longer term. Many of the most useful educational materials are the results of projects that have a finite life. The SMETE Library archiving function is to identify resources that have long term benefits and work with partners to preserve them for the long term. Long-term preservation is not a trivial task in a time of technical change. This is a topic that would benefit from associated research.

The workshop was not explicitly asked to consider how the SMETE Library should be implemented, beyond the question of overall priorities. Several themes, however, were repeated throughout the discussions.

Getting Started

As the previous sections have made clear, getting started is urgent. The fragmentation of resources and the lack of organization are growing worse. The volume of materials that will need to be brought into a systematic framework or will languish unused without the SMETE Library grows daily.

The first step is a clear statement of leadership from the NSF, followed by the establishment of a process that will lead to the creation of the central team and core partnerships. This process should include people with expertise in the various content areas, scientific and technical education, and digital libraries. The objective is to flesh out the recommendations of this report, establish partnerships, and begin the process of developing quality control mechanisms, setting standards for selection and archiving of materials, and so on.

Core Partnerships

The SMETE Library provides a service; it is not a research project. The recommendation is that the technical work of the library, including the core services and collections, be carried out by a number of partners under central coordination. Eventually the SMETE Library organization will include:

overall leadership by the NSF,

a central team that has operational responsibility for central functions, such as recruiting and working with partners, and the development of guidelines for organizing the resources,

core partners who take responsibility for specific parts of the SMETE Library. These might be organized by disciplines, or by other attributes,

other partners who contribute to the library in numerous ways. They could include societies, commercial organizations, colleges and universities, etc.,

an associated research program, carried out jointly with several divisions of the NSF and other federal agencies.

The NSF will have the responsibility of ensuring that the right organizational structure is established and that the core partners are appropriately responsive to the SMETE Library goals.

"Developing a Digital National Library for Undergraduate Science, Mathematics, Engineering, and Technology Education: Report of a Workshop." National Research Council Center for Science, Mathematics, and Engineering Education, 1998. (http://www.nap.edu/readingroom/books/dlibrary/)

"Information Technology: Its Impact on Undergraduate Education in Science, Mathematics, Engineering and Technology: Report on an NSF Workshop", National Science Foundation Directorate for Education and Human Resources, Division of Undergraduate Education, National Science Foundation, NSF 98-82, April 13, 1998.

The following agenda was circulated before the workshop. Minor alterations of timing were made during the actual meeting.

AGENDA

SMETE-Lib Workshop July 21 to 23, 1998

National Science Foundation Arlington, VA

Overview of the Agenda

The agenda for the workshop concentrates on two topics, which have been identified by the Steering Committee and the staff of the National Science Foundation.

Scope and features

There are a variety of concepts for the form that a digital library, or libraries, for undergraduate education might take. They differ in the intended audience, the types of material collected, the methods of organization, and how they might impact education. The first topic is to categorize these options and to list the educational benefits associated with each.

Priorities

The second topic builds on the list of concepts generated by from the first topic. Since it is not possible to address all options at the same time, the task is to recommend priorities. What should be done first and why?

To address these topics, members of the workshop will be divided into breakout groups with approximately 15 to 20 people in each, chaired by members of the Steering Committee. Members of the workshop will be pre-assigned to groups, with the aim of having all viewpoints represented in each group.

Tuesday, July 21 National Science Foundation, Room 375

Take the North Elevator to the second floor of the NSF building and sign in at the security desk.

Wednesday, July 22 National Science Foundation, Room 375

8:00 - 8:30 am

Coffee

8:30 - 8:45

Welcome Frank Wattenberg, NSF

8:45 - 9:15

Background: Review of the National Research Council study, "Developing a Digital National Library for Undergraduate Science, Mathematics, Engineering, and Technology Education", August 1997 Jack Wilson, RPI

A digital library can respond to needs articulated by the National Science Foundation, the academic community, and corporate leaders for accelerating and spreading much needed improvements in science, mathematics, engineering, and technology education.

Capitalizing on recent developments, digital libraries can provide the following.

A forum for the merit review and recognition of quality educational resources.

A mechanism for electronic dissemination of information about high-quality educational materials, pedagogical practices, and implementation strategies.

A centralized registry and archive for educational resources and a mechanism for their dissemination.

A resource for research in teaching and learning.

A workplace facilitating cooperative work with shared educational resources.

The purpose of this workshop is to identify and address issues related to content development and faculty use of digital libraries for SMET undergraduate and graduate education. It will bring together faculty who are potential users, authors who are potential contributors, and other stakeholders with a national library's potential architects to articulate functional capabilities and standards that will enable such a library to reach its goals. As noted in the National Research Council report: Developing a Digital National Library for Undergraduate Science, Mathematics, Engineering, and Technology Education, input and advice from potential library users are critical to the design and development of the library and its content.

Participants

The workshop will include roughly 100 participants representing the following groups.

Content developers - leaders in exploiting the potential of technology, the Internet, the World Wide Web, and multimedia to improve the quality of SMET resources. This group will include representatives from all major SMET disciplines and developers exploiting a variety of technologies.

Users - this group overlaps the first group and will include classroom faculty at all levels and from a variety of educational institutions and workplaces.

Students

Digital library specialists - especially representatives from the six Digital Library Initiatives and the NLII.

Software and component developers.

Educational researchers.

Resource people representing a variety of interests and expertise including publishers, professional societies, businesspeople, and information specialists.

Experts on evaluation and assessment.

Issues

Enlarging the community engaged in the improvement of SMET education and supporting users new to information technology.

Support for assessment and evaluation of courses, curricula, and student learning.

Requirements to enable the sharing of data, tools, and other basic resources and insure that library resources work well together and can be maintained as the underlying technologies evolve.

Kinds of support and resources needed for authors to stimulate the production of high quality resources.

Use of reusable, general purpose educational components to address questions of maintenance of library resources in a rapidly changing technological environment, integration across disciplinary and institutional boundaries, cost, and accessibility. (A component is a resource - for example, a graphing package or a simulation - that is written to functional specifications and can be used as part of many different items in the library. Because components are written to functional specifications, they are interchangeable, allowing for competition among software producers and upgrades that do not require rewriting the items that use them.)

Support for distributed, cooperative work with shared resources and tools.

Support for, and dissemination of results from, research in teaching and learning, including research to inform continuous improvement of digital libraries.

Product

The workshop will outline a report for general distribution addressing the issues raised above that will be written by a working subgroup and issued in Fall 1998. This report will contribute to a needs assessment or statement of requirements from a users' perspective, informed by digital library and other specialists.

Faculty, teachers, software developers, publishers, and a host of others across the United States and around the world are contributing to the development of rich and engaging new environments for active, inquiry-driven learning. This revolution draws much of its vigor from the contemporaneous technological revolution that has put enormous computational power on desktops and in backpacks; that has created networks that bridge distance, discipline, institution, and even culture; and that has created a new generation of powerful, flexible, and inexpensive experimental equipment. The same technology that is playing a major role in the improvement of learning and teaching in the best classrooms and laboratories also has the potential to extend the best education to all students.

The World Wide Web is a medium made for this message. In its brief existence, it has demonstrated extraordinary potential to both disseminate rich learning resources and to support the new communities required for their effective use. The richly linked architecture of the Web matches the richly linked architecture of knowledge, supporting multidisciplinary perspectives and links between research and education. The open architecture of the Web allows us to use a browser as an orchestra conductor, coordinating learners' work with a powerful set of resources and tools. But, the Web has shortcomings – developers often squander precious time with multiple reinventions of the same wheels; it is often difficult to find high quality and appropriate material; and interplatform and intertemporal interoperability are often more rhetoric than reality. The very pace of technological innovation often outruns reflective analysis of its use in science, mathematics, engineering, and technology (SMET) education.

Although the World Wide Web has had a major impact on education, it is a first generation effort, and SMET education is only one of its many uses. In some sense, we are here today to talk about a second generation effort – a true digital library – in which SMET education is the primary focus.

Creation of a national digital library for SMET education presents the challenge of overcoming the Web's weaknesses without losing its strengths – providing selectivity without excessive screening, providing a measure of stability and reliability without being overly restrictive in its operating rules. Such a library should provide an open, robust, and reliable framework for significant advances in SMET education at the same time as it serves as a vehicle for dissemination and study of the best and most current SMET education content and practice. It must have powerful mechanisms to help users find reliable, high quality, and appropriate resources. The success of such a library will be measured in terms of its effectiveness in multiplying the impact of the best work in SMET education and research; its ability to foster and support new communities working with shared resources and tools; and the extent to which it provides a framework for progress in which new developments are able to build on a solid foundation.

You are here as potential users of a national digital library for SMET education to articulate the features that you want in your library. It is important clarify the institutional and human features that are needed as well as the technological features. You are here not only to represent the needs of your disciplines, but also to apply your expertise in these disciplines to education and to the library. Our disciplinary expertise will play a key role in the success of this enterprise.

We have an opportunity to make a real difference. By building on investments by NSF and others in fundamental network and digital library technology, we can build scaffolding for the future – a future in which scientists, mathematicians, engineers, and experts in education collaborate and build on each other's work, improving the quality of SMET education and extending the reach of the highest quality SMET education to all students.

Jack M. Wilson, Rensselaer Polytechnic Institute, "Review of the National Research Council study, 'Developing a Digital National Library for Undergraduate Science, Mathematics, Engineering, and Technology Education' "

Dr. Jack M.Wilson provided background information to the SMETE-Lib Workshop participants on Wednesday morning, July 22, by reporting on the results of the August 7-8, 1997, workshop, National Research Council Workshop on a Digital National Library for SME&T Education, which was undertaken by the NRC Computer Science and Telecommunications Board (CSTB) and the NRC Center for Science Mathematics, and Engineering Education (CSMEE). Dr. Wilson began by saying, "I am amazed that things have changed so rapidly in just one year. Many things about which we were worried have been resolved."

The 1997 workshop had been organized on short notice, and 200 invitations were issued. Fifty participants attended. The issues discussed can be broken down into four categories, each of which engendered a set of questions:

Curricular, pedagogical, and user issues

Who are the potential users?

What types of materials do they need?

What kind of impact can be expected?

Logistic and technology issues

What kind of oversight is needed?

What kind of technology exists now to build the National Library (NL)?

How can a multi-year project like this adapt to new technologies?

Economic and legal issues

How can we estimate or measure cost and benefits?

What are the longtime financial implications?

How could intellectual property rights, copyright, etc., be resolved? (Note: there is a community already at work on these rights issues. We can't make solving them a prerequisite to moving forward on an NL.)

Overarching issues

Is this a good idea?

Is this a better idea than investing in some other project?

Will an NL improve and enhance learning of SMET?

Dr. Wilson shared the following conclusions reached as a result of the study. 1) The Digital National Library for SMET should contain at least pointers to other resources. This would be cheaper than trying to provide all the content at one site; however, on the downside, there would be a lack of control over content if only pointers were provided. 2) Users should be allowed to add materials; this could be a role for publishers and professional societies. 3) It is critical that a decision be made at the start who would provide the editorial oversight, standards, and peer review. A system that is passive is not successful for education. 4) Creators could include faculty, societies, publishers, and students. 5) Better tools for browsing and searching are coming and will be important components. 6) Some people objected to the term "Library", but no one could come up with a better metaphor for what we are hoping to create. 7) The Digital National Library needs to become self-sustaining.

Finally, Dr. Wilson offered these recommendations:

Clarify who the customers of the Digital National Library will be.

Articulate priorities.

Develop an issue Request for Proposals (RFP)--either for one big effort or for several small models.

Make use of focus groups by discipline.

(A published report from the 1997 workshop is entitled Developing a Digital National Library forUndergraduate SMETE Education and is available from the National Academy Press, 2101 Constitution Avenue, NW, Lock Box 285, Washington, DC 20055. The report is also available online at <http://www.nap.edu/readingroom/books/dlibrary/>.)

The first major topic addressed by the workshop was the scope and features of a possible SMETE Library. The topic was introduced by William Y. Arms, chair of the workshop. The participants then divided into breakout groups who discussed this topic separately, before reporting back to a full plenary session.

The following statement of the topic was provided to the breakout groups. This statement was also mounted on the web site before the workshop.

Topic 1, Scope and Features

The agenda for the workshop concentrates on two topics, which have been identified by the Steering Committee and the staff of the National Science Foundation. The topic that will be addressed by the first set of breakout groups is:

What is the possible scope of a digital library for undergraduate science education? What features might the library have? What might be the educational impact of different features, scope, organization, and content?

The background to this question is as follows. There are a variety of concepts for the form that digital libraries for undergraduate education might take. They differ in the intended audience, the types of material collected, the methods of organization, the features and services provided, and their impact on education.

For faculty, the key question is, "If you had this service from the SMETE Library, would you use it, and how? What impact would it have on your teaching?" For learners, the question is, "Would this service from the SMETE Library help me study science? How? Would I use it as part of a course or when studying independently?"

Here is a list of some of the decisions that might be made in designing the SMETE Library. This is not a full list, and the breakout groups are invited to add or modify the topics.

Audience

The SMETE Library could concentrate on the needs of SMETE faculty. Or it could aim to serve students directly. It might ignore independent learners or give them special attention. Are these audiences alternatives? Or is it possible to conceive of a library that serves them all well?

Collection scope

The SMETE Library could concentrate solely on educational materials. Or it could be a general purpose science library, with both primary scientific materials and tools. It could concentrate on open access materials, or it could be linked to restricted access information, such as journal publishers. It could concentrate on formally published materials or include preprints, web sites and similar materials. Which would have the greatest impact?

Location of collections

The SMETE Library might have a large computer system where it collects science materials. Or it might be a federation of smaller libraries each of which maintains a specialized collection. Or it might be a virtual library, with no collections of its own, that provides access to collection maintained by other, independent organizations. What are the benefits (or otherwise) of each approach?

Information discovery and quality of materials

How does SMETE help people find information? Does it provide catalogs and indexes? Does it review educational materials and validate them for scientific content and educational impact? What kinds of reviewing and validation are needed to help users to find high quality and appropriate material?

Collaboration

What features might the library have to facilitate new kinds of collaboration? What impact would such collaborations have on teaching and learning?

The task is to categorize these options and to list the educational benefits associated with each. Each breakout group is invited to submit a list of options, with judgement of the educational impact of each. However, breakout groups should feel free to address the topic in whatever manner they find most productive.

Summary from Topic 1 Breakout Sessions

The first day of the workshop concentrated on defining the scope and features of the SMETE Library. Workshop participants broke out into five groups, each chaired by a member of the steering committee. The working groups met in two one and one-half hour sessions, with all participants reconvening in the afternoon to summarize the results of their discussions. The results of these discussions are presented below.

Audience

There was wide agreement among the workshop participants that the SMETE Library should be open to the widest possible range of students and faculty. As in the model of existing libraries and web resources, the library should serve as a venue for the discovery of information but it should also become a forum where the educational process is mediated and expanded by direct faculty to student, student to student, or faculty to faculty interactions. This new, interactive aspect of the library will offer one critical means for developing a virtual community of users. It is anticipated that this high degree of interactivity will facilitate broad collaborations across traditional disciplinary boundaries.

It was noted as important that the authors of the materials produced for inclusion in the library will also constitute a critical component of the overall audience. Direct contact between the authors and users will permit authors to incorporate feedback from the user community into future revisions of the materials that either expand the content or correct errors. In addition, it is very likely that many faculty as well as undergraduates themselves will author materials such as term papers or research reports that will be posted to the site. The seemingly distinct titles of faculty, student, and author will be blurred by the library.

A third component of the audience that will be addressed by the SMETE Library will include members of the research community and lifelong users. The library offers the unprecedented opportunity for researchers to archive and share the raw data from their research directly with students and other interested faculty. Research results can be checked by students and faulty alike, or combined with observations or methods from other disciplines to forge new conclusions. This aspect of the library will serve to close the perceived gap between teaching and research that exists for many faculty and students as well as the public at large. A vital, active SMETE Library could become a site of choice for access by lifelong users or others such as government or private employees seeking specific information about science, engineering, math, or technology education.

Collection Scope

The workshop participants agreed that the materials to be included within the library should be comprehensive and include all materials related to undergraduate scientific and technical education. For example, the collection should include traditional materials such as course syllabi and problem sets, as well as links to journal and commercial publishers web sites. Faculty and students alike may wish to include lectures notes or lecture outlines, and some faculty might wish to exchange quizzes and exams in a confidential manner. Other non-traditional materials such as expanded data sets (with some of these drawn from the raw data archives), student papers, computer routines such as Java applets for displaying certain interactions, assessment tools, bibliographic records, digital images, animations, and video should also be included. It was of particular importance that materials in the collection be in a form that would facilitate the ready adoption or adaptation of the content by faculty or students to their own particular needs. In order to increase the usefulness of the library, links to specific sites will need to be maintained and updated, and certain sorts of data such as raw data sets will need to be archived in order to ensure their longevity.

Participants generally agreed that the emphasis of the library should be placed on collecting new digital materials rather than simply digitizing the existing collections of traditional libraries. This emphasis should be expansive in scope, and include new technologies that permit the digitizing of three-dimensional objects (e.g., statues, sculptures, buildings, skeletal elements, molecules) for study and manipulation by the user.

Participants noted that future advances in digital technology are certain to permit a critical expansion of the scope of the SMETE Library, and the library should be responsive to these developments. One critical role for NSF will lie in identifying those areas of the collection where its impact could be significantly improved by the solicitation of specific materials. These materials could be commissioned by a request for proposals that would exactly meet this need.

The SMETE Library should provide a set of services in order to facilitate the effective and orderly cataloging of materials and its efficient discovery and retrieval. Some of these services include:

a standardization of taxonomies for categorizing information that cuts across discipline-specific cataloging systems,

search engines based on the standard taxonomy as well as discipline-specific keywords and topics.

Location of Collections

Participants of the workshop agreed that the exact location of the materials was of secondary concern to the idea that the library should appear to the users as an integrated body of information. Access to any of the individual bodies of information should be possible from multiple entry points to minimize the number of links between sites. There was broad support for the idea that the SMETE Library consist of a constellation of independent sites that are managed by a central facility. It was agreed that the collection should be stable and permanent, and that emphasis should be placed on the permanence of links so that access is guaranteed.

Information Discovery and Quality of Materials

The organization of the SMETE Library should easily facilitate the efficient discovery and retrieval of materials as well as interaction with the authors of the materials as well as with students and faculty who have or are currently using the materials. Some of these key attributes should include:

availability and a listing of any services or recommendations for adaptation and adoption,

bibliographic records,

use history,

user profiles that enable automated notification of new materials or updates to existing materials of interest,

links to related courseware (private and commercial),

on-line training in the use of the materials,

human help/training in the adoption or adaptation of the materials.

The issue of quality was one of particular interest among the workshop participants. It was noted by many of those present that one of the strengths of the web in its present configuration is that it is available to anyone who wishes to post any content. One drawback, however, is that much of the material is of limited or no utility. Participants generally agreed with the notion that some form of rating system should be used with the goal of permitting users to better identify those materials that will be of particular value. Some of the possible rating systems include:

peer review of materials,

testimonials,

reviews from users (faculty, students),

learning assessment results.

Collaboration

One the key attributes of the library is the unprecedented level of collaboration that it will facilitate among all members the SMETE community. Key features that will encourage collaboration across the different library audiences include:

student chat groups,

links to other users of the specific materials,

links to the author of the materials,

a history of use that details educational level and utility.

The library should not be permitted to become a simple repository for teaching materials but should instead be designed so that it will evolve into a dynamic entity that integrates its materials into an educational framework that has at its center faculty and student input into the educational process. In this way the SMETE Library will become the vehicle for the development of a community of faculty and students that will lead the country into the next millennium.

(This summary was written by John Kappelman, University of Texas, Austin.)

The second major topic addressed by the workshop was the priorities to be given to the various features of the SMETE Library that were identified by the first group et of breakout groups. The topic was introduced by three short talks by members of the Steering Committee.

Synthesis, an NSF engineering education coalition (www.synthesis.org), has been at the forefront of engineering education reform since 1990. NEEDS is the distributed architecture developed by Synthesis to enable new pedagogical models based on Internet-mediated learning environments. NEEDS currently catalogs and distributes courseware and other instructional technology developed nationally and internationally to provide a resource where both instructors and learners can search, access, and download educational materials. In addition, NEEDS supports a multi-tier courseware evaluation system including a national award competition -- the Premier Award for Excellence in Engineering Education Courseware. The presentation will provide a brief description of NEEDS and our lessons learned over the past eight years.

Each bibliographic record in NEEDS describes the pertinent information about the courseware, in the same manner that traditional on-line public access catalogs provide information on books (i.e., title, author, publisher, subject heading, keywords, etc.). A user can search for courseware by entering terms into a World Wide Web form -- Title, Author, Subject Heading, Keywords, or Platform. NEEDS then performs a Wide Area Information Service (WAIS)-indexed full text search to provide a ranked list of courseware. (The higher the ranking, the closer the courseware matches the requested query.) The user can then view the bibliographic record that describes the courseware and provides hyperlinks to download the courseware for different platform(s). NEEDS goes beyond the traditional on-line library catalog by providing recursive search capabilities and additional guiding and organizing structures. Hyperlinks on the courseware record provide access to related information indexed along multiple axes (i.e., author, publisher, subject heading, and courseware series). Materials held in NEEDS are diverse -- content ranges from single topics that can be covered in a few minutes to fully integrated, term-long courses. One of the most powerful concepts supported by NEEDS is courseware modularity. NEEDS has the capability of cataloging courseware as well as the individual elements (e.g., images, videos, and text) that comprise the courseware. Courseware elements provide a vehicle for continued re-use of content material beyond the life-span of any particular courseware module. The Virtual Disk Drive Design Studio courseware will be shown as an illustrative example.

NEEDS has been driven by feedback from our user community in engineering (faculty, students and staff). Based on this feedback, quality review of courseware, author-initiated cataloging, and extended searches have been added to the functionality. NEEDS has benefited by always building upon international standards for data description (USMARC and emerging metadata descriptors developed by the Educom Instructional Management Systems (IMS) project) data access (http and the World Wide Web), and production database technology (using structured query language (SQL) relational database to store and index courseware records). Through collaboration with the research community, NEEDS has also been able to adopt new technologies as appropriate. Future implementations will focus on: expanding the user community, providing author feedback on use/download statistics, adding feature reviews and introducing new modes for on-line discussions.

Wade Ellis, Jr., West Valley College, Saratoga, California, A Personal View of the National Digital Library: How Will I Use It?

As a community college mathematics instructor whose students can benefit from new computer-based mathematical tools and inquiry-based teaching methods, who authors instructional material, who is active in professional societies and in faculty development, I am very interested in the possibilities of the National Digital Library. I would like the NDL to produce and maintain standards so that I could easily access, review, adapt and adopt the variety of new materials that have been sponsored by the NSF and other organizations that are now available in electronic form. It would be helpful to me to have standards for written text, graphics, mathematical and scientific symbols, methods of changing instructional materials, formats for specifying equipment, hardware, and software requirements for instructional materials, and for managing updates for software and hardware. With these standards, I would be able to modify and adapt these new materials in developing my courses (as I do now with traditional textbooks). Making these materials available to my students would be greatly facilitated by efficient and powerful tools for searching for materials, for modifying and authoring materials, for creating animations and simulations, and for complicated computations (graphical, symbolic, numeric, and combinations of such computations).

To increase the rapidity with which such materials are disseminated among the faculty of higher education institutions, it would also be nice to have collaboration tools that would allow easy communication with students and peers, allow the sharing of graphics and symbol-laden documents and large documents, and allow for instructional materials to be created by groups of faculty members spread across the Internet. Of course, all of these services should focus on the improvement of student performance.

As a community college instructor, I am aware of many difficulties that community college faculty will face in using the powerful capabilities of computer technology in the coming century. Though we all know the physical and financial problems that such faculty face in terms of outdated equipment, slow telecommunication lines, and expensive software, we do not often talk about how difficult it is to gain access to training and to peer level discussion of technical and academic issues surrounding the use of technology. The National Science Foundation should make available the resources and means for community college faculty to gain access to the equipment and training that will be necessary for the dissemination of National Digital Library materials and services to community college students (as it did in the initial states of the use of the Internet).

From the presenter's perspective, the NSF SMETE Library would include a combination of content and services delivered via a distributed, virtual presence. The content itself would use open interoperability specifications, such as the IMS Specifications. A strong central presence would be required to provide core services that other affiliated organizations were not able to or were not interested in providing. NSF would, via the central site, provide the means to achieve improvements in quality and access including requirements, development, research, organizational relationships, policy with NSF award winners, and quality control.

The IMS Initiative addresses a class of software known as instructional management systems or learning server that generally runs on a server and distributes educational or training content to learners on a network, supports collaboration, and records student performance data. Examples of this type of software include, Lotus LearningSpace, Web Course in a Box, TopClass, and others.

If examples exist, then why does the IMS Initiative exist? There are many reasons including the fact that for interactive content there is little interoperability, i.e., the ability to take content from multiple authors and run it, unchanged, from multiple learning servers. Given this lack of interoperability development, costs are high, as publishers have to support multiple proprietary learning server architectures. This increased cost forces publishers to focus on high payoff content areas such as introductory math and English, which have high enrollments nationally. A related problem is the difficulty in finding content. If it is difficult to find, then reuse is problematic at best. Additionally, the lack of an effective electronic commerce solution for small pieces of content threatens the viability of a marketplace in such content.

The solution to these problems is the IMS Specifications, which are a comprehensive set of rules designed to enhance communication between on-line learning systems. The IMS Specifications describe the critical interchanges between users, faculty, content publishers, and service providers, such as schools.

The Initiative's goals are to build the Internet architecture for online learning by making freely available specifications and "starter" software available to developers. The project is composed of 32 investment member organizations (Australia and Virginia Tech have joined since the presentation) including major University systems, branches of the US Government and most of the major technology, platform, and authoring tool companies. The project has an international presence with affiliated centers located in the UK and Australia.

The elements of the scope of the specifications that relate to the SMETE-LIB effort includes Profiles, Meta-data, and Content.

Profiles are collections of information about learners and instructors, and are under their control. The types of information include personal, performance, portfolio, and preferences. Performance information granularity is not specified and, therefore, can include information found in course grade books to transcripts. Preferences include data that an authoring tool, learning server, or content might use to provide a more customized learning or content authoring experience.

IMS Meta-data terms are used during a cataloging or authoring process to describe the intent of educational tools or content. The terms include Attribution, Catalog Entry, Concept, Contact, Coverage, Description, Difficulty, Discipline, and InstallationRemarks. Dublin Core terms are incorporated into the dictionary of IMS terms. IMS Meta-data allows for specialization of terms beyond the base set by different discipline and industry groups. Currently, IMS and NIST (National Institute for Standards and Technology), have created a repository to store the collections of sets of meta-data terms. Over time, it is anticipated that this collection of sets of meta-data terms will become distributed.

The IMS Meta-data system will eventually include a full set of programmatic interfaces allowing authoring tools and content repositories to work with multiple sets of meta-data terms, and leverage learner and author profiles to simplify and customize the process of cataloging, authoring, and finding content.

The key intersections between functionality, likely to be present in a SMETE Library, with IMS includes Meta-data, discipline-specific metadata taxonomies, content repository services (such as distributed searching) and service value-chains. Value chains would include those workflow processes that involve multiple organizations with roles in supporting an effective learning experience for students. IMS-defined characteristics of content likely to be of interest to users of a SMETE Library include capturing data about student use, enabling collaboration, and tracking data about students to enable customization of learning experiences.

Suggestions for how to get involved in the IMS Project included requesting IMS compliance from instructional technology vendors, providing input and requirements, using IMS meta-data, and advocating the development of discipline-specific meta-data.

The Web site for the IMS Initiative is http://www.imsproject.org/. Other project management include: the Project Director, Mark Resmer (resmer@earthlink.); the Director of Market Development, Denis Newman (denis_newman@earthlink.net); and the Director of the IMS Developer Network, Frank Tansey (tansey@sonoma.edu).

Breakout groups on priorities

The participants divided into breakout groups who discussed the topic of priorities separately, before reporting back to a full plenary session. The following statement of the topic was provided to the breakout groups. This statement was also mounted on the web site before the workshop.

Topic 2, Priorities

The agenda for the workshop concentrates on two topics, which have been identified by the Steering Committee and the staff of the National Science Foundation. The topic that will be addressed by the second set of breakout groups is:

The first breakout session listed many options for the SMETE Library and their educational impact. Since it is not possible to address all these options at the same time, what should be the priorities? What should be done first and why? It is particularly important to identify those features that have high priority because other features depend upon them.

Here are some of the issues that might be considered by the breakout groups. This is not an exhaustive list, and the breakout groups are invited to add or modify the topics.

Leverage

A large, coordinated project, such as the SMETE Library, can be more than the sum of its parts. Are there separate activities that SMETE can bring together? Are there existing, fragmented activities that can be combined as the initial nucleus of SMETE?

Enthusiasm and imagination

How can SMETE build on the enthusiasm and imagination that the best faculty and students bring to education?

Building on existing efforts

Many desirable developments are already happening and will take place without the SMETE Library. If something will happen anyway, there is no need for a new initiative. Conversely, the core funding for the SMETE Library will come from the NSF and should concentrate on activities that will not happen without NSF funding.

Educational value

What are the most serious problems with existing educational resources on the World Wide Web and elsewhere? What can the libraries do that will increase their educational value and impact? What are the pivotal needs that can best be addressed by NSF's leadership?

Long term sustainability

The SMETE Library is intended to be a permanent part of the educational landscape in the United States. What can be done now that builds for the long term?

The reports from these breakout groups complete the workshop. Ideally, the session will articulate one or more strategies to create a digital library framework for scientific education, that will provide immediate benefit, with high potential to enhance the quality of education in the long term.

The final session of the workshop, which began with reports from the breakout sessions on priorities, was used as an occasion to synthesize the ideas and opinions expressed by the participants during the workshop. Thus, Section A of this report represents both the overall recommendations of the workshop and a summary of the reports of these breakout groups.

The following information is extracted from the program announcement for the NSF's Digital Libraries Initiative, Phase 2:

To explore the linking of digital library research efforts and testbeds for undergraduate education, NSF's Division of Undergraduate Education will provide a total of $500,000 for planning and study projects in FY 1998. Successful applicants are expected to demonstrate high potential to advance undergraduate science, mathematics, engineering and technology (SMET) education. Three types of proposals are of interest: practical digital library applications for SMET education, technical studies of digital library capabilities, and general policy studies.

The full announcement is at http://www.nsf.gov/pubs/1998/nsf9863/nsf9863.htm

For further information, please see: Special Opportunity for Applications and Testbeds for Undergraduate Education.

Last revised October 29, 1998 WA/BW

The SMETE-Lib web pages and mailing list are hosted by D-Lib with support from NSF's Division of Undergraduate Education.