DelosDLMS - Infrastructure for the Next Generation of Digital Library Management Systems

by Hans-J. Schek and Heiko Schuldt

The overall goal of the DelosDLMS is the implementation of a prototype of a next-generation digital library management system. This system combines text and audio-visual searching, offers personalized browsing using new information visualization and relevance feedback tools, allows retrieved information to be annotated and processed, integrates and processes sensor data streams, and finally, from a systems engineering point of view, is easily configured and adapted while being reliable and scalable. The prototype will be built by integrating digital library functionality provided by the DELOS partners into the OSIRIS/ISIS platform, a middleware environment developed by ETH Zürich and now being extended at the University of Basel.

In the first two years of DELOS - the EC-funded Network of Excellence on Digital Libraries, work has mainly focused on improving digital libraries (DLs) by developing independent, powerful and highly sophisticated prototype systems. Current integration activities began in early 2006 and are coordinated by the architecture work package of DELOS. These prototype systems are being integrated as building blocks into OSIRIS/ISIS, an existing middleware environment that was developed at ETH Zurich. The result of the integration – that is, the middleware infrastructure together with all the advanced DL functionality – will constitute the DelosDLMS. ISIS and OSIRIS

A central task in the second phase of DELOS is the development of a global prototype. The objective is to build a joint prototype for the future Digital Library Management System that makes available results from many groups in DELOS. This will be based on the OSIRIS/ISIS middleware, the development of which began at ETH Zürich for ETHWorld, the virtual campus of ETH. It was further developed for data streams and for medical objects at UMIT, and is currently being extended at the University of Basel. The OSIRIS middleware (Open Service Infrastructure for Reliable and Integrated process Support) supports programming-in-the-large; ie, the combination of arbitrary application services into so-called processes. This is realized by a set of generic (application-independent) services that include the registration of services and processes, interfaces for application development, an engine for decentralized execution of processes, and services for load balancing. In addition, it features reliable execution by applying advanced database concepts - essentially for failure handling and concurrency control – at the level of processes. ISIS (Interactive SImilarity Search). is a set of DL services that have been developed on the basis of the OSIRIS middleware. ISIS includes a sophisticated index structure for similarity searching, which is particularly well suited for high-dimensional vector spaces. Furthermore, in terms of Digital Library functions, ISIS features rudimentary support for textual and content-based audiovisual searching. It also provides basic support for relevance feedback and visualization.

With the DelosDLMS, existing ISIS services will be significantly enriched by other specialized DL services that have been developed within the DELOS network. This will be achieved by integrating these services into the OSIRIS infrastructure, thereby combining them with other ISIS and non-ISIS services into advanced, process-based DL applications.

The DelosDLMS The plan for the DelosDLMS includes the upgrading of existing ISIS components and services and the integration of new functionalities. The final product will support multi-object multi-feature queries over collections of different media types. Personalized browsing and information access, relevance feedback and object annotation will also be considered. Since information (for instance in e-Science Digital Library applications) increasingly originates from software or hardware sensors, sensor datastream processing will also be integrated in the DelosDLMS. Essentially, all this DL functionality will be made available by means of services. The challenge of DelosDLMS is therefore to provide a scalable and reliable infrastructure where these services can be plugged in and used as building blocks.

Two alternatives exist for integrating services with OSIRIS. First, there are tightly coupled services, which are tightly integrated into the OSIRIS runtime infrastructure. Advanced failure handling and load balancing are among the main advantages of this arrangement. In terms of failure handling, compensating services can be registered which are automatically invoked in case of failures. In terms of load balancing, ORISIS can automatically choose the node carrying the lightest load to invoke a service that is deployed several times. This is particularly important for computationally expensive services like feature extraction.

Second, services can be loosely coupled with OSIRIS, meaning that services are described and invoked by standard Web service interfaces (SOAP and WSDL). This reduces the effort needed for integration but does not provide the benefits of tight coupling.

Recently, a 'call for services' has been issued to both members and non-members of DELOS. The goal is to identify services which are best suited for integration into the DelosDLMS. For the first version of the integrated DelosDLMS prototype, services will in most cases be loosely coupled. The final version will then support a higher degree of reliability by tightly coupling as many services as possible. Services will be integrated from the following areas:

sophisticated term extraction from text, text indexing and collection management

annotation services

reliable sensor data management

multimedia indexing

automatic search process generation and personalization services

image feature extraction

3D shape recognition

special indexing techniques for video retrieval

audio feature extraction and audio retrieval

advanced visualization services and visual relevance feedback

Self-organizing maps visualization

active paper (linking digital information and paper)

services for transformations between standards

ontology services and natural language access

preservation services

services for multi-lingual access.

This list will be extended and revised after the evaluation of the call for services and during the actual integration work. Nonetheless, it highlights examples of building blocks that will be considered for DelosDLMS.