You are here

Data Intensive Infrastructures Unit

The requirements to analyse and understand the inner workings and underlying fundamental concepts of the Web along with the elevated scalability requirements for the development of new Web infrastructures have demonstrated the necessity of Data Intensive Supercomputing (DISC) infrastructures to support researchers and developers.

A prominent characteristic of DISC (as promoted by major players such as Google, Yahoo and MSN), as opposed to classic supercomputing, is the importance of very advanced data management software over high-end hardware configurations. DISC data entries are in fact known to be formed by hundreds or thousands of commodity machines connected using common commercial networking infrastructures. It is then up to the software to be able to deliver high capacity, throughput, scalability, re-configurability and fault tolerance.

To be able to conduct credible research and development in the Web domain, DERI requires a DISC infrastructure. As DISC in itself is the subject of ongoing research, the goal of this work programme is to advance the research and applications of data intensive infrastructures, to create, maintain and offer to researchers a state-of-the-art DISC infrastructure, and to research novel algorithms and data structures for scalable handling of large amounts of semantic information.

This work programme will draw on DERI’s previous work on large-scale semantic search engines (SWSE) and will be done in cooperation with the Irish Centre for High-End Computing (ICHEC).

DERI is happy to report that Sindice and the Sindice Team have been awarded recently in two separate occasions.
Sindice was awarded the 1st prize at the European Semantic Web Techology Conference, Business Idea Contest . Stefan Decker presented on behalf of the team and apparently the pitch was very well received in fact. He received the prize itself during the ESTC gala dinner.

DERI's Pipe's Project (formerly Semantic Web Pipes) has been awarded 3rd prize at the Triplification challenge
Inspired by Yahoo's Pipes, DERI Web Data Pipes implement a generalization which can also deal with formats such as RDF (RDFa), Microformats and generic XML.
DERI Pipes are Open Source Software, ad as such they can be easily extended and applyed in use cases where a local deployment is needed.