Grid'5000 user report for

User information

Experiments

Consistency models and protocols experimentations within the JuxMem platform (Middleware) [achieved]Description: JuxMem (Juxtaposed Memory) is a data sharing service for grid computing which provides transparent localisation and data persistence in a dynamic environment. Distributed Shared Memory systems and Peer-to-Peer systems are both sources of inspiration for such an hybrid service. That's why JuxMem relies on the Sun's JXTA (Juxtaposed) platform - a generic P2P platform - to address large scale, dynamic environments. JuxMem is also an experimental platform to test several fault tolerance strategies and data consistency protocols combinations. This set of experiments addresses the problem of efficient visualization of shared data in grid environment based applications. Starting from the entry consistency model, we design an extension of the entry consistency protocol that allows readers to access data without acquiring the lock. The goal is to improve data observation without degrading the performances of regular reads and writes. This new kind of access is named "relaxed read". Because Grid'5000 provides nodes from clusters located in different cities, matching our hierarchical architecture model, we choosed g5k as the main testbed for our experiments. These tests gather up to 30 nodes, spreaded in 3 different cities, automatically deployed by the JDF (JXTA Distributed Framework) deployment tool. The JuxMem version used here is the Java binding of the platform that needs a Java Virtual Machine installed on each node.
More informationResults:More information here

Perfomance scalability of the JXTA P2P framework (Middleware) [achieved]Description: JXTA (Juxtaposed) is a framework developed by Sun Microsystems that proposes a set of standard P2P protocols. JXTA is part of several scientific and industrial projects. However, few experimentations have been driven to evaluate performance scalability of current C and J2SE implementations.
With Mathieu Jan, we use the Grid'5000 testbed to monitor peerview size regarding to the number of nodes involved in the network, the initial topology as well as other JXTA-oriented parameters (up to 580 nodes used from 6 sites simultaneously). Initial results were presented at the Grid'5000 spring school: http://www.grid5000.fr/school2006-slides/school2006-retours-jan-jxta.pdf and won the best presentation award.
In a second time, we try to evaluate the time needed to publish and retrieve advertisements in a JXTA overlay composed by both rendez-vous and edge peers. Experimentations require 2-hour reservations from 100 to 700 nodes, using 9 Grid'5000 sites. This work leads to a paper at IPDPS 2007.
The deployment of all JXTA peers is achieved by the ADAGE tool and its associated JXTA plugin. A dedicated Kadeploy Debian-based image for JXTA is currently developed for future sets of large-scale experiments.Results:More information here

Building a hierachical distributed memory for the grid (Middleware) [achieved]Description: Managing data over distributed applications is a real challenge regarding properties such as data availability, persistency, consistency or transparency. Over the last decade, the computational grid, mainly cluster federations, has emerged as the natural infrastructure for code-coupled applications such as numerical simulations or co-operative design. These applications generate a large amount of data that have to be efficiently shared among participants. JuxMem (Juxtaposed Memory) is a data sharing service for the grid based on both DSM (distributed shared memory) and P2P (peer-to-peer) paradigms. It offers a global transparent access to data stored in physical memory as well as a generic platform to test different fault tolerance strategies and data consistency protocols. Gfarm is a grid file system designed to offer a global virtual file system for high performance distributed and parallel applications. GFarm proposes mechanisms for file fragmentation, replication and smart replica selection. We propose to add secondary persistent storage capabilities to the JuxMem service, using the Gfarm file system. Different common architectures are discussed to benefit from the the complementary properties of both systems. Experimentations show that our proposition provides efficient accesses to shared data thanks to the JuxMem service while offering large storage capabilities and persistency guarantees thanks to the Gfarm FS. This work has been conducted in collaboration with the Gfarm team, Tsukuba, Japan.Results:

CoRDAGe: Co-deployment and re-deployment of grid applications (Middleware) [in progress]Description: Latest experimentations of both JuxMem prototype and JXTA framework involve up to 580 nodes reserved from the Grid'5000 testbed. Deploying applications on such a large scale requires dedicated tools or scripts to automate code and data transfers, node startup and results recovery. The current approach addresses the problem of initial deployment through a static fashion. Projects like ADAGE or JDF are able to transfer code and input files to a given set of computational nodes but, as far as today, cannot offer additional services during the application's life. We think that the deployment of an application should not only provide this initial step. As a piece of example, we can consider a JuxMem service (see below) running on a set of nodes. If a client asks to store a data that requires more nodes than JuxMem is able to provide, the allocation request actually fails. Therefore, the initial deployment roadmap should not be a limitation for the execution of the application and new nodes should be reserved, deployed and started to extend the JuxMem network on-demand. This is the dynamic approach, where deployment is driven by application's needs during its whole execution time. We are currently working on the specification of a dedicated layer called CoRDAGe, that interfaces distributed applications with grid middlewares in charge of node reservation and deployment. As a proof of concept we develop a prototype that makes possible the interactions between applications (JuxMem, Gfarm), and grid middlewares (the OAR node reservation tool and the ADAGE deployment tool). Please check out http://gforge.inria.fr/projects/cordage/ for project status and source code.Results:More information here

{A}s more and more large-scale applications need to generate and process very large volumes of data, the need for adequate storage facilities is growing. {I}t becomes crucial to efficiently and reliably store and retrieve large sets of data that may be shared at the global scale. {B}ased on previous systems for global data sharing (global file systems, grid data-sharing services), this paper proposes a hierarchical approach for grid storage, which combines the access efficiency of {RAM} storage with the scalability and persistence of the global file system approach. {O}ur proposal has been validated through a prototype that couples the {G}farm file system with the {J}ux{M}em data-sharing service. {E}xperiments on the {G}rid’5000 testbed confirm the advantages of our approach.

{N}owadays large-scale, grid-aware applications are intended to run for days or even weeks over hundreds or thousands of nodes. {T}his requires new, and often painful operations for the user in charge of deployment and monitoring. {W}e claim that the applications should themselves manage their run in an autonomic way, by requesting new resources on-demand. {I}n this paper, we introduce {C}o{RDAG}e, a third-party tool, standing between applications and lower-level grid management tools. {I}t provides generic and application-specific facilities to dynamically expand and retract the deployment of a grid-aware application according to its actual needs. {A} prototype has been implemented and a preliminary testing has been conducted on the {G}rid'5000 testbed.

{W}e claim that building a distributed {DBMS} on top of a general-purpose grid data-sharing service is a natural extension of previous approaches based on the distributed shared memory paradigm. {T}he approach we propose consists in providing the {DBMS} with a transparent, persistent and fault-tolerant access to the stored data, within a unstable, volatile and dynamic environment. {T}he {DBMS} is thus alleviated from any concern regarding the dynamic behavior of the underlying nodes. {W}e report on a feasibility study carried out with our {J}ux{M}em grid data-sharing service built on top of the {JXTA} peer-to-peer platform.