Smoking out data solutions – News

As information technology advances at pace, scientists have more opportunities to study new phenomena and discover new natural laws. To fully seize such opportunities, however, they must grasp what these new technologies can do and how research can exploit them. This is a demanding and time-consuming process for anyone without computer science expertise.

Now, help is at hand, with the user-friendly Chiminey platform, which provides a sophisticated computing and data management service. Ian Thomas and his colleagues at RMIT University in Melbourne, Australia, developed Chiminey. They present it in the Elsevier journal Big Data Research, along with a hands-on tutorial on how to use it.

These days, experiments with such high data demands are common in many scientific domains, including quantum mechanics, weather forecasting, climate research, oil and gas exploration, and molecular modelling. High-profile examples include the Large Hadron Collider, the world’s largest and most powerful particle collider, based at CERN, and the Sloan Digital Sky Survey, which processes astronomical data.

Thomas notes that scientists really want to engage with cloud and high-performance computing but are “often stymied by the initial learning curve associated with the new tools and techniques.” When using cloud computing platforms, for example, a scientist needs to learn how to set up an ‘execution environment’. That is, to create and set up virtual machines, collect the experiment’s results, and then destroy the virtual machines.

The Chiminey platform works by providing a set of drop-in components called ‘smart connectors’. Each one supports a certain computation type and provides a particular computing infrastructure. Smart connectors can interact with a service in the cloud and coordinate a cloud-based infrastructure for the user.

“The platform uses smart connectors that allow a researcher’s computational tasks to be packaged up and deployed on a number of different cloud computing and HPC platforms,” Thomas explains. As a result, the scientists can process their data without needing to know the execution environment’s exact set-up.

As well as being user-friendly, smart connectors are adaptable for different tasks. “Smart connectors can be flexibly adapted and extended for addressing researchers’ specific problems,” Thomas adds.

Data Science Tidings is a leading media platform for Data Science Evangelists and entrepreneurs, dedicated to delivering interesting innovative curated stories from the Data Science world. It aims to provide useful and latest curated feed on Data Science. It is a great destination to find the most fresh updates and murky strategies you have missed.