IBM has entered into a five-year research collaboration to develop technology to help scientists study data generated by a radio telescope with a collection area as wide as the US.

The €32.9m (£27.37m) endeavour between IBM and the Netherlands Institute for Radio Astronomy (ASTRON) was announced by the company on Monday. Scientists will use the technology to study the roughly two exabytes of data per day that is expected to be generated by the Square Kilometre Array (SKA) when it is completed in 2024. The data will help academics study galaxy formation, dark matter and the origins of the universe.

IBM is to collaborate on a project to study data from the Square Kilometer Array.Image credit: SPDO/Swinburne Astronomy Productions

"If you take the current global daily internet traffic and multiply it by two you are in the range of the dataset that the Square Kilometre Array radio telescope will be collecting every day," Tom Engbersen, a scientist with IBM Research, said. "This is big data analytics to the extreme."

Together, IBM and ASTRON will research advanced data transport processes, optical interconnect technologies and nanophotonics, along with next-generation storage systems based on phase-change memory technologies and tape systems, to help analyse the deluge of data likely to be generated by the SKA.

The SKA, which will be based in either Australia or South Africa, will use hundreds of antennas spread across a 3,000 kilometre wide landmass to give a collection area equivalent to one square kilometre. Once completed, it will be 50 times more sensitive than any former radio device. Each year, the SKA is expected to store between 300 and 1500 petabytes of processed data, compared to the 15 petabytes produced by the Large Hadron Collider each year.

IBM hopes the technology that will come from its collaboration will help it in its goal of designing a computer capable of an exaflop — 1,000 petaflops — of computing power, roughly 100 times more powerful than the world's current leading publicly disclosed supercomputer, the Fujitsu-designed K Computer. Chipmaker Intel also aims to develop technologies along the same lines as IBM so it can deliver an exascale system by 2018.