Virgin Atlantic is preparing for a significant increase in data as it embraces the internet of things, with a new fleet of highly connected planes each expected to create over half a terabyte of data per flight.

Speaking to Computerworld UK at the Economist Technology Frontiers event, Virgin Atlantic IT director David Bulman said that the airline company is expecting an “explosion” of information generated from a growing number of sources, from employees and customers to cargo containers and planes.

In particular, the introduction of Boeing 787 aircraft – ordered by Virgin Atlantic for delivery in 2014 – is expected to dramatically increase the volume of data the airline will need to deal with.

“The internet of things, in a broad sense, is where we are starting to see everything from planes to cargo devices getting connected,” Bulman said. “The latest planes we are getting, the Boeing 787s, are incredibly connected. Literally every piece of that plane has an internet connection, from the engines, to the flaps, to the landing gear.

He continued: “If there is a problem with one of the engines we will know before it lands to make sure that we have the parts there. It is getting to the point where each different part of the plane is telling us what it is doing as the flight is going on.”

This level of operational insight will involve generating large amounts of data from each 787 aircraft, he explained. “We can get upwards of half a terabyte of data from a single flight from all of the different devices which are internet connected,” Bulman said.

Meanwhile, customer-facing initiatives mean passengers can be tracked throughout their journey, with the use of RFID tags embedded in mobile devices.

“There are a whole range of things that are going on in the airline industry,” Bulman explained. “One that is potentially the most interesting – and the Scandinavians are ahead of this – is loading passport information onto devices. We are not taking part in this at the moment because the UK is not yet there, but there are airports such as in Copenhagen where you can get onto a plane without talking to a single person.”

With RFID tags to track cargo and, in the future, baggage, large demands are being placed on the airline’s IT infrastructure. According to Bulman a scalable cloud solutions will be required to deal with the increase in data, which the company has seen double in the past two years, a rate that is likely to quicken in the future.

“The challenge is what do you do with that amount of data when you are getting terabytes of data a day off your various airplanes? We are getting to the stage right now where we cannot deal with that much.”

He added: “If you are talking that level of data you can’t just chuck ten disks into your data centre anymore, you have to look at cloud based solutions and how you can store data.”

Bulman said that Virgin Atlantic is in the process of setting up private clouds for certain aspects of its business, and is discussing public cloud use for less sensitive data.

“We are doing some private cloud setups at the moment. We are in conversations with some public dealers, particularly for data storage and data analytics, to deal with some of these massive data sets.”

He added: “I cannot create the extra storage that we need off of my own back, given the increased amount of data that we are having to store, so we are having to work with other suppliers to find out how we can get more flexible data storage into our infrastructure.”

Making sense of the big data generated by the internet of things is an ongoing project, with Bulman explaining that the airline is currently looking at a number of software vendors, having had limited success with a Hadoop test in the past.

“We did a Hadoop trial last year, it didn’t go very far because we weren’t getting the intelligence out of it that we thought we would. So we are looking at some other initiatives with different vendors this year.

“We tried to put three different data sets together, and then tried to see if we could find some causality between the data sets that would gives us intelligence that would allow us to manage our operations better, and the trial itself didn’t find the causality between the data sets. So we didn’t find the linkages we hoped we would find.

“Whether that was how we set the trial up or the software I don’t know, so we are going to try some different things.”

He continued: “The thing about big data right now is that it is experimental; you have to try putting different data sets together in different ways to see if you can get the intelligence out that you want. The whole point is looking at the very fixed data sets with unfixed unstructured data sets, and sometimes it doesn’t work, so you try it again.”

The analysis of big data generated throughout its operations will have a number of benefits. On the customer side, Bulman said that this will give Virgin Atlantic greater knowledge of what its customers want, both to sell to them more intelligently and to improve its services.

It could also open up the possibility of pre-emptively flagging problems within aircraft themselves, he explained.

“As you move to a big data world you can start to see the trends in that data. You can move towards predicting what will happen with the plane so that you can do maintenance before a problem occurs, or look at where the efficiencies are and find out how to fly the plane differently to get better fuel efficiency.

“There is going to be some quite incredible information that we can pull from the data.”