Developers are adopting new streaming data frameworks and turning to microservices to meet the need to use data faster according to a report from Lightbend presented at today's Strata Data Conference being held in New York.

The premise which underlies the report starts is that the explosive growth in data, made possible by emerging technologies, has:

huge implications for how enterprises interact with data to create future business opportunities.

To explore the impact of so-called "Fast Data", Lightbend, creator of the Scala language that Apache Spark and many of the other most-used frameworks in big data are written in, surveyed 2,457 developers to examine adoption trends.

“To compete in the digital era, the necessity is rising for enterprises to use data faster. This need for speed is expanding beyond analytics to applications that adapt to changing conditions in real-time, personalize customer engagement, and power the internet of everything.”

The three key findings presented in the report are:

Fast data has clear business value - overall 60% of developers say their senior management understands the value of fast data although the pattern across indutries varies - insurance, construction and government being less likely to, which biotechnology and agriculture leading the field.

Enterprises will need batch and streaming for fast data -developers say 90% of their data processing workloads include a real-time component.

Developers are choosing new frameworks and languages based on fast data requirements.

Adoption of fast data technology appears to be happening three times faster than that of Hadoop and its biggest hurdle is in choosing the right tools and techniques, with the design phase of the software development lifecycle being the most problematic

Lightbend reports that while fast data tools are emerging at a rapid rate some are more on developers' rader than others, with Kafka leading in terms of current use:

It also argues that a key characteristic of fast data architectures is the use of microservices for streaming applications and reveals that 75% of developers with advanced fast data use cases in production rely on microservices:

The survey also found that as fast data use cases move into production microservices were increasingly valued for DevOps agility and predicatability.

In sum, data requirement are influencing the selection of techology with 37% of developers answering:

We’re choosing new frameworks based on their ability to handle data more effectively

and 18% answering:

We’re choosing new languages based on their ability to handle data more effectively

Donald Knuth, author of the classic The Art of Computer Programming, has celebrated his 80th birthday and is still working on improving his monumental work. He's also still hoping people will check ou [ ... ]