Taming the Big Data Beast

If you're not convinced the hype around Big Data is entirely justified, consider the following statistics: in the time it takes an average person to read this article, 72 petabytes of data (that's 72 x 1015 if you're counting) will have been added to the global information pool. Each hour, Walmart processes more than 1 million customer transactions, logging data into databases estimated to contain more than 2.5 petabytes of data - the equivalent of 167 times the information contained in all the books in the U.S. Library of Congress. Facebook uploads 500 terabytes of new information daily. The volume of global business data generated is doubling every 1.2 years.

The Big Data phenomenon is real. It's happening in every sector of the economy, from government to education to private business. In addition to the oft-cited storage, security and analysis questions Big Data brings, it also has an unprecedented opportunity to impact your company and its mobile strategy.

The explosion of mobile device use among business users, for example, is often cited by analysts as one of the main sources of today's data explosion. IT analyst firm Gartner predicts that by 2015 mobile phones will outnumber PCs as the most popular web access devices. Users with the ability to capture date on mobile devices, through bar codes, GPS apps, QR readers, or manual input, have greatly contributed to the growing data stream.

Big Data and mobility share much more than a cause and effect relationship or places on Gartner's list of top 2013 technology trends. They are linked in ways that offer game-changing value-creation opportunities to companies that effectively tie the collection of information to the delivery of real-time decision support to mobile executives. While members of the executive and IT teams continue the debate over data storage options, methods of securing and managing privacy concerns around customer data continue to linger. Mobility executives also face their own set of unique Big Data challenges including the best ways to analyze the dramatically expanding information flow.

If managed correctly, leveraging Big Data as a key component of your mobile development strategy can pay off in increased sales, better corporate performance and improved efficiency. If not properly managed, you'll simply be contributing to the $900 billion in information overload costs industries suffer every year.

More important, it's not just about Big Data, it's about getting the right data into the hands of the right person, at the right time. That means having the strategy, applications and tools necessary to ensure efficient two-way interaction between remote workers and their corporate data systems.

The core of the Big Data promise is developing a strategy that delivers key real-time information to remote users and allows them to access the specific information they need to perform critical tasks, i.e., prep for a sales call, manage a customer issue or drive a critical business decision. Keeping these same remote users insulated from extraneous information that is out of date or unnecessary must also be a part of an effective mobile data management strategy.

How you build mobility solutions that combine multiple data sets into a single usable data-driven application could make the difference between the critical app sales depends on and the one that is abandoned as irrelevant or too difficult to use. Recent studies have shown that sales reps access data from as many as 15 different sources to collect valuable information about contacts and prospects. And a significant majority of executives believe that their sales reps miss opportunities because they can't keep up with this information.

At the other extreme, forcing mobile sales executives to wade through seemingly endless emails and data streams to extract the specific information that could really make a business difference - that the customer has just been placed on credit hold and new sales orders are suspended, for example, or that the next appointment of the day has just cancelled allowing more sales time opportunity with the current prospect - can create equal challenges.

It's not just sales and customer support personnel who benefit from an effective mobile data strategy. Imagine a remote inventory manager with the ability, when alerted in real time to the fact that a key supplier just failed to make delivery on a needed component, to trigger alternative sourcing strategies that would keep the production lines moving and avoid disruption of product delivery. Or the executive, when presented with real-time analysis of poor operating performance in a region, can reach across time zones from a mobile device to make an emergency conference call to implement corrective action. In fact, 63% of executives in a recent SAS/Economist survey said having the ability to rapidly process information is having a significant impact on their strategic decision making ability.

The key to all of the above scenarios is, again, having the right data at hand, delivered on the right device (which is most often defined as the device of choice of the mobile user) without forcing the user to do their own data mining. Accomplishing this requires a mobile strategy that:

Accounts for an increasing volume of data input sources

Provides the flexibility to alter application functionality based on user input

Strikes the appropriate balance between user access and maintaining data security at the enterprise level

Maintains a rapid mobile application development process

As you consider how the convergence of Big Data and mobile will impact you in the months ahead, here are five things that might help keep you on the right path as you develop and implement a longer-term strategy:

Analyze the data streams - look at all potential sources of data to separate what data is being captured, what is stored and what is being constantly updated.

Identify all internal business information needs - start by taking a classic BI approach of identifying what each business organization needs to improve decision-making and competitive advantage.

Map the connection points to find data synergies - you most likely will be able to solve your organization's Big Data challenge in a single strategy but identifying overlapping needs can help establish priorities.

Make this a long-term strategy with regular evaluation milestones - this will allow you to focus on what can be delivered in the near term without sacrificing longer term objectives and should improve the chances of your plan paying off in real value.

Don't overlook the chance to leverage mobile workers as collection points. While getting the right information to remote workers should be a priority, the best strategies deploy a two-way approach that enables easy real-time updating of critical information from mobile users.

Big Data, like mobility, is not a trend or a passing fad; it's a fundamental element of 21st century business operations. Effectively organizing and delivering the results of that analysis to today's mobile management teams must have a place on every mobile and IT executive's priority list.

George Mashini is CEO of Catavolt. He has helped establish Catavolt as a market innovator in the areas of hybrid cloud computing and mobile business applications by creating software that leverages existing systems along with new technology. He is focused on applying new techniques and technologies in innovative ways that bring business value to Catavolt’s customers.

In his previous leadership role at Paragon Software, George experienced all phases of enterprise software growth as part of the core management team. In 2005, he was part of a core team at Paragon that led the company to a successful exit for its investors via a sale to Infor Global Solutions. He has been in the enterprise software field for 13 years with experience in software development, project management, professional services organizations, product management and executive management.

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

"We're a cybersecurity firm that specializes in engineering security solutions both at the software and hardware level. Security cannot be an after-the-fact afterthought, which is what it's become," stated Richard Blech, Chief Executive Officer at Secure Channels, in this SYS-CON.tv interview at @ThingsExpo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.

Consumers increasingly expect their electronic "things" to be connected to smart phones, tablets and the Internet. When that thing happens to be a medical device, the risks and benefits of connectivity must be carefully weighed. Once the decision is made that connecting the device is beneficial, medical device manufacturers must design their products to maintain patient safety and prevent compromised personal health information in the face of cybersecurity threats.
In his session at @ThingsExpo...

In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, discussed how to use Kubernetes to set up a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mortar data centers as well as cloud providers like Digital Ocean, Amazon Web Services, and Rackspace. H...

SYS-CON Events announced today that Grape Up will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct. 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Grape Up is a software company specializing in cloud native application development and professional services related to Cloud Foundry PaaS. With five expert teams that operate in various sectors of the market across the U.S. and Europe, Grape Up works with a variety of customers from emergi...

You know you need the cloud, but you’re hesitant to simply dump everything at Amazon since you know that not all workloads are suitable for cloud. You know that you want the kind of ease of use and scalability that you get with public cloud, but your applications are architected in a way that makes the public cloud a non-starter. You’re looking at private cloud solutions based on hyperconverged infrastructure, but you’re concerned with the limits inherent in those technologies.

SYS-CON Events announced today that Massive Networks will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Massive Networks mission is simple. To help your business operate seamlessly with fast, reliable, and secure internet and network solutions. Improve your customer's experience with outstanding connections to your cloud.

DevOps is under attack because developers don’t want to mess with infrastructure. They will happily own their code into production, but want to use platforms instead of raw automation. That’s changing the landscape that we understand as DevOps with both architecture concepts (CloudNative) and process redefinition (SRE).
Rob Hirschfeld’s recent work in Kubernetes operations has led to the conclusion that containers and related platforms have changed the way we should be thinking about DevOps and...

SYS-CON Events announced today that SkyScale will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. SkyScale is a world-class provider of cloud-based, ultra-fast multi-GPU hardware platforms for lease to customers desiring the fastest performance available as a service anywhere in the world. SkyScale builds, configures, and manages dedicated systems strategically located in maximum-security...

Detecting internal user threats in the Big Data eco-system is challenging and cumbersome. Many organizations monitor internal usage of the Big Data eco-system using a set of alerts. This is not a scalable process given the increase in the number of alerts with the accelerating growth in data volume and user base. Organizations are increasingly leveraging machine learning to monitor only those data elements that are sensitive and critical, autonomously establish monitoring policies, and to detect...

The question before companies today is not whether to become intelligent, it’s a question of how and how fast. The key is to adopt and deploy an intelligent application strategy while simultaneously preparing to scale that intelligence.
In her session at 21st Cloud Expo, Sangeeta Chakraborty, Chief Customer Officer at Ayasdi, will provide a tactical framework to become a truly intelligent enterprise, including how to identify the right applications for AI, how to build a Center of Excellence to...

Everything run by electricity will eventually be connected to the Internet. Get ahead of the Internet of Things revolution and join Akvelon expert and IoT industry leader, Sergey Grebnov, in his session at @ThingsExpo, for an educational dive into the world of managing your home, workplace and all the devices they contain with the power of machine-based AI and intelligent Bot services for a completely streamlined experience.

Because IoT devices are deployed in mission-critical environments more than ever before, it’s increasingly imperative they be truly smart. IoT sensors simply stockpiling data isn’t useful. IoT must be artificially and naturally intelligent in order to provide more value
In his session at @ThingsExpo, John Crupi, Vice President and Engineering System Architect at Greenwave Systems, will discuss how IoT artificial intelligence (AI) can be carried out via edge analytics and machine learning techn...

FinTechs use the cloud to operate at the speed and scale of digital financial activity, but are often hindered by the complexity of managing security and compliance in the cloud. In his session at 20th Cloud Expo, Sesh Murthy, co-founder and CTO of Cloud Raxak, showed how proactive and automated cloud security enables FinTechs to leverage the cloud to achieve their business goals. Through business-driven cloud security, FinTechs can speed time-to-market, diminish risk and costs, maintain continu...

With tough new regulations coming to Europe on data privacy in May 2018, Calligo will explain why in reality the effect is global and transforms how you consider critical data. EU GDPR fundamentally rewrites the rules for cloud, Big Data and IoT. In his session at 21st Cloud Expo, Adam Ryan, Vice President and General Manager EMEA at Calligo, will examine the regulations and provide insight on how it affects technology, challenges the established rules and will usher in new levels of diligence a...

Existing Big Data solutions are mainly focused on the discovery and analysis of data. The solutions are scalable and highly available but tedious when swapping in and swapping out occurs in disarray and thrashing takes place. The resolution for thrashing through machine learning algorithms and support nomenclature is through simple techniques. Organizations that have been collecting large customer data are increasingly seeing the need to use the data for swapping in and out and thrashing occurs ...

I recently had the opportunity to give a 10-minute keynote at DataWorks Summit 2017. I know what most of you are thinking: Schmarzo can barely introduce himself in 10 minutes! What sort of keynote could he give in just 10 minutes? And to be honest, I too struggled with what to say.
But after some brainstorming with my marketing experts (Jeff Abbott, Erin Banks, and Chris Hill), we came up with ...

In his session at 20th Cloud Expo, Mike Johnston, an infrastructure engineer at Supergiant.io, discussed how to use Kubernetes to set up a SaaS infrastructure for your business. Mike Johnston is an infrastructure engineer at Supergiant.io with over 12 years of experience designing, deploying, and maintaining server and workstation infrastructure at all scales. He has experience with brick and mort...

In preparation for General Data Protection Regulation (GDPR) compliance, a global 100 financial services organization embarked on a journey to assess its core information processing environments with the objective of identifying opportunities to strengthen its data privacy protection programs. This article focuses on the technology challenges, approach, and lessons learned for the centralized test...

Because IoT devices are deployed in mission-critical environments more than ever before, it’s increasingly imperative they be truly smart. IoT sensors simply stockpiling data isn’t useful. IoT must be artificially and naturally intelligent in order to provide more value
In his session at @ThingsExpo, John Crupi, Vice President and Engineering System Architect at Greenwave Systems, will discuss h...

Everything run by electricity will eventually be connected to the Internet. Get ahead of the Internet of Things revolution and join Akvelon expert and IoT industry leader, Sergey Grebnov, in his session at @ThingsExpo, for an educational dive into the world of managing your home, workplace and all the devices they contain with the power of machine-based AI and intelligent Bot services for a comple...

FinTechs use the cloud to operate at the speed and scale of digital financial activity, but are often hindered by the complexity of managing security and compliance in the cloud. In his session at 20th Cloud Expo, Sesh Murthy, co-founder and CTO of Cloud Raxak, showed how proactive and automated cloud security enables FinTechs to leverage the cloud to achieve their business goals. Through business...

We all probably remember the movie (“Jurassic Park”), even if we don’t remember this exact scene: Dr. Malcolm, played by Jeff Goldblum is explaining Chaos Theory to Dr. Ellie Sattler, played by Laura Dern.
Dr. Malcolm is explaining how random, seemingly negligible events can disrupt even the most carefully laid out plans.
Dr. Ian Malcolm: [after the T-Rex failed to appear for the tour group]. “Y...

Existing Big Data solutions are mainly focused on the discovery and analysis of data. The solutions are scalable and highly available but tedious when swapping in and swapping out occurs in disarray and thrashing takes place. The resolution for thrashing through machine learning algorithms and support nomenclature is through simple techniques. Organizations that have been collecting large customer...

Docker is sweeping across startups and enterprises alike, changing the way we build and ship applications. It's the most prominent and widely known software container platform, and it's particularly useful for eliminating common challenges when collaborating on code (like the "it works on my machine" phenomenon that most devs know all too well). With Docker, you can run and manage apps side-by-sid...

When you focus on a journey from up-close, you look at your own technical and cultural history and how you changed it for the benefit of the customer. This was our starting point: too many integration issues, 13 SWP days and very long cycles. It was evident that in this fast-paced industry we could no longer afford this reality. We needed something that would take us beyond reducing the developmen...

Increasingly, in a world of compressed times and distances, humans will be the inventors, designers and managers of digital systems and processes, rather than the operators. Operations will be measured in milliseconds, an inhumane speed where only the machines can deliver.
Professor Paul Virilio, a philosopher of speed, urbanist and cultural theorist, wrote at length about the impact of speed on...

As many know, the first generation of Cloud Management Platform (CMP) solutions were designed for managing virtual infrastructure (IaaS) and traditional applications. But that’s no longer enough to satisfy evolving and complex business requirements. In his session at 21st Cloud Expo, Scott Davis, Embotics CTO, will explore how next-generation CMPs ensure organizations can manage cloud-native and m...

"Suddenly a lot of companies started focusing on producing services in the cloud. I like to call it Cloud Native - everything is built for the cloud. The main concept there is to enable developers to work fast," explained Ben Bernstein, CEO & Co-Founder of Twistlock, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.

These days, change is the only constant. In order to adapt and thrive in an ever-advancing and sometimes chaotic workforce, companies must leverage intelligent tools to streamline operations. While we're only at the dawn of machine intelligence, using a workflow manager will benefit your company in both the short and long term. Think: reduced errors, improved efficiency and more empowered employee...

In 2016, artificial intelligence (AI) reached its climax. Research and advisory firm Tractica predicted that the annual worldwide AI revenue will grow from $643.7 million in 2016 to $38.8 billion by 2025. The revenue for enterprise AI applications will increase from $358 million in 2016 to $31.2 billion by 2025, representing a compound annual growth rate (CAGR) of 64.3%. Thus, IT and business deci...

Imagine a world where your Continuous Integration / Continuous Deployment environment is 100% automated, including the passing of credentials. Andrey started by stating what we all know - security needs to be at the forefront. Additionally, a message we heard over and over at the All Day DevOps conference is to automate where you can automate. That combination can be daunting and seemingly impossi...

In our first installment of this blog series, we went over the different types of applications migrated to the cloud and the benefits IT organizations hope to achieve by moving applications to the cloud.
Unfortunately, IT can’t just press a button or even whip up a few lines of code to move applications to the cloud. Like any strategic move by IT, a cloud migration requires advanced planning.

The renowned military strategist John Boyd taught that people and institutions collect favorite philosophies, strategies, theories and ideologies over a period of time, and then try to align the future to fit them. The problem with this is the future is rarely like the past, and trying to fit new data into old paradigms often forces us to perform irrational mental gymnastics, which leaves us fart...

Digital technology innovations and advancements, and our adoption of them, have changed us. We are different consumers, employers and employees. Our expectations have increased. We have become mobile, impatient and demanding. We are global. We demand immediate, accurate and real-time responses. We use our technology not just for reading historic events and news, but also for predicting our f...

Docker is on a roll. In the last few years, this container management service has become immensely popular in development, especially given the great fit with agile-based projects and continuous delivery. In this article, I want to take a brief look at how you can use Docker to accelerate and streamline the software development lifecycle (SDLC) process.

Cloud computing budgets worldwide are reaching into the hundreds of billions of dollars, and no organization can survive long without some sort of cloud migration strategy. Each month brings new announcements, use cases, and success stories.