Big Data

The well-known three Vs of Big Data - Volume, Variety, and Velocity – are increasingly placing pressure on organizations that need to manage this data as well as extract value from this data deluge for Predictive Analytics and Decision-Making. Big Data technologies, services, and tools such as Hadoop, MapReduce, Hive and NoSQL/NewSQL databases and Data Integration techniques, In-Memory approaches, and Cloud technologies have emerged to help meet the challenges posed by the flood of Web, Social Media, Internet of Things (IoT) and machine-to-machine (M2M) data flowing into organizations.

Big Data Articles

COLLABORATE 16 - IOUG Forum is your chance to present among the best of the best and provide the Oracle community with your expert insight and experience. By speaking at COLLABORATE, you're putting the success and clout of your business (and yourself) front and center!

Everyone has had their database, server or even application fail. What matters is how you handle it, correct it and move on. The Q3 issue of SELECT Journal focuses on bouncing back and continuing onward.

IOUG members, further engage with your community and help make COLLABORATE 16 - IOUG Forum great! IOUG is currently looking for session reviewers to support the development of our educational program at COLLABORATE 16.

Basho Technologies today announced Basho Riak TS, a distributed NoSQL database that is designed to enable analysis of massive amounts of sequenced, unstructured data generated from the Internet of Things (IoT) and other time series data sources.

Couchbase is today announcing the general availability of Couchbase Server 4.0, a major new release of its NoSQL database management system. The company is calling Couchbase Server 4.0 a "transformational release," that dramatically increases the types of applications and use cases that Couchbase can now support. The announcement is being made at Couchbase Live New York, where customers such as Marriott, GE, Gannett, Cox Automotive, DIRECTV, Nielsen and others are speaking about their use of Couchbase in a variety of deployments.

At Strata + Hadoop World 2015, Attunity announced the release of Attunity Replicate Express, a downloadable edition of its data replication and loading software. The solution, which answers a growing demand for more accessible real-time big data analytics, is freely available to download online. The new solution supports ingesting data to and from Oracle, SQL Server, and Hadoop Data Lakes for test and development environments.

Built on Hadoop, Kyvos gives business users and analysts the ability to query billions of rows of data within seconds. Kyvos' technology allows users to pre-process data and build cubes on Hadoop for faster performance and instant responses. With this partnership, Kyvos can connect Tableau users to their Hadoop data within minutes, the companies say. "It's a benefit to Tableau because it opens up the data that's available to the business user through Tableau and improves the response time," said Ajay Anand, vice president of product management and marketing at Kyvos."They've been very supportive with what we are trying to do."

One of the noticeable changes this year at Strata + Hadoop World 2015 was the rise of Apache Spark, an engine for large scale data processing. In recent months, many companies have extended support to Spark, which can be complementary to Hadoop, but can also be deployed without it.

At Strata + Hadoop World 2015, SAP showcased its portfolio of big data solutions, including the HANA platform that offers real-time integration of big data and information held in Hadoop with business processes and operational systems, Lumira and SAP BI tools that enable data discovery on Hadoop along with data wrangling capabilities, SAP Data Services, and the newest SAP product for the Hadoop world, HANA Vora, which takes advantage of an in-memory query engine for Apache Spark and Hadoop to speed queries. SAP HANA Vora can be used as a stand-alone, or in concert with SAP HANA platform to extend enterprise-grade analytics to Hadoop clusters and provide enriched, interactive analytics on Hadoop and HANA data.

Pepperdata, a provider of solutions that optimize cluster performance in Hadoop, showed off a new feature of its platform that will help measure and allocate the costs of increasing workloads across distributed systems at Strata + Hadoop World in NYC. With this new chargeback feature IT teams can clearly see how much capacity each user or workload requires and allocate costs back to departments that share a centralized, multi-tenant Hadoop deployment.

At Strata + Hadoop World, TIBCO announced the availability of the Spotfire Cloud's data discovery and advanced analytics connector to Apache Spark SQL, along with a commercial integration with SparkR. The Spark SQL direct connector is now available in TIBCO Spotfire Cloud, and will also be incorporated in the next TIBCO Spotfire on-premises release.

Objectivity, which recently introduced ThingSpan, a purpose-built information fusion platform intended to simplify and accelerate companies' ability to deploy and derive value from industrial Internet of Things (IoT) applications, has announced plans to support Intel's TAP (Trusted Analytics Platform) at Strata + Hadoop World, in NYC. ThingSpan is aimed at helping companies "that are drowning in data but thirsty for answers in time" said Jay Jarrell, CEO and president of Objectivity, during an interview at the conference.

At Strata + Hadoop World in New York City, Talend, a provider of data integration software for the cloud and big data, is announcing a new version of its platform, now offering support for Apache Spark and Spark Streaming. Talend 6 will leverage over 100 Spark components to deliver rapid data processing speed and enable any company to convert streaming big data or IoT sensor information into immediate actionable insights.

DataTorrent is teaming up with two big companies that will allow it to provide access to better security and make adoption of Hadoop easier. DataTorrent is partnering with Cisco to allow integration between its DataTorrent RTS platform and Cisco's Application Centric Infrastructure (ACI) through the Application Policy Infrastructure Controller (APIC), offering a unified management architecture for enterprises to manage their big data applications along with network and security. DataTorrent is also integrating its platform with Microsoft Azure HDInsight via the Microsoft Azure Marketplace.

Pivotal has confirmed it will continue its commitment to advancing open source by contributing its technology to Apache Software Foundation (ASF). Pivotal's contribution of the HAWQ advanced SQL on Hadoop analytics and MADlib machine learning technologies will cement Hadoop's place as the as the cornerstone of advanced data science, business intelligence, and data warehousing, according to Pivotal.

Paxata, provider of an adaptive data preparation platform, is partnering with Cisco, creating a jointly developed solution dubbed Cisco Data Preparation (CDP). "We are delighted to partner with a world-class organization like Cisco as we continue to fulfill our vision to bringing Adaptive Data Preparation to every analyst in the enterprise," said Prakash Nanduri, Co-founder and CEO of Paxata.

Redis Labs is extending its cloud strategy to on-prem, private and hybrid, allowing enterprises to install an enterprise-grade cluster that acts as a container for managing and running multiple Redis databases.

Pentaho is updating its platform to help users blend data more efficiently and manage the analytic data pipeline. "We've learned so much over the last couple of years from our big data customers and customers that have scaled and seen the value of big data and their environments," said Donna Prlich, vice president of products solutions and marketing at Pentaho. "We're really looking at our product line and saying, ‘Where do we take this and where does it need to go?' In 6.0 it's really all about putting big data to work."

Arcadia Data, a provider of a unified visual analytics and business intelligence (BI) platform for big data, is releasing Arcadia Enterprise, a solution that will run natively in Hadoop. The company says the platform, dubbed Arcadia Enterprise, bypasses the restrictions of legacy BI and visualization tools by allowing users to work directly with their data on Hadoop. "We give the analyst the ability to do free-form exploration of the highest granularity of data in the Hadoop system," said Priyank Patel, co-founder and chief product officer at Arcadia.

The Hortonworks DataFlow (HDF) support subscription is now available. HDF, powered by Apache NiFi, a top-level open source project, is intended to help organizations take advantage of data related to the Internet of Anything (IoAT) and helps make it easier to automate and secure data flows and collect, conduct and curate real-time business insights and actions derived from any data, from anything, anywhere. "By flowing that data into HDP, our customers are able to rapidly bring these new data elements under management in a completely secure and purely open way," said Tim Hall, vice president of product management at Hortonworks.

At Strata + Hadoop World in New York City, Cloudera announced a public beta of a new storage to enable faster analytics in Hadoop. Kudu, a new columnar store for Hadoop, enables the combination of fast analytics on fast data. Complementing the existing Hadoop storage options, HDFS and Apache HBase, Kudu is a native Hadoop storage engine that supports both low-latency random access and high-throughput analytics, dramatically simplifying Hadoop architectures for increasingly common real-time use cases.

Cloudera has launched a public beta release of RecordService, a new high-performance security layer for Apache Hadoop that centrally enforces role-based access control policies across the platform. Complementing Apache Sentry (incubating), which provides unified policy definition, RecordService delivers complete row- and column-based security, and dynamic data masking, for every Hadoop access engine. The announcement was made at Strata + Hadoop World in New York City.

Mobile is game changer. There is no doubt about that. The role that mobile plays is growing at such a fast rate that it simply cannot be ignored: a mobile strategy is now a necessity in the business world and the mainframe is here to support it.

SHARE is excited to introduce DevOps in the Enterprise as a brand-new educational track for the SHARE in San Antonio event this winter, and we want YOU to help kick it off. Share your expertise and best practices with attendees who want to help build and enhance DevOps within their organizations.

IBM expanded its array of APIs, technologies, and tools for developers who are creating products, services and applications embedded with Watson. Over the past 2 years, the Watson platform has evolved from one API and a limited set of application-specific deep Q&A capabilities to more than 25 APIs powered by over 50 technologies.

TeamQuest, a provider of IT capacity planning and management solutions, has introduced a new release of its flagship TeamQuest Performance Software. This latest release is intended to help enable organizations to assess the health and potential areas of risk in their IT infrastructure, by applying automated and accurate predictive algorithms using data sources within their IT enterprise.

In advance of an event at the American Museum of Natural History in New York City, at which MongoDB showcased leading use cases for the company's NoSQL database technology, and introduced a new MongoDB University app for iOS, Kelly Stirman, vice president of strategy and product marketing at MongoDB, discussed how companies are putting MongoDB to use now, and upcoming features in MongoDB 3.2 which will be rolled out before the end of the year.

Cambridge Semantics, a provider of data solutions driven by semantic web technology, has formed an alliance with MarkLogic, which provides enterprise NoSQL database technology. According to the vendors, the partnership will will help organizations to rapidly store, access, visualize and act upon diverse data to create scalable, semantic-driven data management and investigative analytics applications at a fraction of the time and cost of traditional approaches.

MarkLogic, which bills itself as the only enterprise NoSQL database provider, completed a $102 million financing round earlier this year that it will use to accelerate the pace of growth in the $36 billion operational database market. Recently, Big Data Quarterly spoke with Joe Pasqua, executive vice president of products at MarkLogic, about the changing database management market, and what MarkLogic is doing to meet emerging enterprise customer requirements.

Traditional data warehousing models and open source alternatives such as Apache Hadoop and Storm have been touted as solutions to a variety of "big data" challenges. However, utilities have found that these approaches cannot handle the scale and complexity of data generated in industrial environments. Additionally, they fail to provide the real-time analysis and situational awareness that utilities need to improve decision making or address critical events in real-time, such as optimizing crews during outages and severe weather events.

StreamSets Inc., a company that aims to speed access to enterprise big data, has closed a $12.5 million round of Series A funding. The single biggest barrier to a successful enterprise analytics platform is the effective and efficient ingest of data, the company says.

Steve Capelli, former president of Sybase Inc., which was acquired by SAP, is joining the team at Bradmark Technologies, a provider of monitoring and performance management software. Capelli will now serve as Bradmark's Chairman of the Board, bringing a wealth of knowledge to the company.

Customers moving to the SAP HANA platform always have a question about who their SAP HANA Database Administrator is and what they can expect from her/him. Handling a complex and large SAP HANA database can be very challenging if not configured and managed correctly and effectively.

The study, entitled, "The Internet of Things Has the Potential to Connect and Transform Businesses," evaluated the capacity for enterprises empowered with IoT solutions to achieve efficiency in their processes, deliver increased value to customers, and create new business models. It found that even early adopters have only scratched the surface of IoT benefits in current use cases.

SAP plans to unleash a powerful portfolio of SAP hybris tools that are envisioned to enable in-the-moment customer profiling, digital commerce and community development, empowering an organization's front office to stay connected with the frequently shifting needs of its customers and prospects and enabling companies to go beyond customer relationship management (CRM) into a new era of digital connectedness, customer service and support.

SAP is helping enterprises safeguard against unnecessary risks when teaming up with other companies by releasing the SAP Business Partner Screening application. SAP Business Partner Screening will enable companies to simplify business partner screening processes, minimize efforts and costs, and optimize alert processing to help reduce exposure to commercial, compliance and reputational risk.

SAP is releasing a set of enhancements for one of its platforms that it says will change the way enterprises interact with customers. A portfolio of SAP hybris tools is being introduced to enable in-the-moment customer profiling, digital commerce and community development, allowing an organization's front office to stay connected with the frequently shifting needs of its customers and prospects.

Tableau has introduced an upgraded version of its comprehensive data visualization platform, which includes increased enterprise security features, such as single Sign On for SAP HANA and Mutual SSL Authentication, new connectors, and visual analytics upgrades.

SAP is overhauling its PartnerEdge program, providing a new simplified way to partner with SAP for bigger opportunities. The extensive update streamlines the framework of partner levels to meet the needs of evolving partner business models, offers a unified reward structure that gives partners credit for their total SAP business, and introduces eContracts that will reduce paperwork and speed processing.

Virtustream, which was recently acquired by EMC, is strengthening its partnership with SAP, becoming one of a select few global partners providing cloud infrastructure services for SAP business-critical applications in SAP HANA Enterprise Cloud. The collaboration will allow SAP to leverage Virtustream's xStream Cloud Management Platform and its Micro-VM technology to deliver an IaaS with Service Level Agreements covering application performance.