Transcription

3 Agenda A Short HPC Market Update Big Data Challenges and Short Comings The High End of Big Data Examples of Very Large Big Data Examples of How Big Data is Redefining High Performance Computing

4 HPC Market Update

5 What Is HPC? IDC uses these terms to cover all technical servers used by scientists, engineers, financial analysts and others: HPC HPTC Technical Servers Highly computational servers HPC covers all servers that are used for computational or data intensive tasks From a $5,000 deskside server up to over $550 million dollar supercomputer

6 Top Trends in HPC 2013 declined overall by $800 million For a total of $10.3 billion Mainly due to a few very large systems sales in 2012 that weren t repeated in 2013 We expect growth in 2015 to 2018 Software issues continue to grow The worldwide Petascale Race is at full speed GPUs and accelerators are hot new technologies Big data combined with HPC is creating new solutions in new areas

15 HPDA Market Drivers More input data (ingestion) More powerful scientific instruments/sensor networks More transactions/higher scrutiny (fraud, terrorism) More output data for integration/analysis More powerful computers More realism More iterations in available time The need to pose more intelligent questions Smarter mathematical models and algorithms Real time, near-real time requirements Catch fraud before it hits credit cards Catch terrorists before they strike Diagnose patients before they leave the office Provide insurance quotes before callers leave the phone

16 Top Drivers For Implementing Big Data

17 Organizational Challenges With Big Data: Government Compared To All Others

24 CERN LHC: the world s leading accelerator -- Multiple Nobel Prizes for particle physics work Innovation driven by the need to distribute massive data sets and the accompanying applications Altas, one of CERN s two detectors, generates 1PB of data per second when running! (Not all of this is distributed). Private cloud distribution to scientists in 20 EU member states plus observer states (single largest user is the U.S.) Today: only % of the data is used

33 The Results $710 million saved in fraud that they wouldn t have been able to detect before (in the first year)

34 There Are New Technologies That Will Likely Cause A Mass Explosion In Data Requiring HPDA Solutions

35 GEICO: Real-Time Insurance Quotes Problem: Need accurate automated phone quotes in 100ms. They couldn t do these calculations nearly fast enough on the fly. Solution: Each weekend, use a new HPC cluster to pre-calculate quotes for every American adult and household (60 hour run time)

36 Something To Think About -- GEICO: Changing The Way One Approaches Solving a Problem Instead of processing each event one-at-a-time, process it for everyone on a regular basis It can be dramatically cheaper, faster and offers additional ways to be more accurate But most of all it can create new and more powerful capabilities Examples: For home loan applications calculate for every adult in the US and every home in the US For health insurance fraud track every procedure done on every US person by every doctor and find patterns

37 Something To Think About -- GEICO: Changing The Way One Approaches Solving a Problem Future Examples (continued): If you add-in large scale data collection via sensors like GPS, drones and RFID tags: New car insurance rules The insurance company doesn t have to pay if you break the law -- like speeding and having an accident You could track every car at all times then charge $2 to see where the in-laws are in traffic if they are late for a wedding Google maps could show in real-time where every letter and package is located But crooks could also use it in many ways e.g. watching ATM machines, looking for when guards are on break,

42 Schrödinger: Cloud-based Lead Discovery for Drug Design NOVARTIS/SCHROEDINGER: Pharmaceutical company Novartis increased resolution of drug discovery algorithm 10x and wanted to use it to test 21 million small molecules as drug candidates Novartis used the Schroedinger drug discovery app in AWS public cloud, with the help of Cycle Computing Initial run used 51,000 AWS cores and took $14,000 and <4 hours and its getting cheaper Later run used 156,000 AWS cores with comparable costs and time

43 Schrödinger: Cloud-based Lead Discovery for Drug Design

44 Global Financial Services: Company X One of the most respected firms in the global financial services industry updates detailed information daily on several million companies around the world. Clients use the firm's credit ratings and other company information in making lending decisions and for other planning, marketing, and business decision making. The firm uses statistical models to develop a company's scores and ratings, and for years, the ratings have been prepared and analyzed locally in near real time by the firm's personnel around the world. This practice is a major competitive advantage but resulted in the creation of hundreds of distinct databases and more than a dozen scoring environments. Several years ago, the company established a goal of centralizing these resources and chose SAS as the centralization mechanism, including SAS Grid Manager as part of the software stack.

45 Global Financial Services: Company X The centralized IT infrastructure created using SAS preserves the advantages of the company's locally created ratings and reports. The new infrastructure provides an effective environment for analytics development and accommodates multiple testing, debt, and production environments in a single stack. It is flexible enough to allow dynamic prioritization among these environments, according to a company executive. With help from SAS Grid Manager, the company can maximize the use of its computing resources. The software automatically assigns jobs to server nodes with available capacity, instead of having users wait in queue for time on fully utilized nodes. The company executive estimates that it might cost 30% more to purchase servers with enough capacity to handle these peak workloads on their own.

48 Outcomes-Based Medical Diagnosis and Treatment Planning Enter the patient s history and symptomology. While the patient is still in the office, sift through millions of archived patient records for relevant outcomes. Provider considers the efficacies of various treatments for similar patients (but is not bound by the findings). Ergo, this functions as a powerful decision-support tool. Benefits: better outcomes + rein in costly outlier practices.

49

50 Digital Television Services A global leader with 30 million subscribers Goal: maximize revenue & customer satisfaction during high-growth period Result: HPC has added 7.5 million in annual revenue while increasing satisfaction

54 Summary: HPDA Market Opportunity HPDA: simulation + newer highperformance analytics IDC predicts fast growth from a small starting point HPC and high-end commercial analytics are converging Algorithmic complexity is the common denominator Economically important use cases are emerging No single HPC solution is best for all problems

The Fusion of Supercomputing and Big Data Peter Ungaro President & CEO The Supercomputing Company Supercomputing Big Data Because some great things never change One other thing that hasn t changed. Cray

BUILDING A SCALABLE BIG DATA INFRASTRUCTURE FOR DYNAMIC WORKFLOWS ESSENTIALS Executive Summary Big Data is placing new demands on IT infrastructures. The challenge is how to meet growing performance demands

Building a Scalable Big Data Infrastructure for Dynamic Workflows INTRODUCTION Organizations of all types and sizes are looking to big data to help them make faster, more intelligent decisions. Many efforts

Big Data Putting data to productive use Fast Forward What is big data, and why should you care? Get familiar with big data terminology, technologies, and techniques. Getting started with big data to realize

FUJITSU Integrated System PRIMEFLEX for HPC Integrated HPC Cluster Solutions Optimized for specific Applications 0 2015 FUJITSU Why High Performance Computing? Wouldn t it be great if you could quickly

DDN Solution Brief Accelerate > ISR With DDN Big Data Storage The Way to Capture and Analyze the Growing Amount of Data Created by New Technologies 2012 DataDirect Networks. All Rights Reserved. The Big

Surfing the Data Tsunami: A New Paradigm for Big Data Processing and Analytics Dr. Liangxiu Han Future Networks and Distributed Systems Group (FUNDS) School of Computing, Mathematics and Digital Technology,

White Paper Make the Most of Big Data to Drive Innovation Through Reseach Bob Burwell, NetApp November 2012 WP-7172 Abstract Monumental data growth is a fact of life in research universities. The ability

THE REAL-TIME OPERATIONAL VALUE OF BIG DATA MATT DAVIES SPLUNK @MATTDAVIES_UK THANK YOU FOR HAVING ME 2 WHY I LOVE SWEDEN #1 IT WAS HOME I LIVED IN STOCKHOLM FOR 3 MONTHS WHY I LOVE SWEDEN #2 FROZEN HAIR

White Paper BIG DATA-AS-A-SERVICE What Big Data is about What service providers can do with Big Data What EMC can do to help EMC Solutions Group Abstract This white paper looks at what service providers

Personalized Medicine and IT Data-driven Medicine in the Age of Genomics www.intel.com/healthcare/bigdata Ketan Paranjape General Manager, Life Sciences Intel Corp. @Portlandketan 1 The Central Dogma of

Modern IT Operations Management Why a New Approach is Required, and How Boundary Delivers TABLE OF CONTENTS EXECUTIVE SUMMARY 3 INTRODUCTION: CHANGING NATURE OF IT 3 WHY TRADITIONAL APPROACHES ARE FAILING

Seagate HPC /Big Data Business Tech Talk December 2014 Safe Harbor Statement This document contains forward-looking statements within the meaning of Section 27A of the Securities Act of 1933, and Section

5 Keys to Unlocking the Big Data Analytics Puzzle Anurag Tandon Director, Product Marketing March 26, 2014 1 A Little About Us A global footprint. A proven innovator. A leader in enterprise analytics for

Enterprise Data Integration Access, Integrate, and Deliver Data Efficiently Throughout the Enterprise brochure How Can Your IT Organization Deliver a Return on Data? The High Price of Data Fragmentation

Government Technology Trends to Watch in 2014: Big Data OVERVIEW The federal government manages a wide variety of civilian, defense and intelligence programs and services, which both produce and require

Dr. John E. Kelly III Senior Vice President, Director of Research Differentiating IBM: Research IBM Research Priorities Impact on IBM and the Marketplace Globalization and Leverage Balanced Research Agenda

White Paper: SAS and Apache Hadoop For Government Unlocking Higher Value From Business Analytics to Further the Mission Inside: Using SAS and Hadoop Together Design Considerations for Your SAS and Hadoop

The Future of Data Management with Hadoop and the Enterprise Data Hub Amr Awadallah (@awadallah) Cofounder and CTO Cloudera Snapshot Founded 2008, by former employees of Employees Today ~ 800 World Class

Business Intelligence Trends For 2013 10 Trends The last few years the change in Business Intelligence seems to accelerate under the pressure of increased business demand and technology innovations. Here

Big Data George O. Strawn NITRD Caveat auditor The opinions expressed in this talk are those of the speaker, not the U.S. government Outline What is Big Data? NITRD's Big Data Research Initiative Big Data

Create and Drive Big Data Success Don t Get Left Behind The performance boost from MapR not only means we have lower hardware requirements, but also enables us to deliver faster analytics for our users.

FALL 2012 VOL.54 NO.1 Thomas H. Davenport, Paul Barth and Randy Bean How Big Data is Different Brought to you by Please note that gray areas reflect artwork that has been intentionally removed. The substantive

White Paper Version 1.2 May 2015 RAID Incorporated Introduction The abundance of Big Data, structured, partially-structured and unstructured massive datasets, which are too large to be processed effectively

on AWS Services Overview Bernie Nallamotu Principle Solutions Architect \ So what is it? When your data sets become so large that you have to start innovating around how to collect, store, organize, analyze

VOLUME 34 BEST PRACTICES IN BUSINESS INTELLIGENCE AND DATA WAREHOUSING FROM LEADING SOLUTION PROVIDERS AND EXPERTS PDF PREVIEW IN EMERGING TECHNOLOGIES POWERFUL CASE STUDIES AND LESSONS LEARNED FOCUSING

ONE platform for ALL YOUR DATA Radim Petrzela February 26 th, 2013 POWER OF HITACHI Founded in 1910 US$118B FY11 900 subsidiaries 324,000 employees More than 760 PhDs INFORMATION and TELECOMMU- NICATIONS

NITRD and Big Data George O. Strawn NITRD Caveat auditor The opinions expressed in this talk are those of the speaker, not the U.S. government Outline What is Big Data? Who is NITRD? NITRD's Big Data Research

5 Key Trends in Connected Health One of the most exciting market opportunities in healthcare today is the near limitless set of innovative solutions that can be created through the integration of the Internet,

Unlock the business value of enterprise data with in-database analytics Achieve better business results through faster, more accurate decisions White Paper Table of Contents Executive summary...1 How can

Industrial Internet @GE Dr. Stefan Bungart The vision is clear The real opportunity for change surpassing the magnitude of the consumer Internet is the Industrial Internet, an open, global network that

White Paper Elastic Private Clouds Agile, Efficient and Under Your Control 1 Introduction Most businesses want to spend less time and money building and managing IT infrastructure to focus resources on

Big Data Executive Full Questionnaire Big Date Executive Full Questionnaire Appendix B Questionnaire Welcome The survey has been designed to provide a benchmark for enterprises seeking to understand the

SAP Thought Leadership Paper Helping the U.S. Government Serve the American People Better Helping the U.S. Government Serve the American People Better innovating with less: the cornerstone of the Digital

Big Use Cases To Start Today Paul Scholey Sales Director, EMEA 1 Exabytes of We all know the amount of data in the world is growing exponentially 40000 30000 YOU ARE HERE 20000 FROM 2010 TO 2015 77% of

The Internet of Things The Power of Actionable Insight An introduction to the Internet of Things Chris Vetor Business Unit Executive, WW Programs cvetor@us.ibm.com More and more of the world s activity