AWS Lake Formation makes it easy for customers to build secure data lakes in days instead of months

SEATTLE--(BUSINESS WIRE)--Nov. 28, 2018-- Today at AWS re:Invent, Amazon Web Services Inc. (AWS), an Amazon.com company (NASDAQ: AMZN), announced three new services that provide automation and prescriptive guidance to help customers build more quickly on AWS. AWS Control Tower gives customers an automated “landing zone” that makes it easy to set up their multi-account environment and continuously govern their AWS workloads with rules for security, operations, and compliance. AWS Security Hub is a central place to manage security and compliance across an AWS environment so that customers can quickly see their AWS security and compliance state in one comprehensive view. AWS Lake Formation makes it much easier for customers to build a secure data lake by simplifying and automating many of the complex manual steps usually required, including collecting, cleaning, and cataloging data.

As the cloud has increasingly become the default choice for organizations of all sizes, across every industry, it has attracted two distinct types of builders. The first group are the tinkerers who want to use the full range of AWS services, pick out various building blocks, and hammer them together to create a specific architecture. The second type of builders are drawn to AWS for many of the same reasons – the breadth and depth of functionality in AWS – but they are willing to trade some of the service granularity to start at a higher abstraction layer where much of the construction has already been done for them and they can get started faster. AWS has always catered to customers who want to build from the ground up, while also adding services over time like AWS Elastic Beanstalk, Amazon Lightsail, and Amazon SageMaker that are designed to serve the needs of this second type of builder. AWS Control Tower, AWS Security Hub, and AWS Lake Formation extend this approach to a wider array of workloads and scenarios, giving customers abstracted services to help with provisioning and governance, monitoring security and compliance, and building and managing data lakes. These services automate multi-step processes to get customers up and running more quickly, while also centralizing management of crucial elements of their AWS environment, helping maintain consistency and providing more complete visibility.

“We know our customers love the breadth of capability available in AWS, but they also tell us they want us to package our services in ways that make it easier for them to build an architecture quickly,” said Charlie Bell, Senior Vice President of AWS. “One of the central benefits of the cloud is that it removes the vast operational complexities of managing physical infrastructure. AWS’s new services abstract away additional complexity, speeding and simplifying the process of deploying and managing cloud workloads, so customers can build faster, operate more securely, and maintain consistent governance in a way that gives them more time to innovate.”

AWS Control Tower makes it easy toset up and govern a secure, compliant multi-account environment (available in preview today)

Enterprises migrating to AWS often need to manage a large number of accounts, distributed teams, and applications. AWS’s provisioning and management services give customers granular control over their environments, but many organizations also want prescriptive guidance and help setting up a secure, multi-account environment. They want to make sure both that they're using all the right tools and that they understand how those tools can be used to create and enforce policies for their teams to deploy workloads in a secure and compliant way. And they want all of this without sacrificing the speed and agility of AWS. AWS Control Tower addresses these challenges by providing central cloud teams with a single, automated “landing zone” where their teams can provision accounts and workloads according to industry and AWS best practices. The automated landing zone employs best-practices blueprints, such as configuring a multi-account structure using AWS Organizations, managing user identities and federated access with AWS Single Sign-on or Microsoft Active Directory, configuring an account factory through AWS Service Catalog, centralizing a log archive using AWS CloudTrail and AWS Config, and more. AWS Control Tower offers pre-packaged governance rules for security, operations, and compliance, which customers can apply enterprise-wide or to groups of accounts to enforce policies or detect violations. All of this can be easily managed and monitored through AWS Control Tower’s dashboard, providing customers with centralized visibility into a customer’s AWS environment including information such as accounts provisioned, preventive and detective guardrails enabled, and the compliance status of accounts as it relates to the guardrails. To learn more about AWS Control Tower, visit: https://aws.amazon.com/controltower.

Change Healthcare is a leading independent healthcare technology company that provides data and analytics-driven solutions that reach approximately 2,100 government and commercial payer connections, 5,500 hospitals, 900,000 physicians, and 33,000 pharmacies. “As a healthcare technology provider, we’re constantly faced with the challenge of maintaining consistent controls across our workloads, while at the same time giving our teams the flexibility they need to iterate quickly and continuously innovate,” said Aaron Symanski, Chief Technology Officer, Change Healthcare. “AWS Control Tower will address this challenge by enabling distributed teams to adopt AWS faster in a self-service model, while giving centralized teams the peace of mind that non-compliant applications will be prevented from being deployed. Automatic enforcement of policies will not only save considerable time and effort, it will also help ensure that compliance standards are consistently upheld across an organization.”

Enterprises today use a broad array of AWS and third-party security tools that are continuously generating findings viewable in multiple consoles and dashboards. To understand their overall security and compliance state, customers must either manually pivot between these tools or invest in developing complex systems to aggregate and analyze the findings. As a result, many security teams find it challenging to separate the signal from the noise, prioritize the findings that matter most, and ensure that critical information isn’t missed. With AWS Security Hub,customers can quickly see their entire AWS security and compliance state in one place. AWS Security Hubcollects and aggregates findings from the security services it discovers in a customer’s environment, such as intrusion detection findings from Amazon GuardDuty, vulnerability scan results from Amazon Inspector, sensitive data identifications from Amazon Macie, and findings generated by a range of security tools from AWS Partner Network (APN) partners. Findings are correlated into integrated dashboards that visualize and summarize a customer’s current security and compliance status, and highlight trends and Amazon Elastic Compute Cloud (Amazon EC2) instances that are generating an increasing number of findings. Customers can run automated, continuous configuration and compliance checks based on industry standards and best practices, such as the Center for Internet Security (CIS) AWS Foundations Benchmark, identifying specific accounts and resources that require attention.

Asurion develops and deploys innovative technology solutions for wireless carriers, pay-TV, and retail partners to help keep millions of people around the world connected. “As a global tech solutions services company, we face a diverse set of security, privacy and compliance requirements,” said Scott Mackelprang, Chief Security Officer at Asurion. “AWS Security Hub helps us to quickly scan a dashboard and review our security and compliance posture in AWS. Using Security Hub, we can now quickly focus on problem areas that will improve our compliance state and more easily satisfy our audit requirements.”

AWS Lake Formation makes it easy to set up a secure data lake (preview available soon)

A lot of customers are talking about data lakes because they understand that it is useful to remove siloes and have all of their data reside in a central place where they can apply analytics and machine learning. Amazon Simple Storage Service (Amazon S3) is the natural place to have a data lake because of all of the security, access control, operational performance, functionality, and management controls it offers, as well as the proximity to the analytics and machine learning services available in AWS. It turns out, however, that building and managing a data lake is a complex and time-consuming process that requires customers to load data from diverse sources, set up buckets and partitions, clean and prepare data, configure and enforce security policies across a range of services, granularly configure access control settings, and more. AWS Lake Formation removes much of this heavy lifting, allowing customers to build a data lake in a matter of days. Using AWS Lake Formation, customers simply define the data sources they wish to ingest and then select from a prescribed list of data access and security policies to apply, removing the need to define and enforce policies across their analytics applications. AWS Lake Formation then collects the data and moves it into a new Amazon S3 data lake, extracting technical metadata in the process to catalog and organize the data for easier discovery. The service automatically optimizes the partitioning of data to improve performance and reduce costs, transforms data into formats like Apache Parquet and ORC for faster analytics, and also uses machine learning to deduplicate matching records to increase data quality.

Customers can use AWS Lake Formation to centrally define and manage security, governance, and auditing policies for their data lake. Policies will then be consistently implemented, reducing the duplicative work of manually defining and enforcing them across a customer’s security, storage, analytics, and machine learning services. AWS Lake Formation also provides a centralized, customizable catalog which describes available data sets and their appropriate business use. This reduces the time that analysts and data scientists must spend hunting down the right data set for their needs, and enables users to conduct combined, self-service analytics jobs with their choice of analytics and machine learning services, including Amazon EMR, Amazon Redshift, Amazon Athena, Amazon Sagemaker, and Amazon QuickSight. To learn more about AWS Lake Formation, visit: https://aws.amazon.com/lake-formation.

Fender Digital is a part of Fender, the iconic guitar brand, that makes applications, websites, platforms, and tools to complement the guitars, amps, and audio gear that Fender makes. “We are generating tons of user and usage data from our digital applications and devices. We are planning to build a data lake on AWS to operate alongside our Amazon Redshift-based data warehouse,” said Joshua Couch, VP Engineering at Fender Digital. “I can’t wait for my team to get our hands on AWS Lake Formation. Lake Formation will make it easy for us to load, transform, and catalog our data and make it securely available within our organization, across a wide portfolio of AWS services. With an enterprise-ready option like AWS Lake Formation, we will be able to spend more time deriving value from our data rather than doing the heavy lifting involved in manually setting up and managing our data lake.”

About Amazon Web Services

For over 12 years, Amazon Web Services has been the world’s most comprehensive and broadly adopted cloud platform. AWS offers over 125 fully featured services for compute, storage, databases, networking, analytics, machine learning and artificial intelligence (AI), Internet of Things (IoT), mobile, security, hybrid, virtual and augmented reality (VR and AR), media, and application development, deployment, and management from 57 Availability Zones (AZs) within 19 geographic regions around the world, spanning the US, Australia, Brazil, Canada, China, France, Germany, India, Ireland, Japan, Korea, Singapore, and the UK. AWS services are trusted by millions of active customers around the world—including the fastest-growing startups, largest enterprises, and leading government agencies—to power their infrastructure, make them more agile, and lower costs. To learn more about AWS, visit https://aws.amazon.com.

About Amazon

Amazon is guided by four principles: customer obsession rather than competitor focus, passion for invention, commitment to operational excellence, and long-term thinking. Customer reviews, 1-Click shopping, personalized recommendations, Prime, Fulfillment by Amazon, AWS, Kindle Direct Publishing, Kindle, Fire tablets, Fire TV, Amazon Echo, and Alexa are some of the products and services pioneered by Amazon. For more information, visit amazon.com/about and follow @AmazonNews.