The chronicles of a Bostonian tech geek navigating through life, technology, and general geekiness.

Menu

Tag Archives: amazon

Welcome to part two of my series on visualizing Office 365 security logs. In my last post I walked through the process of getting the sign-in and security logs and provided a link to some Lambda’s I put together to automate pulling them down from Microsoft Graph. Recall that the Lambda stores the files in raw format (with a small bit of transformation on the time stamps) into Amazon S3 (Simple Storage Service). For this demonstration I modified the parameters for the Lambda to download the 30 days of the sign-in logs and to store them in an S3 bucket I use for blog demos.

When the logs are pulled from Microsoft Graph they come down in JSON (JavaScript Object Notation) format. Love JSON or hate it is the common standard for exchanging information these days. The schema for the JSON representation of the sign-in logs is fairly complex and very nested because there is a ton of great information in there. Thankfully Microsoft has done a wonderful job of documenting the schema. Now that we have the logs and the schema we can start working with the data.

When I first started this effort I had put together a Python function which transformed the files into a CSV using pipe delimiters. As soon as I finished the function I wondered if there was an alternative way to handle it. In comes Amazon Athena to the rescue with its Openx-JsonSerDe library. After reading through a few blogs (great AWS blog here), StackOverflow posts, and the official AWS documentation I was ready to put something together myself. After some trial and error I put together a working DDL (Data Definition Language) statement for the data structure. I’ve made the DDLs available on Github.

Once I had the schema defined, I created the table in Athena. The official AWS documentation does a fine job explaining the few clicks that are provided to create a table, so I won’t re-create that here. The DDLs I’ve provided you above will make it a quick and painless process for you.

Let’s review what we’ve done so far. We’ve setup a reoccurring job that is pulling the sign-in and audit logs via the API and is dumping all that juicy data into cheap object storage which we can further enforce lifecycle policies for. We’ve then defined the schema for the data and have made it available via standard SQL queries. All without provisioning a server and for pennies on the dollar. Not to shabby!

At this point you can use your analytics tool of choice whether it be QuickSight, Tableau, PowerBi, or the many other tools that have flooded the market over the past few years. Since I don’t make any revenue from these blog posts, I like to go the cheap and easy route of using Amazon QuickSight.

After completing the initial setup of QuickSight I was ready to go. The next step was to create a new data set. For that I clicked the Manage Data button and selected New Data Set.

On the Create a Data Set screen I selected the Athena option and created a name for the data source.

From there I selected the database in Athena which for me was named azuread. The tables within the database are then populated and I chose the tbl_signin_demo which points to the test S3 bucket I mentioned previously.

Due to the complexity of the data structure I opted to use a custom SQL query. There is no reason why you couldn’t create the table I’m about to create in Athena and then connect to that table instead to make it more consumable for a wider array of users. It’s really up to you and I honestly don’t know what the appropriate “big data” way of doing it is. Either way, those of you with real SQL skills may want to look away from this query lest you experience a Raiders of The Lost Ark moment.

This query will de-nest the data and give you a detailed (possibly extremely large depending on how much data you are storing) parsed table. I was now ready to create some data visualizations.

The first visual I made was a geospatial visual using the location data included in the logs filtered to failed logins. Not surprisingly our friends in China have shown a real interest in my and my wife’s Office 365 accounts.

Next up I was interested in seeing if there were any patterns in the frequency of the failed logins. For that I created a simple line chart showing the number of failed logins per user account in my tenant. Interestingly enough the new year meant back to work for more than just you and me.

Like I mentioned earlier Microsoft provides a ton of great detail in the sign-in logs. Beyond just location, they also provide reasons for login failures. I next created a stacked bar chat to show the different reasons for failed logs by user. I found the blocked sign-ins by malicious IPs interesting. It’s nice to know that is being tracked and taken care of.

Failed logins are great, but the other thing I was interested in is successful logins and user behavior. For this I created a vertical stacked bar chart that displayed the successful logins by user by device operating system (yet more great data captured in the logs). You can tell from the bar on the right my wife is a fan of her Mac!

As I gather more data I plan on creating some more visuals, but this was great to start. The geo-spatial one is my favorite. If you have access to a larger data set with a diverse set of users your data should prove fascinating. Definitely share any graphs or interesting data points you end up putting together if you opt to do some of this analysis yourself. I’d love some new ideas!

That will wrap up this series. As you’ve seen the modern tool sets available to you now can do some amazing things for cheap without forcing you to maintain the infrastructure behind it. Vendors are also doing a wonderful job providing a metric ton of data in their logs. If you take the initiative to understand the product and the data, you can glean some powerful information that has both security and business value. Even better, you can create some simple visuals to communicate that data to a wide variety of audiences making it that much more valuable.

Today I’m going to look the capabilities within the AWS Managed AD to handle Active Directory schema modifications. If you’ve read my series on Microsoft’s Azure Active Directory Domain Services (AAD DS) you know that the service doesn’t support the schema modifications. This makes Amazon’s service the better offering in an environment where schema modifications to the standard Windows AD schema are a requirement. However, like many capabilities in a managed Windows Active Directory (Windows AD) service, limitations are introduced when compared to a customer-run Windows Active Directory infrastructure.

If you’ve administered an Active Directory environment in a complex enterprise (managing users, groups, and group policies doesn’t count) you’re familiar with the butterflies that accompany the mention of a schema change. Modifying the schema of Active Directory is similar to modifying the DNA of a living being. Sure, you might have wonderful intentions but you may just end up causing the zombie apocalypse. Modifications typically mean lots of application testing of the schema changes in a lower environment and a well documented and disaster recovery plan (you really don’t want to try to recover from a failed schema change or have to back one out).

Given the above, you can see the logic of why a service provider providing a managed Windows AD service wouldn’t want to allow schema changes. However, there very legitimate business justifications for expanding the schema (outside your standard AD/Exchange/Skype upgrades) such as applications that need to store additional data about a security principal or having a business process that would be better facilitated with some additional metadata attached to an employee’s AD user account. This is the market share Amazon is looking to capture.

So how does Amazon provide for this capability in a managed Windows AD forest? Amazon accomplishes it through a very intelligent method of performing such a critical activity. It’s accomplished by submitting an LDIF through the AWS Directory Service console. That’s right folks, you (and probably more so Amazon) doesn’t have to worry about you as the customer having to hold membership in a highly privileged group such as Schema Admins or absolutely butchering a schema change by modifying something you didn’t intend to modify.

In the first step we have to create a LDAP Data Interchange Format (LDIF) file. Think of the LDIF file as a set of instructions to the directory which in this could would be an add or modify to an object class or attribute. I’ll be using a sample LDIF file I grabbed from an Oracle knowledge base article. This schema file will add the attributes of unixUserName, unixGroupName, and unixNameIinfo to the default Active Directory schema.

To complete step one I dumped the contents below into an LDIF file and saved it as schemamod.ldif.

For the step two I logged into the AWS Management Console and navigated to the Directory Service Console. Here we can see my instance AWS Managed AD with the domain name of geekintheweeds.com.

I then clicked hyperlink on my Directory ID which takes me into the console for the geekintheweeds.com instance. Scrolling down shows a menu where a number of operations can be performed. For the purposes of this blog post, we’re going to focus on the Maintenance menu item. Here we the ability to leverage AWS Simple Notification Service (AWS SNS) to create notifications for directory changes such as health changes where a managed Domain Controller goes down. The second section is a pretty neat feature where we can snapshot the Windows AD environment to create a point-in-time copy of the directory we can restore. We’ll see this in action in a few minutes. Lastly, we have the schema extensions section.

Here I clicked the Upload and update schema button and entered selected the LDIF file and added a short description. I then clicked the Update Schema button.

If you know me you know I love to try to break stuff. If you look closely at the LDIF contents I pasted above you’ll notice I didn’t update the file with my domain name. Here the error in the LDIF has been detected and the schema modification was cancelled.

I went through made the necessary modifications to the file and tried again. The LDIF processes through and the console updates to show the schema change has been initialized.

Hitting refresh on the browser window updates the status to show Creating Snapshot. Yes folks Amazon has baked into the schema update process a snapshot of the directory provide a fallback mechanism in the event of your zombie apocalypse. The snapshot creation process will take a while.

While the snapshot process, let’s discuss what Amazon is doing behind the scenes to process the LDIF file. We first saw that it performs some light validation on the LDIF file, it then takes a snapshot of the directory, then applies to the changes to a single domain controller by selecting one as the schema master, removing it from directory replication, and applying the LDIF file using the our favorite old school tool LDIFDE.EXE. Lastly, the domain controller is added back into replication to replicate the changes to the other domain controller and complete the changes. If you’ve been administering Windows AD you’ll know this has appeared recommended best practices for schema updates over the years.

Once the process is complete the console updates to show completion of the schema installation and the creation of the snapshot.

Today I’m going to interrupt the series on AWS Managed Microsoft AD For the past few weeks, in between writing the entries for the recent deep dive series, I’ve been preparing for the AWS Cloud Practitioner exam. I thought it would be helpful to share my experience prepping for and passing the exam.

If you’re not familiar with the AWS Certificated Cloud Practitioner exam, it’s very much an introductory exam into the Amazon Web Services’ overarching architecture and products. Amazon’s intended audience for the certification are your C-levels, sales people, and technical people who are new to the AWS stack and potentially cloud in general. It’s very much an inch deep and mile wide. For those of you who have passed your CISSP, the experience studying for it similar (although greatly scaled down content-wise) in that you need to be able to navigate the shallow end of many pools.

Some of you may be asking yourselves why I invested my time in getting an introductory certificate rather than just going for the AWS Certified Solutions Architect – Associate. The reason is my personal belief that establishing a solid foundation in a technology or product is a must. I’ve encountered too many IT professionals with a decade more of experience and a hundred certificates to their name who can’t explain the basics of the OSI model or the difference in process between digitally signing something versus encrypting it. The sign of a stellar IT professional is one who can start at the business justification for an application and walk you right down through the stack to speak to the technology standards being leveraged within the application to deliver its value. This importance in foundation is one reason I recommend every new engineer start out by taking the CompTIA A+, Network+, and Security+ exams. You won’t find exams out there that better focus on foundational concepts than CompTIA exams.

The other selling point of this exam to me was the audience it’s intended for. Who wouldn’t want to know the contents and messaging in an exam intended for the C-level? Nothing is more effective influencing the C-level than speaking the language they’re familiar with and pushing the messaging you know they’ve been exposed to.

Let me step off this soapbox and get back to my experience with the exam. 🙂

As I mentioned above I spent about two weeks preparing for the exam. My experience with the AWS stack was pretty minimal prior to that restricted to experience for my prior blogs on Azure AD and AWS integration for SSO and provisioning and Microsoft Cloud App Security integration with AWS. As you can tell from the blog, I’ve done a fair amount of public cloud solutions over the past few years, just very minimally AWS. The experience in other public cloud solutions such as Microsoft Azure and Google Cloud Platform (GCP) proved hugely helpful because the core offerings are leveraging similar modern concepts (i.e. all selling computer, network, and storage). Additionally, the experiences I’ve had over my career with lots of different infrastructure gave me the core foundation I needed to get up and running. The biggest challenge for me was really learning the names of all the different offerings, their use cases, and their capabilities that set them apart from the other vendors.

For studying materials I followed most of the recommendations from Amazon which included reviews of a number of whitepapers. I had started the official Amazon Cloud Practitioner Essentials course (which is free by the way) but didn’t find the instructors engaging enough to keep my attention. I ended up purchasing a monthly subscription to courses offered by A Cloud Guru which were absolutely stellar and engaging at a very affordable monthly price (something like $29/month). In addition to the courses I read each of the recommended whitepapers (ended reading a bunch of others as well) a few times each taking notes of key concepts and terminology. While I was studying for this exam, I also was working on my AWS deep dive which helped to reinforce the concepts by actually building out the services for my own use.

I spent a lot of time diving into the rabbit whole of products I found really interesting (RedShift) as well as reading up on concepts I’m weaker on (big data analytics, modern nosql databases, etc). That rabbit hole consisted of reading blogs, Wikipedia, and standards to better understand the technical concepts. Anything I felt would be worthwhile I captured in my notes. Once I had a good 15-20 pages of notes (sorry all paper this time around), I grabbed the key concepts I wanted to focus on and created flash cards. I studying the deck of 200 or so flash cards each night as well as re-reading sections of the whitepapers I wanted to familiarize myself with.

For practice exams I used the practice questions Amazon provides as well as the quizzes from A Cloud Guru. I found the questions on the actual exam more challenging, but the practice question and quizzes were helpful to getting into the right mindset. The A Cloud Guru courses probably covered a good 85-90% of the material, but I wouldn’t recommend using it was a sole source of study, you need to read those whitepapers multiple times over. You also need to do some serious hands on because some of the questions do ask you very basic questions about how you do things in the AWS Management Console.

Overall it was a well done exam. I learned a bunch about the AWS product offerings, the capabilities that set AWS apart from the rest of the industry, and gained a ton of good insight into general cloud architecture and design from the whitepapers (which are really well done). I’d highly recommend the exam to anyone who has anything to do with the cloud, whether you’re using AWS or not. You’ll gain some great insight into cloud architecture best practices as well seeing modern technology concepts put in action.

I’ll be back with the next entry in my AWS Managed Microsoft AD series later this week. Have a great week and thanks for reading!

I’m back again with another entry in my deep dive into AWS Managed Microsoft Active Directory (AD). So far I’ve provided an overview of the service, covered how to configure the service, and analyzed the Active Directory default configuration such as the directory structure, security principals, password policies, and group policy setup by Amazon for new instances. In this post I’m going to look at the setup of LDAPS and how Amazon supports configuration of it in the delegated model they’ve setup for the service.

Those of you that have supported a Windows AD environment will be quite familiar with the wonders and sometimes pain of the Lightweight Directory Access Protocol (LDAP). Prior to the modern directories such as AWS Cloud Directory, Azure Active Directory the LDAP protocol served critical roles by providing both authentication and a method of which to work with data stored in directory data stores such as Windows AD. For better or worse the protocol is still relevant today when working with Windows AD for both of the above capabilities (less for authentication these days if you stay away from backwards-thinking vendors). LDAP servers listen on port 389 and 636 with 389 maintaining traffic in the clear (although there are exceptions where data is encrypted in transit such as Microsoft’s usage of Kerberos encryption or the use of StartTLS(credit to my friend Chris Jasset for catching my omission of StartTLS)) and 636 (LDAPS) providing encryption in transit via an SSL tunnel (hopefully not anymore) or a TLS tunnel.

Windows AD maintains that pattern and serves up the content of its data store over LDAP over ports 389 and 636 and additionally ports 3268 and 3269 for global catalog queries. In the assume breach days we’re living in, we as security professionals want to protect our data as it flows over the network which means we’ll more often than not (exceptions are again Kerberos encryption usage mentioned above) be using LDAPS over ports 636 or 3269. To provide that secure tunnel the domain controllers will need to be setup with a digital certificate issued by a trusted certificate authority (CA). Domain Controllers have unique requirements for the certificates they use. If you’re using Active Directory Certificate Services (AD CS) Microsoft takes care of providing the certificate template for you.

So how do you provision a certificate to a Domain Controller’s certificate store when you don’t have administrative privileges such as the case for a managed service like AWS Managed Active Directory? For Microsoft Azure Active Directory Domain Services (AAD DS) the public certificate and private key are uploaded via a web page in the Azure Portal which is a solid way of doing it. Amazon went in a different and instead takes advantage of certificate autoenrollment. If you’re not familiar with autoenrollment take a read through this blog. In short, it’s an automated way to distribute certificates and eliminate some of the overheard of manually going through the typical certificate lifecycle which may contain manual steps.

If we bounce back to the member server in my managed domain, open the Group Policy Management Console (GPMC), and navigate to the settings tab of the AWS Managed Active Directory Policy we see that autoenrollment has been enabled on the domain controllers. This setting explains why Amazon requires a member server joined to the managed domain be configured running AD CS. Once the AD CS instance is setup, the CA has been configured either to as a root or subordinate CA, and a proper template is enabled for autoenrollment, the domain controllers will request the appropriate certificate and will begin using it for LDAPS.

If you’ve ever worked with AD CS you may be asking yourself how you’ll be able to install AD CS in a domain where you aren’t a domain administrator when the Microsoft documentation specifically states you need to be a member of the Enterprise Admins and root domains Domain Admins group. Well folks that is where the AWS Delegated Enterprise Certificate Authority Administrators group comes into play. Amazon has modified the forest to delegate the appropriate permissions to install AD CS in a domain environment. If we navigate to the CN=Public Key Services, CN=Services, CN=Configuration using ADSIEdit and view the Security for the container we see this group has been granted full permissions over the node allowing the necessary objects to be populated underneath it.

I found it interesting that in the instructions provided by Amazon for enabling LDAPS the instructions state the Domain Controller certificate template needs to modified to remove the Client Authentication EKU. I’d be interested in knowing the reason for modifying the Domain Controller certificate. If I had to guess it’s to prevent the domain controller from using the certificate outside of LDAPS such as for mutual authentication during smart card logon. Notice that from this article domain controllers only require the Server Authentication EKU when a certificate is only used to secure LDAPS.

I’ve gone ahead and installed AD CS on SERVER01 as an Enterprise root CA and thanks to the delegation model, the CA is provisioned with all the necessary goodness in CN=Public Key Services. I also created the new certificate template following the instructions from Amazon. The last step is to configure the traffic flow such that the managed domain controllers can contact the CA to request a certificate. The Amazon instructions actually have a typo in them. On Step 4 it instructs you to modify the security group for your CA and to create a new inbound rule allowing all traffic from the source of your CA’s AWS Security group. The correct security group is actually the security group automatically configured by Amazon that is associated with the managed Active Directory instance.

At this point you’ll need to wait a few hours for the managed domain controllers to detect the new certificates available for autoenrollment. Mine actually only took about an hour to roll the certificates out.

To test the service I opened LDP.EXE and established a secure session over port 636 and all worked as expected.

Since I’m a bit OCD I also pulled the certificate using openssl to validate it’s been issued by my CA. As seen in the screenshot below the certificate was issued by the geekintheweeds-CA which is the CA I setup earlier.

Beyond the instructions Amazon provides, you’ll also want to give some thought as to how you’re going to handle revocation checks. Keep in mind that by default AD CS stores revocation information in AD. If you have applications configured to check for revocation remember to ensure those apps can communicate with the domain controllers over port 389 so design your security groups with this in mind.

Well folks that will wrap up this post. Now that LDAPS is configured, I’ll begin the tests looking at the protocols and ciphers supported when accessing LDAPS as well as examining the versions of NTLM supported and the encryption algorithms supported with Kerberos.

Today I’ll continue my deep dive into AWS Managed Microsoft AD. In the last blog post I provided an overview of the reasons an organization would want to explore a managed service for Windows Active Directory (Windows AD). In this post I’ll be providing an overview of my lab environment and demoing how to setup an instance of AWS Managed Microsoft AD and seamlessly joining a Windows EC2 instance.

Let’s dive right into it.

Let’s first cover what I’ll be using as a lab. Here I’ve setup a virtual private cloud (VPC) with default tenancy which is a requirement to use AWS Managed Microsoft AD. The VPC has four subnets configured within it named intranet1, intranet2, dmz1, and dmz2. The subnets intranet1/dmz1 and intranet2/dmz2 provide us with our minimum of two availability zones, which is another requirement of the service. I’ve created a route table that routes traffic destined for IP ranges outside the VPC to an Internet Gateway and applied that route table to both the intranet1 and intranet2 subnets. This will allow me to RDP to the EC2 instances I create. Later in the series I’ll configure VPN connectivity with my on-premises lab to demonstrate how the managed AD can be used on-prem. Below is a simple Visio diagraming the lab.

To create a new instance of AWS Managed Microsoft AD, I’ll be using the AWS Management Console. After successfully logging in, I navigate to the Services menu and select the Directory Service link under the Security, Identity & Compliance section as seen below.

The Directory Service page then loads which is a launching pad for configuration of the gamut of AWS Directory Services including AWS Cloud Directory, Simple AD, AD Connector, Amazon Cognito, and of course AWS Managed Microsoft AD. Any directory instance that you’ve created would appear in the listing to the right. To create a new instance I select the Set up Directory button.

The Set up a directory page loads and I’m presented with the options to create an instance of AWS Managed Microsoft AD, Simple AD, AD Connector, or an Amazon Cognito User Pool. Before I continue, I’ll provide the quick and dirty on the latter three options. Simple AD is actually Samba made to emulate some of the capabilities of Windows Active Directory. The AD Connector acts as a sort of proxy to interact with an existing Windows Active Directory. I plan on a future blog series on that one. Amazon Cognito is Amazon’s modern authentication solution (looks great for B2C) providing Open ID Connect, OAuth 2.0, and SAML services to applications. That one will warrant a future blog series as well. For this series we’ll be select the AWS Managed Microsoft AD option and clicking the Next button.

A new page loads where we configure the directory information. Here I’m given the option to choose between a standard or enterprise offering of the service. Beyond storage I’ve been unable to find or pull any specifications of the EC2 instances Amazon is managing in the background for the domain controllers. I have to imagine Enterprise means more than just 16GB of storage and would include additional memory and CPU. For the purposes of this series, I’ll be selecting Standard Edition.

Next I’ll provide the key configuration details for forest which includes the fully qualified domain name (FQDN) for the forest I want created as well as optionally specifying the NetBIOS name. The Admin password set here is used for the delegated administrator account Amazon creates for the customer. Make sure this password is securely stored, because if it’s lost Amazon has no way of recovering it.

After clicking the Next button I’m prompted to select the virtual private cloud (VPC) I want to service deployed to. The VPC used must include at least two subnets that are in different availability zones. I’ll be using the intranet1 and intranet2 subnets shown in my lab diagram earlier in the post.

The next page that loads provides the details of the instance that will be provisioned. Once I’m satisfied the configuration is correct I select the Create Directory button to spin up the service.

Amazon states it takes around 20 minutes or so to spin up the instance, but my experience was more like 30-45 minutes. The main Directories Services page displays the status of the directory as Creating. As part of this creation a new Security Group will be created which acts as a firewall for the managed domain controllers. Unlike some organization that try to put firewalls between domain-join clients and domain controllers, Amazon has included all the necessary flows and saves you a ton of troubleshooting with packet captures.

One of the neat features offered with this service is the ability to seamlessly domain-join Windows EC2 instances during creation. Before that feature can be leveraged an AWS Identity and Access Management (IAM) role needs to setup that has the AmazonEC2RoleforSSM attached to it. AWS IAM is by far my favorite feature of AWS. At a very high level, you can think of AWS IAM as being the identity service for the management of AWS resources. It’s insanely innovative and flexible in its ability to integrate with modern authentication solutions and in how granular you can be in defining rights and permissions to AWS resources. I could do multiple series just covering the basics (which I plan to do in the future) but to progress this entry let me briefly explain AWS IAM Roles. Think of an AWS IAM Role as a unique security principal similar to a user but without any credentials. The role is assigned a set of rights and permissions which AWS refers to as a policy. The role is then assumed by a human (such as federated user) or non-human (such as EC2 instance) granting the entity the rights and permissions defined in the policy attached to the role. In this scenario the EC2 instance I create will be assuming the AmazonEC2RoleforSSM. This role grants a number of rights and permissions within AWS’s Simple System Manager (SSM), which for your Microsoft-heavy users is a scaled down SCCM. It requires this role to orchestrate the domain-join upon instance creation.

To create the role I’ll open back up the Services menu and select IAM from the Security, Identity & Compliance menu.

The IAM dashboard will load which provides details as to the number of users, groups, policies, roles, and identity providers I’ve created. From the left-hand menu I’ll select the Roles link.

The Role page then loads and displays the Roles configured for my AWS account. Here I’ll select the Create Role button to start the role creation process.

The Create Role page loads and prompts me to select a trusted entity type. I’ll be using this role for EC2 instances so I’ll select the AWS service option and chose EC2 as the service that will use the role. Once both options are selects I select the Next: Permission button.

Next up we need to assign a policy to the role. We can either create a new policy or select an existing one. For seamless domain-join with AWS Managed Microsoft AD, EC2 instances must use the AmazonEC2forSSM policy. After selecting the policy I select the Next: Review button.

On the last page I’ll name the role, set a description, and select the Create role button. The role is then provisioned and available for use.

Navigating back to the Directory Services page, I can see that the geekintheweeds.com instance is up and running. This means we can now create some EC2 instances and seamlessly join them to the domain.

The EC2 instance creation is documented endless on the web, so I won’t waste time walking through it beyond showing the screenshot below which displays the options for seamless domain-join. The EC2 instance created will be named SERVER01.

After a few minutes the instance is ready to go. I start the Remote Desktop on my client machine and attempt a connection to the EC2 instance using the Admin user and credentials I set for the AD domain.

Low and behold I’m logged into the EC2 instance using my domain credentials!

As you can see setup of the service and EC2 instances is extremely simple and could made that much more simple if we tossed out the GUI and leveraged cloud formation templates to seamlessly spin up entire environments at a push of a button.

We covered a lot of content in this entry so I’ll close out here. In the next entry I’ll examine the directory structure Amazon creates including the security principals and key permissions.

Unless your organization popped up in the last year or two and went the whole serverless route you are still managing operating systems that require centralized authentication, authorization, and configuration management. You also more than likely have a ton of legacy/classic on-premises applications that require legacy protocols such as Kerberos and LDAP. Your organization is likely using Windows Active Directory (Windows AD) to provide these capabilities along with Windows AD’s basic domain name system (DNS) service and centralized identity data store.

It’s unrealistic to assume you’re going to shed all those legacy applications prior to beginning your journey into the public cloud. I mean heck, shedding the ownership of data centers alone can be a huge cost driver. Organizations are then faced with the challenge of how to do Windows AD in the public cloud. Is it best to extend an existing on-premises forest into the public cloud? What about creating a resource forest with a trust? Or maybe even a completely new forest with no trust? Each of these options have positives and negatives that need to be evaluated against organizational requirements across the business, technical, and legal arenas.

Whatever choice you make, it means additional infrastructure in the form of more domain controllers. Anyone who has managed Windows AD in an enterprise knows how much overhead managing domain controllers can introduce. Let me clarify that by managing Windows AD, it does not mean opening Active Directory Users and Computers (ADUC) and creating user accounts and groups. I’m talking about examining performance monitor AD counters and LDAP Debug logs to properly size domain controllers, configuring security controls to comply with PCI and HIPAA requirements or aligning with DISA STIGS, managing updates and patches, and troubleshooting the challenges those bring which requires extensive knowledge of how Active Directory works. In this day an age IT staff need to be less focused on overhead such as this and more focused on working closely with its business units to drive and execute upon business strategy. That folks is where managed services shine.

AWS offers an extensive catalog of managed services and Windows AD is no exception. Included within the AWS Directory Services offerings there is a powerful offering named Amazon Web Services Directory Service for Microsoft Active Directory, or more succinctly AWS Managed Microsoft AD. It provides all the wonderful capabilities of Windows AD without all of the operational overhead. An interesting fact is that the service has been around since December 2015 in comparison to Microsoft’s AAD DS which only went into public preview at in 3rd Q 2017. This head start has done AWS a lot of favors and in this engineer’s opinion, has established AWS Managed Microsoft AD as the superior managed Windows AD service over Microsoft’s AAD DS. We’ll see why as the series progresses.

Over the course of this series I’ll be performing a similar analysis as I did in my series on Microsoft AAD DS. I’ll also be examining the many additional capabilities AWS Managed Microsoft AD provides and demoing some of them in action. My goal is that by the end of this series you understand the technical limitations that come with the significant business benefits of leveraging a managed service.

It seems like it’s become a weekly occurrence to have sensitive data exposed due to poorly managed cloud services. Due to Amazon’s large market share with Amazon Web Services (AWS) many of these instances involve publicly-accessible Simple Storage Service (S3) buckets. In the last six months alone there were highly publicized incidents with FedEx and Verizon. While the cloud can be empowering, it can also be very dangerous when there is a lack of governance, visibility, and acceptance of the different security mindset cloud requires.

Organizations that have been in operation for many years have grown to be very reliant on the network boundary acting as the primary security boundary. As these organizations begin to move to a software defined data center model this traditional boundary quickly becomes less and less effective. Unfortunately for these organizations this, in combination with a lack of sufficient understanding of cloud, gives rise to mistakes like sensitive data being exposed.

One way in which an organization can protect itself is to leverage technologies such as cloud access security brokers (cloud security gateways if you’re Forrester reader) to help monitor and control data as it travels between on-premises and the cloud. If you’re unfamiliar with the concept of a CASB, I covered it in a previous entry and included a link to an article which provides a great overview.

Microsoft has its own CASB offering called Microsoft Cloud App Security (CAS). It’s offered as part of Microsoft’s Enterprise Mobility and Security (EMS) E5/A5 subscription. Over the past several months multiple connectors to third party software-as-a-service (SaaS) providers have been introduced, including one for AWS. The capabilities with AWS are limited at this point to pulling administrative logs and user information but it shows promise.

As per usual, Microsoft provides an integration guide which is detailed in button pushing, but not so much in concepts and technical details as to what is happening behind the scenes. Since the Azure AD and AWS blog series has attracted so many views, I thought it would be fun and informative to do an entry for how Cloud App Security can be used with AWS.

I’m not in the habit of re-creating documentation so I’ll be referencing the Microsoft integration guide throughout the post.

The first thing that needs doing is the creation of a security principal in AWS Identity and Access Management (AWS IAM) that will be used by your tenant’s instance of CAS to connect to resources in your AWS account. The first four steps are straightforward but step 5 could a bit of an explanation.

Here we’re creating a custom IAM policy for the security principal granting it a number of permissions within the AWS account. IAM policies are sets of permissions which are attached to a human or non-human identity or AWS resource and are evaluated when a call to the resource is made. In the traditional on-premises world, you can think of it as something somewhat similar to a set of NTFS file permissions. When the policy pictured above is created the security principal is granted a set of permissions across all instances of CloudTrail, CloudWatch, and IAM within the account.

If you’re unfamiliar with AWS services, CloudTrail is a service which audits the API calls made to AWS resources. Some of the information included in the events include the action taken, the resource the action was taken upon, the security principal that made the action, the date time, and source IP address of the security principal who performed the action. The CloudWatch service allows for monitoring of metrics and optionally triggering events based upon metrics reaching specific thresholds. The IAM service is AWS’s identity store for the cloud management layer.

Now that we have a basic understanding of the services, let’s look at the permissions Microsoft is requiring for CAS to do its thing. The CloudTrail permissions of DescribeTrails, LookupEvents, and GetTrailStatus allow CAS to query for all trails enabled on an AWS account (CloudTrail is enabled by default on all AWS resources), lookup events in a trail, and get information about the trail such as start and stop logging times. The CloudWatch permissions of Describe* and Get* are fancy ways of asking for READ permissions on CloudWatch resources. These permissions include describe-alarms-history, describe alarms, describe-alarms-for-metric, get-dashboard, and get-metric-statistics. The IAM permissions are similar to what’s being asked for in CloudWatch, basically asking for full read.

Step number 11 instructs us to create a new CloudTrail trail. AWS by default audits all events across all resources and stores them for 90 days. Trails enable you to direct events captured by CloudTrail to an S3 bucket for archiving, analysis, and responding to events.

The trail created is consumed by CAS to read the information captured via CloudTrail. The permissions requested above become a bit more clear now that we see CAS is requesting read access for all trails across an account for monitoring goodness. I’m unclear as to why CAS is asking for read for CloudWatch alarms unless it has some integration in that it monitors and reports on alarms configured for an AWS account. The IAM read permissions are required so it can pull user information it can use for the User Groups capability.

After the security principal is created and a sample trail is setup, it’s time to configure the connector for CAS. Steps 12 – 15 walk through the process. When it is complete AWS now shows as a connected app.

After a few hours data will start to trickle in from AWS. Navigating to the Users and Accounts section shows all of the accounts found in the IAM instance for my AWS account. Envision this as your one stop shop for identifying all of the user accounts across your many cloud services. A single pane of glass to identity across SaaS.

On the Activity Log I see all of the API activities captured by CloudTrail. If I wanted to capture more audit information, I can enable CloudTrail for the relevant resource and point it to the trail I configured for CAS. I haven’t tested what CAS does with multiple trails, but based upon the permissions we configured when we setup the security principal, it should technically be able to pull from any trail we create.

Since the CAS and AWS integration is limited to pulling logging information, lets walk through an example of how we could use the data. Take an example where an organization has a policy that the AWS root user should not be used for administrative activities due to the level of access the account gets by default. The organization creates AWS IAM users accounts for each of its administrators who administer the cloud management layer. In this scenario we can create a new policy in CAS to detect and alert on instances where the AWS root user is used.

First we navigate to the Policies page under the Control section of CAS.

On the Policies page we’re going to choose to create a new policy settings in the image below. We’ll designate this as a high severity privileged account alert. We’re interested in knowing anytime the account is used so we choose the Single Activity option.

We’ll pretend we were smart users of CAS and let it collect data for a few weeks to get a sampling of the types of events which are captured and to give us some data to analyze. We also went the extra mile and leveraged the ability of CAS to pull in user information from AWS IAM such that we can choose the appropriate users from the drop-down menus.

Since this is a demo and my AWS lab has a lot of activity by the root account we’re going to limit our alerts to the creation of new AWS IAM users. To do that we set our filter to look for an Activity type equal to Create user. Our main goal is to capture usage of the root account so we add another filter rule that searches for a User with the name equal to aws root user where it is the actor in an event.

Finally we configure the alert to send an email to the administrator when the event occurs. The governance capabilities don’t come into play in this use case.

Next we jump back to AWS and create a new AWS IAM user named testuser1. A few minutes after the user is created we see the event appearing in CloudTrail.

After a few minutes, CAS generates and alert and I receive an email seen in the image below. I’m given information as to the activity, the app, the date and time it was performed, and the client’s IP address.

If I bounce back to CAS I see one new Alert. Navigating to the alert I’m able to dismiss it, adjust the policy that generated it, or resolve it and add some notes to the resolution.

I also have the option to dig deeper to see some of the patterns of the user’s behavior or the pattern of the behaviors from a specific IP address as seen below.

All this information is great, but what can we do with it? In this example, it delivers visibility into the administrative activities occurring at the AWS cloud management layer by centralizing the data into a single repository which I can then send other data such as O365 activity, Box, SalesForces, etc. By centralizing the information I can begin doing some behavioral analytics to develop standard patterns of behavior of my user base. Understanding standard behavior patterns is key to being ahead of the bad guys whether they be insiders or outsiders. I can search for deviations from standard patterns to detect a threat before it becomes too widespread. I can also be proactive about putting alerts and enforcement (available for other app connectors in CAS but not AWS at this time) to stop the behavior before the threat is realized. If I supplemented this data with log information from my on-premises proxy via Cloud App Discovery, I get an even larger sampling improving the quality of the data as well as giving me insight into shadow IT. Pulling those “shadow” cloud solutions into the light allow me to ensure the usage of the services complies with organizational policies and opens up the opportunity of reducing costs by eliminating redundant services.

Microsoft categorizes the capabilities that help realize these benefits as the Discover and Investigate capabilities of CAS. The solution also offers a growing number of enforcement mechanisms (Microsoft categorized these enforcement mechanisms as Control) which add a whole other layer of power behind the solution. Due to the limited integration with AWS I can’t demo those capabilities with this post. I’ll cover those in a future post.

I hope this post helped you better understand the value that CASB/CSGs like Microsoft’s Cloud App Security can bring to the table. While the product is still very new and a bit sparse on support with 3rd party applications, the list is growing every day. I predict the capabilities provided by technology such as Microsoft’s Cloud App Security will be as standard to IT as a firewall in the years to come. If you’re already in Office 365 you should be ensuring you integrate these capabilities into your arsenal to understand the value they can bring to your business.

Post navigation

About Me

Hi there! My name is Matt Felton and I am a long time geek with a passion for technology. I have over 10 years experience in the industry that spans the technology stack. Over the past few years I’ve had the opportunity to dig deeper into security and identity which I’ve been more than happy to do.

I started Journey Of The Geek over 6 six years ago when I saw an opportunity to provide in-depth technical deep dives to peel back the onion on technologies and products. I enjoy sharing what I’ve learned and giving back to the industry. Plus there is no better way to learn a topic than to teach it.

I hope you enjoy and if you have questions feel free to reach out via the comments, LinkedIn, or Twitter.

DISCLAIMER

All views expressed on this site are my own and do not represent the opinions of any entity whatsoever of which I have been, am now, or will be affiliated.