securekomodo:~#

hacker of things

Powershell Incident Response

During the past few months I have been rather quiet with my online presence mainly due to my professional life becoming more and more demanding, thus not leaving time to blog about my experiences. Even though I have not been sharing any experiences with the online community, I have been working diligently on leveraging PowerShell Incident Response tool. I want to share a scenario in which I used PowerShell scripts to gather info, determine a scope, and begin remediation for a particular security related incident with a client out of the country. To protect the privacy of the client, I have omitted any identifying details and summarized the incident into the below scenario:

The Scenario: Malicious traffic and has been identified on multiple servers and attackers have gained a foothold in the DMZ and other critical zones on the network. Security staff have proven that the attacker(s) were able to pivot to multiple servers and ex-filtrate information to their command and control servers.

Given: Security professionals have identified the network flow and hardened firewall rules, added logging devices, and modified the network design so no further data can leave the network. New logs have shown that some malicious traffic was attempting to exit from an unmanaged IP Address in the DMZ using a local account with administrator access.

The Task:

Find all unauthorized “rogue” devices in the DMZ and any other internet facing subnets.

Identify all servers (Internal and Perimeter) and retrieve all privileged administrator accounts. Both local and domain.

Set complex password requirements for all authorized privileged accounts, and remove all unauthorized members of the local administrator group.

Task 1 Solution: Find Rogue Devices in DMZ

So the first task of finding rogue devices was my favorite of the three. I was finally able to put a relatively simple PowerShell tool I had developed for personal use, into production! The task was to search a particular subnet for any active devices that are not authorized to be on that network. I knew I would be able to retrieve that information so long as I am running the script from a server that is located within that network. I started in the DMZ and I was given access to a virtual server which was located within that DMZ and I made sure that PowerShell version 4.0 was installed (though I could have been alright with PowerShell version 3.0 too). Aside from verifying the PowerShell version, there was nothing else I needed but the code to execute. Let me summarize how this script works.

The script will first determine its hosts IP address and validate network connectivity.

Then, it will begin send ICMP requests to every IP address in the subnet.

Any responding devices are then shown to the console as being active.

Then, since the active IPs are now stored in the virtual servers ARP cache, I can extract that information to determine the MAC Address corresponding to the IP to help identify what type of device was found.

Then, the script will attempt to get the DNS name from the IP address by querying DNS.

After, the results are compared with the approved “known” server list, and any unauthorized devices are then exported to a spreadsheet in CSV format for review.

Of course I could not publish the real output so I have fuzzed the below output information with fake IPs, Names, and Random MAC addresses.

This script takes only seconds to run is is essential in finding any rogue devices on your network. Since the scan is asynchronous, it can scan thousands of IPs in seconds! (Though it is important to note that only the local subnet is stored in the servers ARP cache, if you are scanning devices past a router, the mac address information will not be found in ARP cache). After analyzing the results of the rogue device scan, there were multiple unauthorized devices that were identified that assisted the investigation of the security incident. I wrote about this a bit in some previous posts where I published my asynchronous network and host discovery scanner called Get-SecNetMap. I highly recommend checking it out!

**Task 2 Solution: Find all Privileged Accounts

**

The misuse of administrative accounts can lead to an attackers ability to traverse from server to server. In addition to keeping a close eye on local administrators, it is imperative that each account has a unique password on every server to limit lateral movement. It seems elementary but this is something I see time and time again in many organizations… So I wrote a script that uses ADSI to connect to a remote server and retrieve all members of the Local Administrator group. The results could include local accounts, Active Directory Users, Active Directory Groups, Computer Objects, and SIDs. There are many ways to retrieve members of the Local Administrator group using PowerShell and I would always recommend using Remoting as the first (and best) method. However in this situation, the enterprise did not have Remoting configured so I had to think of alternate methods to retrieve this information.

Essentially I took the list of known servers and devices found during the rogue device scan and used it as my target list for the Local Admin audit. Unfortunately multithreading is not built into this script so I had to loop through the server list using a foreach loop. Again, I fuzzed the actual results to show you what the output will look like.

Of course the audit is going to pull all members of the local admin group, and some could be legitimate. In order to determine which results were unauthorized, I consulted with the on site system administrators. They said that they maintained a list of all known accounts with admin rights which they swore to be accurate (this made me laugh…). Once the audit completed scanning I had a certain sense of enjoyment when I presented the retrieved evidence to them in a conference room. They were good sports about it though, we all knew there was a common goal in mind and getting an accurate baseline is the best course of action in gaining control of your critical systems.

Taking the list and throwing in some Excel pivot tables really helped display the results so the decision makers can see the risk involved. I was able to show each admin account by count of servers they were identified on with fancy charts and colors! We can look at the stats above and clearly see that majority of privileged accounts were local accounts, followed by Active Directory members that were direct members of Local Administrator group. That makes the accounts more difficult to manage since each time an account is removed/added, the server has to be interacted with. The best practice would be to add a Active Directory Group as a member of the servers Local Admin Group, so that members can be dynamically added/removed without need to interact with the server at all.

This data was also able to give some statistics showing how many servers a particular account was found on. The bad news is that one of the top accounts identified had the same password used to login to all the servers it was a member on. So if the account was compromised, an attacker would have inherent access to many other servers! During this time, management was able to coordinate with the local account owners to get the passwords changed immediately, or have the account removed. For the Active Directory accounts identified, I was given the task of forcing a complex randomized password for every account.

It goes without saying that one of the best ways to protect an account from unauthorized access is using a strong password. As computer performance increases, and software becomes more sophisticated, successful password attacks are becoming easier to achieve. Management wanted to make sure every account password was changed. First they coordinated with the System Administrators again to make sure that complexity requirements were configured in Active Directory. Then they sent an email to all employees mandating a password change within a certain time frame. If the account was not changed during that time, I would set a complex random password and disable the account for good measure. Again, PowerShell saves the day!

By querying Active Directory for all accounts that have not had their password changed in the time frame specified by management, I was able to get an accurate list of accounts in scope. For “in scope” accounts, they would get a forced random password and be disabled through a few built in cmdlets of PowerShell. Get-Random was really useful during this process. The snippet of code below shows where I take the entire character array of the ASCII table and create a truly random password of length 16

[code language=”PowerShell” collapse=”false”]

#Set Password Complexity & Length

$Password=””

$ascii=$NULL;For ($a=48;$a –le 122;$a++) {$ascii+=,[char][byte]$a }

$pwlength = 16

For ($loop=1; $loop –le $pwlength; $loop++) {

$Password+=($ascii | GET-RANDOM)

}

[/code]

Running it only took a minute or two and the verbose output and logging really assisted with providing confidence in its execution. I provided an example of how the output would look like.

Takeaways

Team coordination: I found that the biggest time delay in this security incident was team coordination. Prior to my arrival, there were no business units that were aware that I was going to be auditing their service and privileged accounts. So when the time came to either remove or reset the password, no one knew what systems were going to break, or if the account was even needed.

Account Identification: Adding on to the problem of team coordination, it seemed nobody knew what any of the accounts were, or what they were used for. I was able to handle this later by coordinating a communication to ALL IT departments and have a working “command center” where all of us would go through the findings one by one and document document, and document. It is a tedious process but it is critical in base-lining your privileged accounts. Go through each account and find answers for these questions…

Who owns this account?

What is the account used for?

Why does it need administrator access?

Admin Group Management: Rather than having local accounts or Active Directory accounts be direct members of a server. Try to use groups and keep the members of your Local Admin group clean. The less you have to ‘touch’ a server for admin management the better. By keeping your administrators in an AD group and having only that group and admin on the server, you can dynamically add/remove as needed without needing to ‘touch’ the server at all. This makes your life so much better 🙂

Inherited Permissions: There were many AD Groups that were found during the audit that brought attention to knowing who IS an admin and who is actually APPROVED to be an admin. Scanning the group members recursively was crucial in finding out unauthorized privileged accounts. One of the servers had a group nested so far in (almost 5 levels deep) that I found particularly interesting. “Domain Users” was discovered as a nested member of an admin group on that server!!!! ಠ_ಠ Meaning… ANY user had full admin rights on that server, simply through misconfiguration in inherited permissions. That made my recursive scan completely time out due to the overwhelming results. But we were able to clean all those up just the same so now the client his some real solid security controls in place going forward.

Before everyone asks. All the scripts used during this process I am going to make publicly available for use. I did not include them in this initial post since the scripts contain some client specific data that I still need to omit before publishing. Be sure to follow PoshSec on twitter @PoshSec and on Github where I will commit the scripts to. https://github.com/PoshSec

I would love to know your thoughts. Do you think there were better ways to accomplish the tasks? let me know!