Problem

You have deleted a computer object with BitLocker Recovery information on it

You then restored that computer account from recycle bin.

No BitLocker Recovery information exists on the recovered computer object !!!!! What the heck ?!

Reason

Going back to basics…. do you know where the BitLocker information is stored for a computer object ? They are stored as a child object below the computer object itself.

Now, when the computer get deleted from Active Directory and moved to the AD recycle bin, the links between the child objects and the parent are broken. In the AD recycle bin you will see both computer objects and child objects randomly stored there. If you put your hand in this recycle bin and pull a computer object, you will not see any of its child objects attached to it any more. This is exactly what happened when you restored the AD computer from recycle bin, you will get the computer object without its child objects.

Solution

Lucky for all of us, each child objects of type (BitLocker Recovery Information) will have an attribute called (lastKnownParent). So theoretically if you go to the recycle bin and asked ” i have a parent called ComputerX, so which of you guys are the sons of this computer (which of you has the lastknownParent = ComputerX).

Download Script

Go to your Domain Controller or any machine with ActiveDirectory PowerShell Module, open PowerShell using a domain administrator account (only domain admin can restore from AD recycle bin), run the script from there. Make sure AD PowerShell module exist on that machine.

Do not forget that you may need to run Set-ExecutionPolicy Unrestricted on PowerShell to allow script execution.

I have no single credit writing this script. You can find the script here written by (Norman Bauer). I copied the script also to my repository so you can download it directly .

How does the script work

It will ask you about the name of computer to restore

Validation check : checking if that computer exists in AD first

If not, then the computer may be in recycle bin, search there and report if it is not there also ($deleted = Get-ADObject -IncludeDeletedObjects -Filter {sAMAccountName -eq $computername -and Deleted -eq $True}

If the computer in recycle bin, we will going to restore it ($deleted | Restore-ADObject)

Like this:

Mission :

I do not want to use third party software to backup my Active Directory (Domain Controller’s system state) because usually those third party requires high privileges and rights on domain controllers.

Instead, i will have a protected file share server, i will use a script on the domain controller to backup its system state to that protected file share. Then i will use my favorite third party solution to backup the protected file share server.

Prepare the File Server

Now as the file share will host your AD backup, it is important to protect and restrict access on that file server. I would recommend to install a VM with C drive for the O.S and D drive to host the system state backup.

Also, make sure that the administrators group on that file share is restricted to only domain admins. Do not install any other server roles on the server and do not host any other shares on it.

Now on the D drive, create a hidden share with Full permissions given to (Domain Controllers) group on both sharing and NTFS permissions. (Domain Controllers) is a built in security group that exist on your AD by default.

Prepare the Domain Controller

Nothing to prepare really here. You need to schedule the below script on one of your domain controllers. That’s it.

Script Breakdown

The script should be scheduled to run on any domain controller and it should run using the built in (System) scrutiny context. This will give it the right to take backups to your AD without any additional rights 🙂

The script starts by importing the (Server Manager) module using the Import-Module ServerManager

Then we will get the current date [string]$date = get-date -f ‘yyyy-MM-dd’.

Following that we will define the folder on the remote file share $TargetUNC = “\\FileServer\ADBackup$\AD-$date”.

This assumes that the remote file share name is (FileServer) and the hidden share we created is called ADBackups$

Notice that we are assuming that backups will be taken in a folder structure where the name of the folder contains the date on which the backup is taken.

So we will check first to see if a folder is already there that contains today’s date, and if it exists, we will delete it. This means that we will not maintain two backups taken in the same date. This is only my own way. You can do yours.

Because the script will try to create folders on the remote share, (Domain Controllers) group will need access on that remote file share

Finally, we will start taking backup using WEBADMIN command. This command requires that in order to do backups to remote file share, a user name and password should be supplied. So create a username (i.e ServiceADBackup) and give it share and NTFS permissions to write to the remote file share.

Schedule Script

In order to schedule the script on your DC, open Task scheduler , create basic task with your own schedule preference, and when you reach the Action window, make sure to put (C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe) on the Program/script field, and the full path of your script in the (Add arguments (optional)) filed.

When you are done, open the task again, and change the scrutiny context that the script is using, and replace it with (System) and check the (Run with highest privileges)

Like this:

The Requirement

“Send me email if my AD was not backed up recently”

I was given a task for making sure that AD backups (system state) is working fine and to get alerts if it fails.

If you go to the internet, you will find many solutions for taking backup for the domain controller’s system state. If you are using Windows Built in Backup Software, then you can write a script to search for specific event IDs under the Backup Event Log category. This was my initial thought. You will find many scripts out there who look at that place. But those scripts will only work if you are using Windows Backup Built in software.

I wanted a more direct, more reliable, more abstracted way to check AD backups. I want to go to AD and ask it : When the last time you get backed up, and then get alerts if this exceeded my backup cycle Window.

How to solve it ?

Lucky me, i found this nice article talking about PowerShell and AD stuff. It is a smart way to get the backup status for each AD partition, and the script is written in a way that works in all environments and no hard coding is used.

They use those line of code to get the last backup stamp for each AD partition

What we need to do is to use the same line of codes but instead of printing out the last backup stamp for each partition, we will configure the script to send us a nice email if the last backup stamp for any partition exceeded our backup window for AD. For example, if we are suppose to take AD backup everyday using our favorite backup solution, then an email will be sent if the last backup stamp on any AD partition is more than 1 day.

Download the script

The script does not need any special permissions. Any domain user can execute it. The script will take from you your AD backup frequency and will send email alert for you ONLY if the AD backup exceeded your backup frequency (i.e you have AD backup failure).

Like this:

One of the important things you need to do is to backup your certificate authority servers. In this post, i will give you some of my best practices about how to backup a CA that i came up from my experience.

1. System State Backup (Full backup ,not differential)

This is the preferred method of backup up a CA. It includes the following components related to CA services:

CA database: includes information about any certificate issued or revoked.

CA key Pair: The backup should include all versions of the CA certificates in case of CA certificate renewal.

IIS metabase: important if changes are made to the Certificate services web enrollment pages.

Registry settings:CA settings.

2. Manual Backup

Can backup only (CA Database) and (CA Key pair). Performing backup to the registry or IIS metabase is required additionally.

Manual backup can be performed via the GUI Console or by using the (certutil) :

Manual backup using CA console :

1. From the Start menu, point to Administrative Tools and click Certification Authority. In the console tree, ensure that Certificate Services is running.

3. In the console tree, right-click CA Name, point to All Tasks and click Backup CA.

4. On the Welcome to the Certification Authority Backup Wizard page, click Next.

5. On the Items to Backup page, input the following options:

Private Key and CA certificate. Includes the CA’s certificate and private key(s) in the backup set. Select this option only if you are using software CSP. If using hardware CSP, leave this check box cleared.

Certificate database and certificate database log. Always select this option to ensure that you include the CA database and log files in the backup set.

Perform incremental backup. This check box is not usually selected. Full backups of the CA database and log files are recommended instead.

Backup to this location. Select a folder on the local file system that does not contain any existing data.

6. If the Certification Authority Backup Wizard dialog box appears, click OK to create the location designated on the Items to Backup page.

7. If you choose to back up the private key and CA certificate, open the Select a Password page, type and confirm a password to protect the PKCS #12 file generated by the backup procedure, and click Next.

Once the backup is complete, open the folder designated in step 5. In the folder, there is a *.p12 file (the PKCS #12 backup of the CA’s certificate and private key) and a sub folder named Database that contains the backup of the CA database and log files.

Manual backup using certutil :

If you are using a software CSP, ensure that the backup set includes both the CA database and the CA’s key pair. To do this, use the following procedure:

5. At the command prompt, at the Enter New Password prompt, type a complex password and press ENTER.

6. At the command prompt, at the Confirm New Password Prompt, type the same password again and press ENTER.

7. When the backup is complete, ensure there are no error messages and close the command prompt.

You are providing a password to protect the PKCS #12 file containing the CA’s key pair. To create a successful backup of the private key, you must be a local administrator of the computer; to create the backup of the CA database, you can only hold the Common Criteria role of backup operator. In other words, you can only run this command successfully if Common Criteria role separation is not enforced.

If Common Criteria role separation is enforced, you can separate the two backups by running two certutil commands.

To backup only the CA database, a backup operator can use the –backupdb option, as shown here:

5. At the command prompt, at the Enter New Password prompt, type a complex password and press ENTER.

6. At the command prompt, at the Confirm New Password prompt, type the same password and press ENTER.

7. When the backup is complete, ensure there are no error messages and close the command prompt.

Note Ensure that you have included the registry in the backup by including the SystemState in the backup set or by manually backing up the HKLM\System\CurrentControlSet\Services\CertSVc\Configuration\CAName registry key.

So what is my recommendation ? Well, i recommend to take system state backups in daily basis of the CA server, and to schedule a batch file to take backup of the database using the certutil -backupdb to a folder, and then to include this folder on your normal backup cycle. I also recommend if you can export the private key of the CA and keep it safe. Don’t forget to backup HKLM\System\CurrentControlSet\Services\CertSVc\Configuration\CANameregistry key. It contains all your CA configuration

Like this:

It is without any doubts, one of the most critical tasks that Active Directory administrators forget/ignore !

It is not enough to take backups of your Active Directory (which can be done simply by backing up the domain controller’s System State) , as you also need to verify that the backup can be restored.

Note: Backing up the domain controller’s system state will backup your whole Active Directory, SYSVOL (Your Group Policies) and your[ DNS zones (only if they are integrated in Active Directory).]

Scenario : What is the test scenario

Suppose you have couple of domain controllers at your enterprise. You are taking AD backups in regular basis which you should always do (by taking backup to DC’s system state).

You are asked to verify that the Active Directory backup that you are taking is healthy and can be restored. You may have also been asked to perform regular restores as part of a certain regulations or procedures.

So you want to create a virtual machine, restore the Active Directory backup on it, and have a look to your Active Directory Users and Computers snap in to verify your AD objects are restored, and may be verify all your GPOs are restored. Then you can destroy this VM and you are done.

Let us do it

1. Create a virtual machine

Virtual Machine name : DOES NOT MATTER

Virtual Machine network connectivity : it should have a disablednetwork card at this stage. Never Ever allow this machine to access or route to your live environment in any way.

Virtual machine domain membership : Not joined to any domain (should be a workgroup)

It is recommended to have an additional disk on this VM to host the restored files

2. Now go to one of your domain controller and let us start creating a backup job:

We need to take backup for the domain controller’s system state.

We will be using the Windows Built in Server Backup software and we are assuming that the domain controller is running Windows 2008 and above

To use it, you need to go to Add Features, and add (Windows Server Backup) component manually.

Now open the Windows Server Backup console.

Click on (Backup Once) to start the backup job.

In the (Backup Options), click (Different Options)

In the (Select Backup Configuration), select (Custom)

In the (Select Items for Backup) click (Add Items) and click the (System State)

In the (Specify Destination Type), click what fits you

That’s it. Just wait for the backup to finish, and you will see a folder named (WindowsImageBackup).

You can also go to the DC Event Log, under Microsoft>Backup>Operational and find the event ID = 4 that indicates successful backup operation.

3. Now, go to you VM, i assume that it has C and D drive, and do the following :

In a secure and isolated way, move the WindowsImageBackup folder as is , to the root of the D drive D drive of the VM. This should happen without connecting the VM to the network at all.

Note: ALWAYS located the WindowsImageBackup to the root of the data drive of the VM. This will allow the Backup software to locate it easily

4. As the restored files are now located under (BackupDC) folder on the VM D drive, and after ensuring that the VM is isolated and not connected to any network and cannot route traffic to your live environment, perform the following to start the restore:

Notice that this VM doesn’t have any active directory on it, but still you will this option available .

Now the VM will boot in the (Directory Services Mode)

Now from this mode, open the Windows Server Backup console on the VM (install it from the Add Features if it is not installed yet).

Click on the (recovery) option to start the recovery wizard.

on the (Getting Started) page, click (A backup stored on another location)

On the (Specify Location Type) click (Local drives).

In the (Select Backup Date), leave defaults

On the (Select Recovery Type) click (System State)

On the (Select Location for System State Recovery) leave the defaults (which is Original Location)

You will get an confirmation box, click OK and continue

Acknowledge the Confirmation box

5. Now after the recovery process is completed, you can go to the VM > C:\Windows\NTDS and confirm that the AD databases are there, and you can go to the SYSVOL directory and confirm that your group policies are there

6. This is the tricky part !! If you try to open the Active Directory Users and Computers or even GPMC.msc console from the VM , you will get an error that the domain does not exist. This is absolutely normal. The reason is that the restored DC in the VM needs to point to itself as a DNS server. So what you should do is to enable the network card on the VM and giving it fake IP and subnet mask, and configure the DNS on its network card to point to itself (to its fake IP). MAKE SURE THAT STILL THE VM CANNOT ROUTE TRAFFIC TO YOUR LIVE ENVIRONMENT.

Now, wait a little bit or restart the VM and then try to browse the Active Directory Users and Computers, and it will work. You can now see all your AD objects. If you open GPMC.MSC , you can see all your group policies.

Note: If you didn’t find the Active Directory Users and Computer console on the VM after the restore, go to run>mmc.msc and add the Active Directory Users and Computers snap-in manually

7. After you have confirmed everything looks fine, destroy the VM and never connect it to your network. Have a nice restore day !!!

Notes:

First of all you cannot take backup from a version of Windows and restore to another version. The Windows Backup will give you catalog corrupted error. For example, if you are taking a backup from Windows 2012 DC, you can it restore it using Windows Backup on a Windows 2008 R2 server

After you finish all the restore, and when you will notice that the DNS may not show you any data because it was Active Directory to do some initial synchronization. On the other hand, the AD cannot start without a DNS. To solve this issue, on the VM add this registry

Like this:

Exchange 2010 Disaster Recovery Across data centers

Hi everyone !!! Have you ever considered how to perform Exchange 2010 data center switch over ? or what will happen when your CAS servers in your main data center are down and you want to switch to CAS servers in secondary data centers?

Or have you ever considered to stretch your DAG over data centers and you wonder what are the benefits? It is all here and explained with piratical experience right from the field

Like this:

BitLocker is a great out of the box encryption tool for disk volumes. If you read about this technology, you realize that the most challenging part is the disk recovery key and how to maintain/backup it.

Well, Microsoft did a great job documenting different ways for doing that. One of those methods is to backup keys to Active Directory. Simple and easy, even you can control this behavior via Group Policies.

Problem

Let me describe the problem with BitLocker AD Key Backup and Recovery

Now, Imagine that you enabled BitLocker key recovery in Active Directory. This will simply create an entry per volume on a specific multi value attribute in the computer object.

Now, suppose that you have deleted the computer object from AD.

Or think about this scenario : The computer has C drive with O.S and D drive for data, both are bitlocker encrypted. You decided to format the C drive and join it again to the domain, so you format the C drive, delete the computer object AD, so you could join it to the domain again. Now think about the recovery key for the D drive in this scenario !!!!! It is lost when you deleted the computer object.

Bad things happen and believe me that you will always find your self in a situation where computer objects get deleted, even as part of organized cleanup process.

You will end up, getting back to AD restore or AD recycle bin, and believe they are not that easy to deal with.

Solution !!!

I have created a simple script that needs only read access to Computer objects and to BitLocker Recovery Information.

Now here is the script that will go to all computer objects in your Active Directory, and create a nice CSV file for you with all recovery keys for all BitLocker Computers. You can schedule it to run daily and you can keep those CSV for a month and then automatically delete the oldest.

This way, you will have a solid place to go to when some one deleted a computer object and you need the BitLocker Recovery Key. Believe me , this helped me a lot.

Like this:

In Part 1 , we have restored an Exchange database to a recovery database called RecoveryDB. Now let us extract the content of a mailbox for User named “John Smith” whose SamAccountName is “JohnS”

First thing you need to get information about mailboxes in the recovery database after it is mounted

Get-MailboxDatabaseRecoveryDB |Get-MailboxStatistics

Note: All restore operations uses the mailbox DisplayName or GUID to identify mailboxes in the recovery database and not SamAccountName

2. Restore data to the live user mailbox (John Smith) under a folder named Recovery:

New-MailboxRestoreRequest

–SourceDatabaseRecoveryDB

– SourceStoreMailbox“John Smith”

–TargetMailbox“John Smith”

–TargetFolderRecoveryItems

Where :

New-MailboxRestoreRequest : is the new way for restoring items. It is a background process so, when you run the command, nothing will happen, but in the background, the exchange will start working on the request. To get information about what is happening after submitting the request, type : Get-MailboxRestoreRequest |Get-MailboxRequestStatistics

SourceStoreMailbox : is the user that you want to recover his mailbox. You cannot use the user’s SamAccountName here (JohnS) for example, you only can use the user’s Display Name or GUID

TargetMailbox : is the user mailbox that you want to restore things to.

TargetFodler: is the folder that is created in the (TargetMailbox) user with all restored items. If you don’t mention this parameter, then the command will merge content from the restored items and the (TargetMailbox) items.

So, running the above command, will create a folder Named (recoveryItems) in John Smith mailbox with his recovery items after the restore command is completed. If you didnt mention the (TargetFolder) option, then John Smith will find his mailbox merged with items from the restore operation. This is useful in some scenarios where the user deleted all his mailbox items.

3. (Or) Restore data to a temp mailbox named (TestUser)

New-MailboxRestoreRequest

–SourceDatabaseRecoveryDB

– SourceStoreMailbox“John Smith”

–TargetMailbox“TestUser”

–AllowLegacyDNMismatch

Where :

AllowLegacyDNMismatch: is simply telling Exchange that we want to restore John Smith mailbox to another temp mailbox.

BIG NOTE : If you get error while restoring a certain mailbox, then you can get more information about the error by typing (Get-MailboxDatabaseRecoveryDB |Get-MailboxStatistics |FL). Sometimes, using the old Exchange restore command solves the problem :

Like this:

It is one of the pain points of Exchange admins to restore mailbox. Well, i did my homework and I want to share with you how to restore Exchange mailboxes using Backup Exec (Software from Symantec) or even snapshot software:

Step 1: Creating the Recovery Exchange Database

Create a Recovery DB named RecoveryDB by running this Power Shell command:

Mount the database and check (This database can be overwritten by a restore).

Dismount the database before starting the restore job.

Step 2: Restore Database from Tape (Backup Exec)

Use Backup Exec to run a restore job. ServerName in the below picture represent the Exchange Mailbox Server where the recovery database is hosted (mounted)

or using the new Backup Exec 2012 , here is the settings:

Settings are simple and I cannot go to each setting and explain it. I am supposing that you are restoring from a full backup Exchange database and using either Backup Exec 2010 or 2012.

Backup Exec is great because it will restore the database and then go and replay all logs. In short words, after the restore, the restored database will be in a clean shutdown scenario.

Step 3: Restore Database from Snapshot

If you are restoring from Snapshot software, then the restore will bring the recory database to dirty shutdown simply because snapshot restore will not replay the logs back. You have to do it manually by :

Make sure your logs and DB files are in same directory.

Delete the CHK file

To dump the database and check if the database is dirty shutdown, type (eseutil /mh dbfile.edb )

To replay logs and bring the database to healthy (clean shutdown), type (eseutil /R EXX /I /D), Where EXX is the log prefix