Wednesday, April 27, 2011

I was recently asked by an ex-colleague about having an issue with mounting a Windows server share from within an ESX 3.5 server and my first instinct turned out to be wrong so I thought it might be worth while to write up a blog post with the specific error he encountered and stating the resolution.

Problem

You’ve logged into an ESX server either through SSH or the console.

You create a directory with MKDIR.

You make an attempt to mount a windows share but get the following error message:

Since my ex-colleague was trying to mount a Windows Server 2003 share, the first response I had with my coworker was to ensure that he has disabled the Microsoft network server: Digitally sign communcations (always) policy in the Local Security Policy under Local Policies –> Security options. This turned out to not be the cause because he had already done it. After spending a few minutes to think about it over the phone, I asked if he had actually enabled the SMB Client service on the ESX server with the esxcfg-firewall command and as it turns out, he hasn’t. Here’s what he ended up having to do to get it to work:

Tuesday, April 26, 2011

As mentioned in one of my earlier posts, I’ve been performing some tests with Active Directory restores to provide myself with a refresh of the procedure as I haven’t done it for awhile and just so I won’t need to do this again in the future, I thought it would be beneficial to myself to document the process so I have screenshots and steps to reference to in the future.

We’re all familiar with the native NTBackup utility that Windows Server 2003 provides and it is important to remember that whenever a change is to be made to your directory service, always remember to perform a backup of your domain controller. I almost always perform a backup of more than one domain controller whenever I’m about to make any changes.

Backup Process

Start up NTBackup on your domain controller and select the System State from the list of items to be backed up:

Click on the Start Backup button and wait for the backup to complete:

Now that we have a backup completed, let’s begin mimicking an unintended change in Active Directory. For the purpose of this example, we’ll be deleting the Public Key Services node in the Configuration container:

As shown in the screenshot above, the Public Key Services node in the Configuration container has been deleted (bye bye CA services) from Active Directory.

Restore Process

Prior to actually proceeding to restore the System State that was backed up, we will need to restart the domain controller in Directory Services Restore Mode either locally or remote. Locally is done by simply hitting F8 during start up while for remote, we’ll need to edit the start up options in the boot.ini file:

To force the domain controller to restart in Directory Services Restore Mode simply append the following to the end of the boot.ini file:

Once the changes have been completed, proceed with rebooting the domain controller:

If you have access to the console, you’ll notice that once the domain controller begins to start up, the wallpaper will look different:

Wait for the boot up process to complete and you’ll be able to remote desktop to the domain controller:

It’s important to note that you will need to login with the Active Directory Restore Mode password and if for some reason you need to have forgotten it, use the following KB article to reset it: http://support.microsoft.com/kb/322672.

Once you’ve successfully logged in, you’ll be promopted with the following warning/informational message:

One of the first things I do once I successfully get into the domain controller is to remote the /SAFEBOOT:DSREPAIR switch from the boot.ini file:

Once you’ve completed the change to the boot.ini file, continue with restoring the the backed up System State to the domain controller:

Note that if the domain controller you are restoring is the only domain controller in your environment, you will need to select the last option:

When restoring replicated data sets, mark the restored data as the primary data for all replicas.

Once the backup is complete, click on the Close button and remember to select No when asked to reboot:

The following text file with a list of authoritatively restored objects has beencreated in the current working directory:

ar_20110420-145704_objects.txt

None of the specified objects have back-links in this domain. No link restore file has been created.

Authoritative Restore completed successfully.

authoritative restore:

Once the restore is complete, proceed to reboot the server:

Verifying the restored object

Once the domain controller has been rebooted, open ADSIEdit, connect to domain controller you restored the partition on and navigate to the node that contains the restored object:

As seen in the screenshot above, while DC1 has the Public Key Services node in the Configuration container restored, the changes haven’t been replicated to DC2 yet:

What’s also worth noting is that opening Replication Monitor and forcing a replication shows that certain partitions aren’t replicating properly:

What’s important at this point is not to panic because it does take a bit of time for the 2 domain controllers (or more if you had more than 2) to resolve the conflicts they seen between each other after a restore.

For those who are interested in how this conflict is resolved, domain controllers actually have an attribute named uSNChanged that it uses to track which domain controller has the most up-to-date value for the attribute and just so I don’t accidentally give the wrong explanation for this attribute because of my poor memory, I will simply provide a copy and paste from Microsoft:

When a domain controller modifies an object it sets that object's uSNChanged attribute to a value that is larger than the previous value of the uSNChanged attribute for that object, and larger than the current value of the uSNChanged attribute for all other objects held on that domain controller. As a consequence, an application can find the most recently changed object on a domain controller by finding the object with the largest uSNChanged value. The second most recently changed object on a domain controller will have the second largest uSNChanged value, and so on.

This means that if you open up the properties of the object you’re restoring, you can compare the uSNChanged values for the same object on 2 domain controllers to determine which one will eventually take precedence against the other:

Depending on your environment, it may take some time before the restored object shows up on the other domain controllers so what I typically do is to force a replication periodically in Replication Monitor and use the results of the replication to determine whether the changes have been replicated already. Just in case the previous sentence wasn’t clear, what I basically mean is:

If you see problems in the forced replication, it means you still need to wait.

If you see a successful replication when you force a replication, it means the restore has been replicated.

Once no errors are reported in the replication, you should be able to see the restored object within ADSIEdit:

Hope this helps anyone out there who have not seen what the backup and restore process looks like. Reference articles from Microsoft can be found here:

Sunday, April 24, 2011

I recently deployed Microsoft Lync Server 2010 in a small environment and was told that Microsoft LCS and OCS was never deployed so I went ahead and treated the environment as a greenfield but as soon as I began to enable users for Lync, I noticed that some users would throw the following error when I try to enable them in PowerShell:

As some of you have already guessed, the reason why I am getting this error is because the user I was trying to enable for Lync was at one point enabled for Microsoft OCS and therefore the Enable-CsUser cmdlet is recommending that I try using the cmdlet to move a legacy user instead. It would have been easy to get these users over if the old OCS environment still existed but that wasn’t the case for this organization.

Solution

What I ended up doing was to open up adsiedit.msc, navigate to the user’s object, open the properties, and removing or setting the following attributes to “<not set>” and “0”:

msRTCSIP-ArchivingEnabled

msRTCSIP-FederationEnabled

msRTCSIP-InternetAccessEnabled

msRTCSIP-OptionFlags

msRTCSIP-PrimaryHomeServer

msRTCSIP-PrimaryUserAddress

msRTCSIP-UserEnabled

Here’s an example of a user who still had legacy OCS attributes populated for their account:

Here’s an example of a user account that had the legacy OCS attributes removed:

I’d have to say that while this solution works, it properly doesn’t scale well if you had more than 50 users in the organization so if you fall into that situation, I would suggest that you make an attempt to script the procedure for everyone in Active Directory (make sure you run the script before you enable anyone for Lync).

Friday, April 22, 2011

As I prepare to plan for a infrastructure upgrade for a client, I decided to build a small lab environment to do some tests with directory services restoration as it’s been years since I’ve had to do a restore and seeing how most projects never seem to go as anticipated, I figure it would be best to be as prepared as I could through this refresher exercise. While performing test restores with my two test domain controllers, I noticed an odd error message that I would continuously receive when using the NTDSUtil command to mark OUs, objects or partitions as authoritative when I execute it in the Windows Support Tools directory. The following is the prompt with the error is what I would receive:

ntdsutil.exe – Unable To Locate Component

This application has failed to start because SAMSRV.dll was not found. Re-installing the application may fix this problem.

Failed to recover database from external backup. Error 0x7f(The specified procedure could not be found.).

Authoritative Restore failed.

Error 8000ffff parsing input - illegal syntax?

authoritative restore:

I originally didn’t know what was causing this because I would try rebooting the server, complete a restore of the system state again, execute ntdsutil with an authoritative restore command and it would work. Performing some searches on Google returned some results that lead me to the following SymantecKB:

Though I went ahead to try and delete this key, the restore would continuously fail.

It was not until after going through this process 5 to 6 times that I noticed that it would fail whenever I executed the ntdsutil.exe command in the C:\Program Files (x86)\Support Tools folder. The following screenshot is when I executed the command in a regular Windows Command Prompt successfully:

Note: I understand that the example above shows I restored a the domain partition but take my word that the difference doesn’t matter and that the reason why it worked was because I wasn’t in the support tools folder.

I’m not sure if this problem is specific to the server OS I used but just in case you’re encountering the same issue as I did, the following is the information about the domain controller:

Server OS: Windows Server 2003 R2 64-bit

Service Pack: SP2

Hotfixes and Updates installed: All patches installed up to April 22, 2011

Forest Functional Level: Windows 2003

Domain Functional Level: Windows 2003

Restoring Active Directory no matter the whole database, OUs, objects or partitions is always nerve racking so I hope this post will be able to help anyone who may encounter this problem during a restore.