Elsewhere

This problem ate my head for the past 2 days and wasted a lot of time. For such a simple issue it drove me quite mad.

Built a bunch of DCs for our branch offices. One of them gave trouble with the DHCP server. I authorized it successfully, but the service kept complaining that it wasn’t authorized. Event ID 1046.

The DHCP/BINL service on the local machine, belonging to the Windows Administrative domain mydomain.dom, has determined that it is not authorized to start. It has stopped servicing clients. The following are some possible reasons for this:

This machine is part of a directory service enterprise and is not authorized in the same domain. (See help on the DHCP Service Management Tool for additional information).

This machine cannot reach its directory service enterprise and it has encountered another DHCP service on the network belonging to a directory service enterprise on which the local machine is not authorized.

Some unexpected network error occurred.

Did the obvious ones like reboot server :p and restart service :) and un-authorize and re-authorize the server (no errors either time). Also went ahead and removed the role itself and added back. Nothing helped!

Browsed to the Services section (which can be enabled from the View menu if not already visible).

Browsed to the NetServices section within this.

On the right pane I had an entry for the IP address for the DHCP server I was trying to authorize. Not an entry by name, but by IP. Dunno why. (All other entries were by name, so I am guessing this is a leftover or a mistake by someone in the past).

I deleted this entry.

Waited a while, and then authorized the server.

No errors now!

Screenshot of the offending entry just for the heck of it (the blacked out part was an IP address):

Alternatively one can open ADSI Edit and go to CN=NetServices,CN=Services,CN=Configuration,DC=myDomain,DC=dom. Then delete the entry (as above) from there.

What’s odd in my case is that the IP that I deleted was assigned to the DHCP server I wanted to authorize. Am guessing the CNF (short for conflict?) following by the GUID indicates some issue.

Our deployment team needed a few DHCP options set for all our scopes. There was a brickload of these scopes, no way I was going to go to each one of them and right-click add the options! I figured this was one for PowerShell!

Yes, I ended up taking longer with PowerShell coz I didn’t know the DHCP cmdlets but hey (1) now I know! and (2) next time I got to do this I can get it done way faster. And once I get this blog post written I can refer back to it that time.

The team wanted four options set:

Predefined Option 43 – 010400000000FF

Custom Option 60 – String – PXEClient

Predefined Option 66 – 10.x.x.x

Predefined Option 67 – boot\x86\wdsnbp.com

PowerShell 4 (included in Windows 8.1 and Server 2012 R2) has a DHCP module providing a bunch of DHCP cmdlets.

First things first – I needed to filter out the scopes I had to target. Get-DhcpServerv4Scope is your friend for that. It returns an array of scope objects – filter these the usual way. For example:

Now, notice that one of the options to be added is a custom one. Meaning it doesn’t exist by default. Via GUI you would add it by right clicking on “IPv4” and selecting “Set Predefined Options” then adding the option definition. But I am doing the whole thing via PowerShell so here’s what I did:

I had a bit of trouble with option 43 because it has a vendor defined format and I couldn’t input the value as given. From the help pages though I learnt that I have to give it in chunks of hex. Like thus:

Today I learnt that to authorize a DHCP server in a child domain you must be an Enterprise Admin or a Domain Admin in the forest root domain or have the rights delegated to you. I was always under the impression you only needed to be a Domain Admin (not necessarily of the forest root domain).

The reason you need to have forest level rights is because authorization happens under CN=NetServices,CN=Services,CN=Configuration,DC=mydomain – which as you can see is the Configuration partition, which is replicated forest-wide, so to which you need forest level rights. Armed with this knowledge you can either assign permissions to this container directly or use the Active Directory Sites and Services MMC to delegate permissions. In case of the latter, note that you have to click the Active Directory Sites and Services node and select the View menu (or right click that node and select View) to then reach the Show Services Node option.

By default only domain members (domain joined computers & domain users) are allowed to update the zone for secure updates. This is controlled by the ACLs on the zone (which can be viewed via the Security tab of the zone – check out the ACE for “Authenticated Users“). See this link for more.

Nonsecure and secure => Both secure and nonsecure updates are allowed.

Scavenging

Dynamic DNS updates result in records being added and deleted to DNS. But while records are correctly added, it is not always the case that the record is also correctly removed. For instance, a client could have got an IP address from DHCP and dynamically registered its A record. Maybe the client then crashed so it never removed the dynamically registered record. The address, however, is removed from DHCP after the lease expires and could later be assigned to another client – who also dynamically registers itself in DNS – resulting in two A records, both to the same IP address, but one of them incorrect. To prevent such issues DNS scavenging is required. This removes stale DNS records after a pre-defined period.

Scavenging is set at 3 places on a Windows server and all three must coincide for a record to be scavenged. These places are:

an individual record;

the zone; and

the server performing the scavenging.

The scavenging setting on an individual record can be viewed only after selecting View > Advanced in the DNS MMC and then viewing the properties of a record.

When a dynamic DNS record is created it has a timestamp (rounded down to the nearest hour when the record was created).

When a record is first created it is considered an “Update”.

When an existing record is updated with the same IP address it is considered a “Refresh”.

When an existing record is updated with a new IP address it is considered an “Update”.

Every 24 hours Windows clients will attempt to to dynamically update the DNS record. The update could be considered an update or a refresh depending on whether the IP changes as above.

If a record is enabled for scavenging its properties window will have a tick next to “Delete this record when it becomes stale”.

Static records don’t have this ticked by default (because they are not meant to be scavenged).

If this is manually ticked (for a static or dynamic record) then a timestamp will be set to when it was ticked (rounded down to the nearest hour).

It is possible to set scavenging on a zone and all its records via the following command: dnscmd /ageallrecords

This is not recommended as it enables scavenging on all records – even static. Do not use this command on zones with static records.

The scavenging setting for a zone can be viewed via the Aging button in the zone properties (by default the setting is off).

The aging values for a zone are replicated to all DNS servers hosting the zone.

Two intervals are in play here:

No-refresh interval: Once a record is refreshed, it is not refreshed again until this time period has passed.

The purpose of this seems to be to reduce replication traffic. If a client refreshes its DNS record every 24 hours, those are ignored by the DNS server for the no-refresh interval, and not replicated to other DNS servers.

Refresh interval: How much time to wait once a record is refreshed before it can be scavenged?

So this interval specifies how long the server should wait after a record has refreshed before it can considered it a candidate for scavenging.

The default value is 7 days. This means, if a record is refreshed today, the server will wait for 7 days to see if it’s refreshed again. If it is not, the server considers this record ready for scavenging.

Both intervals must be passed for a record to be expired. By default both are 7 days, so what this means is:

If a record is created/ updated/ refreshed today, for the next 7 days the record is considered current – irrespective of whether any refreshes happen or not (because remember: during the no-refresh interval refreshes, if they happen, are ignored so the server considers the record as current for this period).

After those 7 days have passed, the server checks if there are any refreshes.

If there are, the timestamp is accordingly updated and it goes back into waiting the no-refresh interval again.

If there are no refreshes, the server now waits 7 days of refresh interval to see if any refreshes happen. If they do, the record goes back into the no-refresh interval; if there aren’t any, the record is ready for scavenging.

As an aside, the default lease duration for Windows server DHCP leases is 8 days. Which is why the no-refresh interval is set to 7 days by default. During these 7 days the address won’t be allocated to any other client, nor will it change with the client, so chances of a refresh are minimal.

DHCP leases and Dynamic DNS updates can conflict if clients are responsible for updating DNS with their addresses (which is usually the default).

Say a client got an IP address from DHCP (leased to it for 8 days, remember). The client will update that in DNS. For the next 7 days any refreshes from the client are ignored (no-fresh interval, expected). From the 7th day any refreshes/ updates will be considered.

Say our client went offline on the 3rd day. So on day 7 it doesn’t send a refresh – no problemo, DNS will not scavenge the record yet, it will simply wait for another 7 days.

On the 8th day, however, DHCP will release that IP address for others to use. Any new client that comes up will now get this address. This new client will send a Dynamic DNS update to the DNS server – creating a new A record to the same address, but with this new client’s name. Thus there are two DNS entries now to the same IP address!

Only after the refresh interval expires (7 days) can the old record be actually scavenged by the server (and even then there could be a delay based on the server setting – see below).

For this reason it is recommended that the DHCP lease duration match the “no-refresh+refresh” interval of DNS scavenging. In the default case, either increase the DHCP lease to 14 days (7+7 days) or decrease the no-refresh and refresh intervals to 4 days (so the sum is 8 days, the DHCP lease).

Typical solution is to put all DHCP servers in a group called DnsUpdateProxy, but that’s not recommended – especially for DCs – because if a server is in this group the dynamic DNS records it creates have no security (so in the case of a DC this means the SRV records written by netlogon can be changed by anyone!)

It is better to create a low privilege AD user and get all DHCP users to use that account to register records. This way the dynamic DNS records are secured to that user.

Also note: if dynamic DNS records are written by a server in the DnsUpdateProxy group – i.e. with no security – if any other machine (even one not in this group) changes this record (because the records are open to all) the ACLs of that record will be changed to only grant that machine permissions to the record. Thus the original DHCP server will lose rights to that record. DnsUpdateProxy is not a good idea.

It is important that clients be disabled from registering dynamic DNS updates in this case. Else the ACLs on the DNS record created by the client will prevent updates/ deletions from the DHCP server to the DNS server for those records.

When scavenging is enabled for a zone, the “Date and time” the zone can be scavenged after value is set to the time the setting was enabled (rounded down to the nearest hour) plus the refresh interval period.

It is also possible to right click a DNS server and set scavenging values for all zones on that server. These only apply to zones created after this setting was changed (unless the setting to modify existing zones is explicitly selected).

The scavenging setting on the server can be enabled via the “Advanced” tab of the server properties in the DNS MMC (by default the setting is off).

When this setting is enabled, the scavenging period is set to a default of 7 days. The scavenging period defines how often the server will try to scavenge records.

Does this mean every time the server starts scavenging all records are immediately deleted? No – because you have to also consider the no-refresh and refresh intervals of above. When a server runs its scavenging task, if a record to be scavenged has not crossed the refresh interval, it will not be removed. Similarly, if a record has crossed its refresh interval and is ready to be scavenged, if the server’s scavenging period isn’t due for a few more days nothing will happen. It’s only when the scavenging period is due that this record will be scavenged.

When the server scavenges records it logs an event ID 2501 indicating how many records were scavenged. If no records were scavenged, an event ID 2502 will be logged.

Note: You needn’t enable the scavenging setting on all servers hosting a zone. As long as any one server scavenges, the changes will propagate to others. In fact, it’s preferred to have only one server (or a set of servers) scavenge a zone as that will make it easier to troubleshoot. If all servers hosting a zone have scavenging on and the zone records are not being scavenged, we will have to check all these servers to see why scavenging isn’t happening.

In practice, it is likely that all servers have scavenging turned on (because they are hosting multiple zones and could be responsible for scavenging one of those zones). But once a server has scavenging turned on it will scavenge any zones that has scavenging turned on. It is possible to restrict the servers that are allowed to scavenge a zone – even if the server and zone have scavenging turned on – via the dnscmd command. The syntax is as follows:

1

dnscmd/zoneresetscavengeservers

The IP addresses are optional. If no address is given, all servers are allowed to scavenge it. Example:

The first of my (hopefully!) many posts on Active Directory, based on the WorkshopPLUS sessions I attended last month. Progress is slow as I don’t have much time, plus I am going through the slides and my notes and adding more information from the Internet and such.

This one’s on the services that are critical for Domain Controllers to function properly.

DHCP Client

In Server 2003 and before the DHCP Client service registers A, AAAA, and PTR records for the DC with DNS

In Server 2008 and above this is done by the DNS Client

Note that only the A and PTR records are registered. Other records are by the Netlogon service.

File Replication Services (FRS)

Replicates SVSVOL amongst DCs.

Starting with Server 2008 it is now in maintenance mode. DFSR replaces it.

To check whether your domain is still using FRS for SYSVOL replication, open the DFS Management console and see whether the “Domain System Volume” entry is present under “Replication” (if it is not, see whether it is available for adding to the display). If it is present then your domain is using DFSR for SYSVOL replication.

Alternatively, type the following command on your DC. If the output says “Eliminated” as below, your domain is using DFSR for SYSVOL. (Note this only works with domain functional level 2008 and above).

1

2

3

4

C:\>dfsrmig/getglobalstate

CurrentDFSRglobalstate:'Eliminated'

Succeeded.

Stopping FRS for long periods can result in Group Policy distribution errors as SYSVOL isn’t replicated. Event ID 13568 in FRS log.

Apart from the dfsrmig command mentioned in the FRS section, the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\DFSR\Parameters\SysVols\Migrating Sysvols\LocalStateregistry key can also be checked to see if DFSR is in use (a value of 3 means it is in use).

If a DC is offline/ disconnected from its peers for a long time and Content Freshness Protection is turned on, when the DC is online/ reconnected DFSR might block SYSVOL replications to & from this DC – resuling in Group Policy distribution errors.

Content Freshness Protection is off by default. It needs to be manually turned on for each server.

Content Freshness Protection exists because of the way deletions work.

DFSR is multi-master, like AD, which means changes can be made on any server.

When you delete an item on one server, it can’t simply be deleted because then the item won’t exist any more and there’s no way for other servers to know if that’s the case because the item was deleted or because it wasn’t replicated to that server in the first place.

So what happens is that a deleted item is “tombstoned“. The item is removed from disk but a record for it remains the in DFSR database for 60 days (this period is called the “tombstone lifetime”) indicating this item as being deleted.

During these 60 days other DFSR servers can learn that the item is marked as deleted and thus act upon their copy of the item. After 60 days the record is removed from the database too.

In such a context, say we have DC that is offline for more than 60 days and say we have other DCs where files were removed from SYSVOL (replicated via DFSR). All the other DCs no longer have a copy of the file nor a record that it is deleted as 60 days has past and the file is removed for good.

When the previously offline DC replicates, it still has a copy of the file and it will pass this on to the other DCs. The other DCs don’t remember that this file was deleted (because they don’t have a record of its deletion any more as as 60 days has past) and so will happily replicate this file to their folders – resulting in a deleted file now appearing and causing corruption.

It is to avoid such situations that Content Freshness Protection was invented and is recommended to be turned on.

Here’s a good blog post from the Directory Services team explaining Content Freshness Protection.

DNS Client

For Server 2008 and above registers the A, AAAA, and PTR records for the DC with DNS (notice that when you change the DC IP address you do not have to update DNS manually – it is updated automatically. This is because of the DNS Client service).

Note that only the A, AAAA, and PTR records are registered. Other records are by the Netlogon service.

DNS Server

The glue for Active Directory. DNS is what domain controllers use to locate each other. DNS is what client computers use to find domain controllers. If this service is down both these functions fail.

Kerberos Distribution Center (KDC)

Required for Kerberos 5.0 authentication. AD domains use Kerberos for authentication. If the KDC service is stopped Kerberos authentication fails.

NTLM is not affected by this service.

Netlogon

Maintains the secure channel between DCs and domain members (including other DCs). This secure channel is used for authentication (NTLS and Kerberos) and DC replication.

Writes the SRV and other records to DNS. These records are what domain members use to find DCs.

The records are also written to a file %systemroot%\system32\config\Netlogon.DNS. If the DNS server doesn’t support dynamic updates then the records in this text file must be manually created on the DNS server.

The Windows Time service on every domain member looks to the DC that authenticates them for time time updates.

DCs in the domain look to the domain PDC for time updates.

Domain PDCs look to the domain PDC of the domain above/ sibling to them. Except the forest root domain PDC who gets time from an external source (hardware source, Internet, etc).

From this link: there are two registry keys HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config\MaxPosPhaseCorrection and HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Config\MaxNegPhaseCorrection that restrict the time updates accepted by the Windows Time service to the number of seconds defined by these values (the maximum and minimum range). This can be set directly in the registry or via a GPO. The recommended value is 172800 (i.e. 48 hours).

w32tm

The w32tm command can be used to manage time. For instance:

To get an idea of the time situation in the domain (who is the master time keeper, what is the offset of each of the DCs from this time keeper):

1

w32tm/monitor

To ask the Windows Time service to resync as soon as possible (the command can target a remote computer too via the /computer: switch)

1

w32tm/resync

Same as above but before resyncing redetect any network configuration changes and rediscover the sources:

1

w32tm/resync/rediscover

To get the status of the local computer (use the /computer: switch to target a different computer)

1

w32tm/query/status

To show what time sources are being used:

1

w32tm/query/source

To show who the peers are:

1

w32tm/query/peers

To show the current time zone:

1

w32tm/tz

You can’t change the time zone using this command; you have to do:

1

tzutil/s"Time Zone Name"

On the PDC in the forest root domain you would typically run a command like this if you want it to get time from an NTP pool on the Internet:

specify a list of peers to sync time from (in this example the NTP Pool servers on the Internet);

the /update switch tells w32tm to update the Windows Time service with this configuration change;

the /syncfromflags:MANUAL tells the Windows Time service that it must only sync from these sources (other options such as “DOMHIER” tells it to sync from the domain peers only, “NO” tells it sync from none, “ALL” tells it to sync from both the domain peers and this manual list);

the /reliable:YES switch marksthis machine as special in that it is a reliable source of time for the domain (read this link on what gets set when you set a machine as RELIABLE).

Note: You must manually configure the time source on the PDC in the forest root domain and mark it as reliable. If that server were to fail and you transfer the role to another DC, be sure to repeat the step.

On other machines in the domain you would run a command like this:

1

w32tm/config/update/syncfromflags:DOMHIER/reliable:NO

This tells those DCs to follow the domain hierarchy (and only the domain hierarchy) and that they are not reliable time sources (this switch is not really needed if these other DCs are not PDCs).

Active Directory Domain Services (AD DS)

Provides the DC services. If this service is stopped the DC stops acting as a DC.

Pre-Server 2008 this service could not be stopped while the OS was online. But since Server 2008 it can be stopped and started.

The Active Directory Database Mounting Tool was new to me so here’s a link to what it does. It’s a pretty cool tool. Starting from Server 2008 you can take AD DS and AD LDS snapshots via the Volume Snapshots Service (VSS) (I am writing a post on VSS side by side so expect to see one soon). This makes use of the NTDS VSS writer which ensures that consistent snapshots of the AD databases can be taken. The AD snapshots can be taken manually via the ntdsutil snapshot command or via backup software or even via images of the whole system. Either ways, once you have such snapshots you can mount the one(s) you want via ntdsutil and point Active Directory Database Mounting Tool to it. As the tool name says it “mounts” the AD database in the snapshot and exposes it as an LDAP server. You can then use tools such as ldp.exe of the AD Users and Computers to go through this instance of the AD database. More info on this tool can be found at this and this link.

AD WS is what the PowerShell Active Directory module connects to.

It is also what the new Active Directory Administrative Center (which in turn uses PowerShell) too connects to.

AD WS is installed automatically when the AD DS or AD LDS roles are installed. It is only activated once the server is promoted to a DC or if and AD LDS instance is created on it.