We have a new customer whose Group Policies (GPO’s) and Scripts were not synchronizing between their two Domain Controllers, even though changes to Users and Computers were synchronized.
Did you know that Active Directory changes are sync’d between DC’s using two different services?
- User and computer changes in Active Directory are synced between DC’s using the Active Directory Domain Services (AD DS)
- Group Policies and Scripts (things in the SysVol) are synced between DC’s using the Distributed File Service (DFS)
- Yes, this is the same DFS that you can use to build name spaces, but even if you do not have DFS installed on a DC, DFS is running
From this we were able to figure out that the problem was with DFS. Here is the weird part. After more than a week of working with Microsoft tech support, we found the problem: DFS Uses The Volume NUMBER, Not The Drive Letter to get to the SYSVOL path
You might ask why anyone would care what DFS is using to look for the path to the SysVol. Good Question.
Our customer had cloned a Windows Server 2012 R2 disk using Macrium Reflect, then upgraded the cloned install to Server 2022. Macrium, cloned not only the files, but also the Volume information. Our customer then removed the drive letter from the old Server 2012 R2 install making it virtually hidden to us. We had no idea it existed.
Because they rarely made changes to GPO’s and scripts, they did not know that their Active Directory DFS was broken for many months, and so they could not tell us what may have precipitated this failure (i.e. new patches, reboot, addition of a new DC, demotion of an old DC… WHAT HAPPENED!?!?!?!)
This is one of the strangest problems I have ever run into on a Microsoft product.
After a few days of troubleshooting DFS, we called Microsoft Tech support. After 10 more hours Microsoft asked if this server had ever been restored from backup because the DFS logs were showing DFS could not find DCE8-5809 (see screenshot above). Our new customer told us that they had not restored from backup but they had decommissioned an old DC about a year ago… of course they failed to mention they used Macrium Reflect to clone it first!
I stumbled onto DISK MANAGER and noticed the the DISK 0 volume with no drive letter. I right clicked on it, assigned it a drive letter and found their old Server 2012 R2 installation AND that the volume had the same volume name as their active Server 2022 volume. OMG!!!
There are two ready ways to change the VOLUME NUMBER of a drive. The first is to use the free Volume ID tools from Microsoft SysInternalsand the second is to reformat a volume. We chose to reformat and then reboot the server.
CAUTION: Because Disk 0 likely still contains the Boot Loader in the SYSTEM RESERVED partition, we did NOT wipe the disk. We simple reformatted the operating system volume shown in the screenshot as UNUSUED 222.96GB.
Bingo! The SysVol started replicating and all the angles in heaven starting to sing at the same time.
0 Comments