29 October 2026, 15:11Виталий Лещенко

Windows Server for beginners. Part 18: How do you troubleshoot the domain?

All parts of the series “Windows Server for beginners”

"The domain is broken" means one of ten different states, and each has its own symptoms. In earlier parts the diagnostic tools appeared one at a time: dcdiag in part 4, event 1644 in part 5, Kerberos errors in part 6, repadmin in part 10, w32tm /monitor in part 12. Here they come together into a procedure, and the second half of the part is about three directory states that a reboot doesn't cure: objects back from the dead, a controller that went back in time, and a controller that missed half a year.

All commands below run from the administrator's workstation ws1 with RSAT (part 15) in PowerShell 7; whatever reads the logs and registry of the controllers goes through Invoke-Command (part 14). The examples use the domain corp.example.com with four controllers: dc1 and dc2 in the site Office, dc3 and dc4 in the site Branch; dc1 is the PDC, dc3 runs Windows Server 2019, the rest run 2025.

The order of checks

Five steps, always in this order:

1. dcdiag /s:dc1 /e /q — all controllers of the forest, errors only. Without /s: on a workstation dcdiag answers "is not a Directory Server. Must specify /s:<Directory Server>…": by default it tests the local machine. If the output isn't empty, read which test failed on which controller, and remember that not every failed is a fault (more on that below).

2. repadmin /replsummary — replication across the forest in one table: the largest delta and the number of failures for each controller as a source and as a destination. Then repadmin /showrepl <controller> for those with errors, with the code.

PS C:\> repadmin /replsummary
Replication Summary Start Time: 2026-10-29 07:44:36

Beginning data collection for replication summary, this may take awhile:
  .......


Source DSA          largest delta    fails/total %%   error
 DC1                       11m:17s    0 /  10    0
 DC2                       10m:17s    0 /  10    0
 DC3                       13m:31s    0 /  15    0
 DC4                       10m:17s    0 /   5    0


Destination DSA     largest delta    fails/total %%   error
 DC1                       03m:31s    0 /  10    0
 DC2                       11m:17s    0 /  10    0
 DC3                       10m:17s    0 /  15    0
 DC4                       13m:31s    0 /   5    0

largest delta is how long ago the most lagging link last replicated successfully; minutes within a site and no more than the site link interval between sites (180 minutes by default) are normal. fails/total is failed links out of all links; DC3 has 15 in both directions because it is the bridgehead of the Branch site and is linked to DC1, DC2 and DC4 for five partitions.

3. The logs of the controllers: Directory Service, DNS Server, DFS Replication, System for the last day, errors and warnings only, grouped by source — the template from part 1, run on the controllers through Invoke-Command (a ready version is at the end of the part).

4. w32tm /monitor — time, because half of all authentication and replication errors are time.

5. nltest on the client that complains: does it find a controller, is the secure channel intact.

Then follow whatever turned up.

dcdiag

The tests we saw green in part 4 (dcdiag /? lists 29 of them), and the switches that make the utility a tool rather than a ritual:

Switch	What
`/s:dc2`	test the specified controller; without it the local machine, so on a workstation it is required
`/e`	all controllers of the forest (`/a` — all controllers of the site)
`/q`	print errors only
`/v`	verbose: what exactly was checked and what came back
`/c`	all tests, including those not run by default (DNS, Topology, CutoffServers, OutboundSecureChannels), except DcPromo and RegisterInDNS
`/test:X`, `/skip:X`	only this test, or everything but it; the switches can be repeated
`/f:file`	output to a file
`/fix`	affects only the MachineAccount test: fixes the SPNs on the controller's account
`/test:dns /DnsBasic`, `/DnsForwarders`, `/DnsDelegation`, `/DnsDynamicUpdate`, `/DnsRecordRegistration`, `/DnsResolveExtName`	the DNS subtests one by one; without a subtest all of them run except `/DnsResolveExtName`
`/test:CheckSecurityError /ReplSource:dc1`	security checks that get in the way of replication; with `/ReplSource`, also the link to a specific partner
`/test:CutoffServers`	controllers nobody can replicate with; according to the documentation the test calls `DsReplicaSyncAll` and thereby triggers replication
`/test:Topology`, `/test:CrossRefValidation`, `/test:VerifyEnterpriseReferences`	connectivity of the KCC topology (analysis without replication), partition objects, references to controllers from the configuration

The first run from a workstation usually looks like this:

PS C:\> dcdiag /s:dc1 /e /q | Select-String 'failed test'

         ......................... DC1 failed test DFSREvent
         ......................... DC1 failed test KccEvent
         ......................... DC1 failed test SystemLog
         ......................... DC3 failed test DFSREvent
         ......................... DC3 failed test KccEvent
         ......................... DC3 failed test Services
         ......................... DC3 failed test SystemLog
         ......................... DC4 failed test DFSREvent
         ......................... DC4 failed test KccEvent
         ......................... DC4 failed test SystemLog

Ten "failures", and none of them means a directory fault. The DFSREvent, KccEvent and SystemLog tests (and FrsEvent) read the controllers' logs over the EventLog Remoting Protocol on top of RPC, and the Remote Event Log Management firewall rules are disabled on a server by default (part 14): above each such line stands "could not be queried, error 0x6ba "The RPC server is unavailable."". Microsoft describes this as expected behaviour: either enable the three Remote Event Log Management rules on the controllers, or skip these tests and read the logs another way. In this series the logs are read through Invoke-Command, so we skip the tests:

PS C:\> dcdiag /s:dc1 /e /q /skip:SystemLog /skip:DFSREvent /skip:KccEvent
            Invalid service type: DnsCache on DC3, current value
            WIN32_SHARE_PROCESS, expected value WIN32_OWN_PROCESS
         ......................... DC3 failed test Services

One is left, and it is false as well. The Services test checks not only the state of the services but also their process type against a list built into the utility itself. In Windows Server 2025 and Windows 11 the DNS client (DnsCache) runs in its own process, in 2019 in a shared svchost, and dcdiag on a Windows 11 workstation treats that as an error on the 2019 controller. The same test run on dc3 itself with its own version of dcdiag passes:

PS C:\> Invoke-Command dc3 { dcdiag /test:services } | Select-String 'Services'
      Starting test: Services
         ......................... DC3 passed test Services

Microsoft describes the same story for the RpcSs service and Windows Server 2003 and advises against changing the service type on the word of a dcdiag from another version. The general rule: in a forest with controllers of different versions, results of tests tied to the OS version are checked with the utility of that same controller.

When a test fails for real, what matters is not the failed line but what stands above it. For the Replications test the Microsoft documentation (the article on error −2146893022) gives this template:

[Replications Check,<DC Name>] A recent replication attempt failed:
From <source DC> to <destination DC>
Naming Context: <DN path of directory partition>
The replication generated an error (-2146893022):
The target principal name is incorrect.
The failure occurred at <date> <time>.
The last success occurred at <date> <time>.
<X> failures have occurred since the last success.

The code, the direction, the partition, the time of the last success, the number of failures — enough to tell which link is broken and since when. A healthy controller prints only the passed line here:

PS C:\> dcdiag /s:dc3 /test:replications

Directory Server Diagnosis

Performing initial setup:
   * Identified AD Forest.
   Done gathering initial info.

Doing initial required tests

   Testing server: Branch\DC3
      Starting test: Connectivity
         ......................... DC3 passed test Connectivity

Doing primary tests

   Testing server: Branch\DC3
      Starting test: Replications
         ......................... DC3 passed test Replications


   Running partition tests on : ForestDnsZones

   Running partition tests on : DomainDnsZones

   Running partition tests on : Schema

   Running partition tests on : Configuration

   Running partition tests on : corp

   Running enterprise tests on : corp.example.com

The Replications test fails if the last replication attempt over a link ended with an error, if replication is disabled, or if latency exceeds 12 hours.

CheckSecurityError is a test worth knowing separately: it doesn't run by default and checks what repadmin shows as a single number like 8453 "Replication access was denied". Without a partner it looks for a KDC of the domain, checks that the controller's object has replicated, the flags and the minimal set of SPNs of its account, security errors in the replication state, and whether the packet size of Kerberos over UDP gets in the way. With /ReplSource:dc1 it adds the time skew with the partner (more than 300 seconds and Kerberos refuses), permissions on the partitions, access to SYSVOL and NETLOGON, and the version of the controller's object on the partner.

PS C:\> dcdiag /s:dc3 /test:CheckSecurityError /v | Select-String 'Found KDC|Checking machine account|security related|passed test CheckSecurityError'
         Found KDC DC3 for domain corp.example.com in site Branch
         Checking machine account for DC DC3 on DC DC3.
         [DC3] No security related replication errors were found on this DC!
         ......................... DC3 passed test CheckSecurityError

repadmin

Four commands from part 10 plus the ones needed for analysis:

PS C:\> repadmin /showrepl * /csv | ConvertFrom-Csv | Where-Object 'Number of Failures' -gt 0 |
    Format-Table 'Destination DSA', 'Source DSA', 'Naming Context', 'Number of Failures', 'Last Failure Status', 'Last Success Time'

/showrepl * /csv gives all inbound links of all controllers as one table, and PowerShell takes it from there; this is the command people put into a scheduled task that sends mail. In a healthy forest this filter prints nothing. What the rows look like without the filter:

PS C:\> repadmin /showrepl * /csv | ConvertFrom-Csv | Select-Object -First 3 |
    Format-Table 'Destination DSA', 'Source DSA', 'Naming Context', 'Number of Failures', 'Last Success Time'

Destination DSA Source DSA Naming Context                             Number of Failures Last Success Time
--------------- ---------- --------------                             ------------------ -----------------
DC2             DC3        DC=corp,DC=example,DC=com                  0                  2026-10-29 07:33:19
DC2             DC1        DC=corp,DC=example,DC=com                  0                  2026-10-29 07:36:32
DC2             DC1        CN=Configuration,DC=corp,DC=example,DC=com 0                  2026-10-29 07:33:19

All CSV fields are strings, which is why the numbers are left-aligned. The same from the ActiveDirectory module: Get-ADReplicationFailure and Get-ADReplicationPartnerMetadata. With -Scope Forest both need -Target with the forest name; without it the second one answers "The format of the specified domain name is invalid."; Get-ADReplicationPartnerMetadata shows only the domain partition by default, all partitions with -Partition *.

PS C:\> Get-ADReplicationFailure -Target corp.example.com -Scope Forest
PS C:\> Get-ADReplicationPartnerMetadata -Target corp.example.com -Scope Forest |
    Format-Table Server, @{n='Partner'; e={($_.Partner -split ',')[1] -replace 'CN='}}, LastReplicationResult, ConsecutiveReplicationFailures, LastReplicationSuccess

Server               Partner LastReplicationResult ConsecutiveReplicationFailures LastReplicationSuccess
------               ------- --------------------- ------------------------------ ----------------------
dc2.corp.example.com DC1                         0                              0 29.10.2026 7:36:32
dc2.corp.example.com DC3                         0                              0 29.10.2026 7:33:19
dc1.corp.example.com DC2                         0                              0 29.10.2026 7:37:43
dc1.corp.example.com DC3                         0                              0 29.10.2026 7:26:05
dc3.corp.example.com DC4                         0                              0 29.10.2026 7:38:18
dc3.corp.example.com DC1                         0                              0 29.10.2026 7:34:20
dc3.corp.example.com DC2                         0                              0 29.10.2026 7:34:19
dc4.corp.example.com DC3                         0                              0 29.10.2026 7:37:45

Partner is the DN of the partner's NTDS Settings object; its second component is the server name.

repadmin /queue dc1 — the inbound replication queue: what the controller still has to request from its partners. In a healthy forest it's Queue contains 0 items.; a growing queue means the controller can't keep up (too many partners, too many changes or too little bandwidth).
repadmin /showutdvec dc1 DC=corp,DC=example,DC=com — the up-to-dateness vector: up to which USN of each controller dc1 has caught up, including long-removed controllers and former invocationIDs (they are marked (retired) or shown as a GUID). With /latency the rows are sorted from the most lagging.
repadmin /showchanges dc1 <objectGUID of dc2's NTDS Settings> DC=corp,DC=example,DC=com — what dc1 hasn't yet received from dc2. The source is given not by name but by the GUID of its NTDS Settings object.
repadmin /latency — replication latency from ISTG time stamps; according to the utility's own help these stamps aren't updated in forests at the Windows Server 2003 level or higher, and repadmin /showutdvec … /latency is needed instead. In our forest /latency shows stamps that are years old.
repadmin /showsig dc1 — the current invocationID of the controller and the list of former, "retired" ones: each entry is a restore from backup or a rollback noticed through VM-GenerationID.
repadmin /istg and /bridgeheads — who builds the intersite topology and through whom replication between sites goes.

PS C:\> repadmin /showsig dc1
Office\DC1

Current DSA invocationID: 9b8a7c6d-5e4f-4a3b-8c2d-1e0f9a8b7c6d

No retired signatures.

Logs

Events that reveal a problem before a user puts it into words. The texts below were checked against the message tables of Windows Server 2025 itself (ntdsmsg.dll, netmsg.dll, dfsrress.dll) and against the log manifests.

Directory Service:

Id	What
1864	a summary per partition: how many controllers this one hasn't replicated with for more than a day, a week, a month, two months and the tombstone lifetime
2042	no replication with the source for longer than the tombstone lifetime, replication with it has been stopped (error 8614 in `repadmin`)
1988	in strict mode: a partner sent a change to an object that was deleted here and already garbage-collected — a lingering object; replication of the partition from that source is blocked; 1388 — the same in loose mode, the object is accepted
2095	the controller detected a USN rollback: a partner has already seen changes with these USNs
2103	the database was restored with an unsupported procedure, Netlogon is paused
1311, 1865, 1308	KCC: no connected topology can be built; the list of unreachable sites; repeated failures with a partner, the KCC builds a temporary connection around it
2087, 2088	the DNS name of the source controller couldn't be resolved: 2087 — replication failed, 2088 — it succeeded through the NetBIOS or fully qualified name, but DNS is broken
1925, 1232	a replication link to the source couldn't be established; an RPC call to the partner timed out and was cancelled
1084	received changes couldn't be applied to an object (with the object GUID and the source), replication from the source is stopped until it's fixed
2091, 2092	FSMO role: the owner is deleted or the role is not validated (part 17)
1644	an expensive LDAP query (only with diagnostics enabled)
2886, 2887, 2889	insecure LDAP binds: 2886 and 2887 — daily summaries, 2889 — per client, only with the "16 LDAP Interface Events" diagnostics set to 2 (part 5)
2089	the partition hasn't been backed up for a long time; logged daily until there is a backup (the subject of part 20)

DNS Server (part 7): 4013 — the server is waiting for the directory to signal that initial synchronization is complete (normal after a controller reboot, bad if it lasts), 4015 — a critical directory error, 4000 — the directory couldn't be opened, the zone isn't loaded, 7062 — the server received a packet addressed to itself (forwarding to itself).

DFS Replication: 2213 — after an unclean shutdown the DFSR database wasn't closed cleanly, auto recovery is disabled, replication on the volume is stopped until the ResumeReplication method is called (the command is in the event text); 2212 — an unexpected shutdown was detected, the service recovers on its own, 2214 — it has recovered; 2202 — an NTFS change journal wrap (USN journal wrap); 4012 — the folder hasn't replicated for longer than MaxOfflineTimeInDays (60 days), the data is considered stale, replication is stopped; 4614, 4604 and 4602 — SYSVOL initialization (part 10); 5002 and 5014 — an error communicating with a partner, the service will retry (a pair of them after every reboot of a neighbour is normal).

System on controllers and domain members:

Id	Source	What
5719	NETLOGON	a secure session with a domain controller couldn't be set up; on a domain member — DNS or the network, on a controller right after boot — the service start order, harmless if it's a single one
3210	NETLOGON (on a domain member)	the computer couldn't authenticate with a controller: the computer account password wasn't accepted, or there's another machine with the same name on the network
5805, 5722	NETLOGON (on a controller)	the session setup from the named computer failed to authenticate; 5722 also names the account
5774, 5781	NETLOGON	the controller's DNS records couldn't be registered: the server refused (with a response code) or is unreachable
5807	NETLOGON	over the past hours there were clients from subnets not mapped to any site; names and addresses are in `netlogon.log`
4	Security-Kerberos	`KRB_AP_ERR_MODIFIED`: the server couldn't decrypt the ticket — the SPN is registered on the wrong account (part 6)
40960, 40961	LsaSrv	an authentication error for a server with the protocol's failure code; a secured connection couldn't be established — no suitable protocol
1054, 1058, 1030	GroupPolicy	policies not applied: no controller name obtained; a policy file couldn't be read from SYSVOL; settings couldn't be retrieved
129, 50, 36	Time-Service	no domain time source found; the time difference exceeds the threshold, the service no longer considers itself synchronized; time hasn't been synchronized for a long while (part 12)

Security on a controller: 4740 — an account was locked out, with the name of the computer the bad passwords came from (Caller Computer Name); 4771 — Kerberos pre-authentication failed, with the client address; 4776 — credential validation over NTLM. 4740 is logged by default (User Account Management auditing, success), while 4771 and 4776 failures are logged only if the audit policy enables failures in the Kerberos Authentication Service and Credential Validation subcategories: Windows Server doesn't log them by default, Microsoft's baseline recommendations for controllers turn them on. On a busy controller the Security log wraps within a day or two, so the source of a lockout has to be looked for right away.

nltest and netlogon.log

nltest is the Netlogon utility, and it answers questions nothing else answers:

PS C:\> nltest /sc_query:corp.example.com
Flags: 30 HAS_IP  HAS_TIMESERV  Authentication Service: Netlogon
Trusted DC Name \\dc1.corp.example.com
Trusted DC Connection Status Status = 0 0x0 NERR_Success
The command completed successfully

/sc_query shows which controller the machine has its secure channel with and in what state it is; it's the only one of the three switches that changes nothing. /sc_verify:corp.example.com checks the channel again and, if it doesn't work, according to the documentation removes it and builds a new one; /sc_reset:corp.example.com\dc2 rebuilds the channel to a specific controller. The PowerShell counterpart is Test-ComputerSecureChannel; PowerShell 7 doesn't have it in its own Microsoft.PowerShell.Management module, and it arrives through the compatibility layer (a proxy module version 1.0):

PS C:\> Test-ComputerSecureChannel
True

For a domain member with "trust relationship failed" this is the diagnosis, and the cure is Test-ComputerSecureChannel -Repair from part 4. Replication error −2146893022 "The target principal name is incorrect" between controllers most often means the KDC that issued the destination its ticket has an old version of the source's account password. Microsoft advises first disabling the KDC on the destination for a while and restarting it, so that another controller issues the ticket, and if that doesn't help — netdom resetpwd /server:<controller to send the password change to> /userd:CORP\alice.admin /passwordd:* on the console of the controller that needs a new password.

nltest /dsgetdc and /dclist come from part 3. On the controller itself there is also nltest /dsquerydns — the result of the last registration of its DNS records:

PS C:\> Invoke-Command dc1 { nltest /dsquerydns }
Flags: 0
Connection Status = 0 0x0 NERR_Success
There was no failure in the last update for all DC-specific DNS records
The command completed successfully

Which records the controller must register, Netlogon writes to %SystemRoot%\System32\config\netlogon.dns. Checking them against DNS takes a few lines of PowerShell:

PS C:\> $lines = Invoke-Command dc1 { Get-Content C:\Windows\System32\config\netlogon.dns }
PS C:\> $lines | Where-Object { $_ -match ' SRV ' } | ForEach-Object {
    $f = $_ -split ' '
    $found = Resolve-DnsName $f[0] -Type SRV -DnsOnly -ErrorAction SilentlyContinue | Where-Object NameTarget -eq $f[7].TrimEnd('.')
    [pscustomobject]@{ Record = $f[0]; Port = $f[6]; InDns = [bool]$found }
} | Format-Table

Record                                                                           Port InDns
------                                                                           ---- -----
_ldap._tcp.corp.example.com.                                                     389   True
_ldap._tcp.Office._sites.corp.example.com.                                       389   True
_ldap._tcp.gc._msdcs.corp.example.com.                                           3268  True
_ldap._tcp.Office._sites.gc._msdcs.corp.example.com.                             3268  True
_ldap._tcp.7c1d2e3f-4a5b-4c6d-8e7f-9a0b1c2d3e4f.domains._msdcs.corp.example.com. 389   True
_kerberos._tcp.dc._msdcs.corp.example.com.                                       88    True
_kerberos._tcp.Office._sites.dc._msdcs.corp.example.com.                         88    True
_ldap._tcp.dc._msdcs.corp.example.com.                                           389   True
_ldap._tcp.Office._sites.dc._msdcs.corp.example.com.                             389   True
_kerberos._tcp.corp.example.com.                                                 88    True
_kerberos._tcp.Office._sites.corp.example.com.                                   88    True
_gc._tcp.corp.example.com.                                                       3268  True
_gc._tcp.Office._sites.corp.example.com.                                         3268  True
_kerberos._udp.corp.example.com.                                                 88    True
_kpasswd._tcp.corp.example.com.                                                  464   True
_kpasswd._udp.corp.example.com.                                                  464   True
_ldap._tcp.DomainDnsZones.corp.example.com.                                      389   True
_ldap._tcp.ForestDnsZones.corp.example.com.                                      389   True
_ldap._tcp.Office._sites.DomainDnsZones.corp.example.com.                        389   True
_ldap._tcp.Office._sites.ForestDnsZones.corp.example.com.                        389   True
_ldap._tcp.pdc._msdcs.corp.example.com.                                          389   True

Twenty-one SRV records for a controller that is both a GC and the PDC (the others have no pdc record, and a controller without the GC has no gc and _gc records), plus A and AAAA records for the domain name, gc._msdcs, DomainDnsZones and ForestDnsZones, and the CNAME <GUID>._msdcs discussed below. A row with False is a record Netlogon failed to register; look for event 5774 or 5781 next to it. nltest /dsregdns on the controller starts re-registration (part 4); otherwise Netlogon repeats it on its own every hour.

When the Locator finds the wrong controller, or none, or the client ends up in the wrong site, you need the Netlogon log — %windir%\debug\netlogon.log. On a controller it is kept even without debugging, and clients from subnets without a site settle there — the same ones event 5807 reports as a summary:

PS C:\> Invoke-Command dc1 { Get-Content C:\Windows\debug\netlogon.log -Tail 3 }
10/28 12:13:24 [3188] CORP: NO_CLIENT_SITE: LAPTOP-7 192.168.9.44
10/28 13:53:25 [3188] CORP: NO_CLIENT_SITE: LAPTOP-7 192.168.9.44
10/28 15:52:26 [3288] CORP: NO_CLIENT_SITE: LAPTOP-7 192.168.9.44

Date, time, thread number, domain, client name and address: the subnet 192.168.9.0/24 isn't mapped to any site, and this client may get a controller on the other side of the country. The cure is a subnet object in CN=Subnets (part 10).

The detailed Locator log on a client is enabled with nltest /dbflag:0x2080ffff; since Windows Server 2012 R2 and Windows 10 Netlogon usually doesn't need a restart, it's enough to make sure new lines appear in the log. It shows the controller lookups and the DNS queries Netlogon made along the way. The log grows to 20 MB, then is renamed to netlogon.bak and started anew; nltest /dbflag:0x0 turns it off.

Typical causes

Six states the domain gets into most often, with symptoms in the order they show up.

DNS. repadmin — 8524 "The DSA operation is unable to proceed because of a DNS lookup failure"; Directory Service — 2087, 2088; dcdiag — Connectivity fails. The destination looks the source up not by name but by the CNAME <objectGUID of NTDS Settings>._msdcs.<forest root>, so that's what has to be checked — on the sick controller, against the DNS server it uses:

PS C:\> $guid = (Get-ADObject (Get-ADDomainController dc1).NTDSSettingsObjectDN).ObjectGUID
PS C:\> Resolve-DnsName "$guid._msdcs.corp.example.com" -Type CNAME

Name                                                         Type  TTL Section NameHost
----                                                         ----  --- ------- --------
2f3e4d5c-6b7a-4899-8a7b-6c5d4e3f2a1b._msdcs.corp.example.com CNAME 600 Answer  dc1.corp.example.com

Then the A record of dc1, the DNS client of the controller (the server order from part 10), the netlogon.dns check above and dcdiag /test:dns with subtests. The most common cause of all.

Time. Clients — logon errors for everyone at once in one site; repadmin — 8453 or −2146893022; System — 40960 with a Kerberos code; dcdiag /test:CheckSecurityError /ReplSource:… names the skew in words. Check w32tm /monitor; correct with w32tm /resync /force and find out where it came from: a virtual machine snapshot, the motherboard battery, a PDC without an external source (part 12).

Secure channel. On a domain member — "trust relationship failed", System 5719 and 3210; on a controller — 5805 and 5722 with the machine name. The cause is almost always a virtual machine rolled back to a snapshot or restored from an image taken before the latest change of the computer account password (the machine changes it itself, every 30 days by default): the machine knows the old password. Test-ComputerSecureChannel -Repair.

RPC and the firewall. repadmin — 1722 "The RPC server is unavailable" or 1753 "There are no more endpoints available from the endpoint mapper"; dcdiag — Replications and Connectivity fail while DNS works. Check the endpoint mapper port:

PS C:\> Test-NetConnection dc2 -Port 135 -InformationLevel Quiet
True

Then the dynamic ports: portqry -n dc2 -e 135 (PortQry is a separate download from Microsoft) lists the RPC interfaces and the ports they listen on; between sites the replication ports are pinned and opened, part 10. Error 1753 also comes when the name resolved to the address of a machine that isn't a controller.

SYSVOL. Policies apply on some machines and not on others; GPMC shows different versions in the directory and in SYSVOL; on domain members — GroupPolicy 1058; on a controller — DFS Replication 2213 or 4012; dcdiag — SysVolCheck and Advertising fail, the controller doesn't advertise itself to clients. The state of the SYSVOL replicated folder on all controllers is one WMI query on each:

PS C:\> $dcs = (Get-ADDomainController -Filter *).HostName
PS C:\> Invoke-Command $dcs {
    Get-CimInstance -Namespace root\microsoftdfs -ClassName DfsrReplicatedFolderInfo -Filter "ReplicatedFolderName='SYSVOL Share'"
} | Sort-Object PSComputerName | Format-Table PSComputerName, ReplicatedFolderName, State

PSComputerName       ReplicatedFolderName State
--------------       -------------------- -----
dc1.corp.example.com SYSVOL Share             4
dc2.corp.example.com SYSVOL Share             4
dc3.corp.example.com SYSVOL Share             4
dc4.corp.example.com SYSVOL Share             4

State: 0 — uninitialized, 1 — initialized, 2 — initial sync, 3 — auto recovery, 4 — normal, 5 — in error. The backlog between a pair of controllers:

PS C:\> dfsrdiag backlog /rgname:"Domain System Volume" /rfname:"SYSVOL Share" /smem:dc1 /rmem:dc2

No Backlog - member <dc2> is in sync with partner <dc1>

Operation Succeeded

The Get-DfsrBacklog cmdlet gives the same (in PowerShell 7 the DFSR module loads through the compatibility layer; with no backlog it prints nothing). After 2213 — ResumeReplication with the command from the event, there's no need to delete the DFSR database; after 4012 or diverged content — a non-authoritative synchronization of SYSVOL, the subject of part 20.

Lockouts. A user gets locked out every five minutes without typing a password. The source is wherever the old password stayed: a saved network drive mapping, a scheduled task, a service, a phone with mail, an RDP session left a week ago. A controller rechecks a wrong password with the PDC Emulator, lockouts converge there too (part 17), so 4740 is easiest to look for on it:

PS C:\> Invoke-Command (Get-ADDomain).PDCEmulator {
    Get-WinEvent -FilterHashtable @{LogName='Security'; Id=4740; StartTime=(Get-Date).AddDays(-1)} -ErrorAction SilentlyContinue |
        Select-Object TimeCreated, @{n='User'; e={$_.Properties[0].Value}}, @{n='From'; e={$_.Properties[1].Value}}
} | Format-Table TimeCreated, User, From

In a domain without lockouts the command prints nothing. When there are lockouts, the User column holds the locked account (the Account Name field) and From the Caller Computer Name, the machine the bad passwords come from; on it, Get-ScheduledTask | Where-Object { $_.Principal.UserId -like '*carol*' }, Get-CimInstance Win32_Service -Filter "StartName like '%carol%'", cmdkey /list and query session. If From is empty, the attempts came through a server that didn't pass on the client name (web, VPN, mail), and then you need events 4771 on all controllers with the address in the Client Address field (Properties[6]) — with failure auditing enabled, see above.

Lingering objects

An object was deleted, became a tombstone, lived out its term (tombstoneLifetime, 180 days) and was garbage-collected on all controllers. Except one: dc3 was off the whole time and still holds the object alive — it never even received the notice of deletion. Such an object is called lingering. When dc3 comes back and the object changes on it, a partner receives a change to an object it doesn't have. What it does is decided by the Strict Replication Consistency value in HKLM\SYSTEM\CurrentControlSet\Services\NTDS\Parameters on the destination. According to the documentation it is 1 on controllers installed into a new forest; in forests upgraded from Windows 2000, new controllers get 0 until a special object CN=94fdebc6-8eeb-4640-80de-ec52b9ca17fa,CN=Operations,CN=ForestUpdates,… is added to the forest configuration.

strict mode (1): the destination refuses the change, logs event 1988 with the object name, its GUID and the source address in the form <GUID>._msdcs.<forest>, and replication of that partition from that source is blocked (error 8606 "Insufficient attributes were given to create an object");
loose mode (0): the destination requests the whole object and recreates it (event 1388), and the deleted user comes back to life across the forest — with its SID, group memberships and a half-year-old password.

The first outcome is the right one, and event 1988 is the instruction: it contains everything needed for removal. repadmin /removelingeringobjects does the removal: you give it the controller to clean, the GUID of the NTDS Settings object of a reference controller with a writable copy of the partition, and the partition. First in report mode:

repadmin /removelingeringobjects dc3.corp.example.com 2f3e4d5c-6b7a-4899-8a7b-6c5d4e3f2a1b DC=corp,DC=example,DC=com /advisory_mode

The result is in the Directory Service log of dc3: 1938 at the start of the check, 1946 for each object found and 1942 with a summary. Once the list is as expected — the same command without /advisory_mode: then events 1937, 1945 for each removed object and 1939. Repeat for each partition (configuration, schema, DNS partitions) and for each controller that may have lingering objects, global catalogs included. Microsoft prefers the Lingering Object Liquidator v2 tool for this, which goes through all pairs at once; after event 1388 (the object already accepted) neither it nor repadmin will remove the object — another procedure from the same documentation is needed.

The mode on all controllers at once:

PS C:\> Invoke-Command $dcs {
    Get-ItemProperty 'HKLM:\SYSTEM\CurrentControlSet\Services\NTDS\Parameters' | Select-Object 'Strict Replication Consistency'
} | Sort-Object PSComputerName | Format-Table PSComputerName, 'Strict Replication Consistency'

PSComputerName       Strict Replication Consistency
--------------       ------------------------------
dc1.corp.example.com                              1
dc2.corp.example.com                              1
dc3.corp.example.com                              1
dc4.corp.example.com                              1

Where the value is 0 or missing, repadmin /regkey <controller> +strict turns it on.

USN rollback

The directory of controller dc2 was brought back from a week-old virtual machine snapshot. Its USN rolled back: yesterday highestCommittedUSN was 13,400, today it's 12,900 again. In their up-to-dateness vectors the partners remember that from dc2 with this invocationID they have seen everything up to 13,400. Now dc2 creates new objects with USNs 12,901, 12,902 — and when asked "give me everything after 13,400" it never hands them over. Users created on dc2 after the rollback exist only there; passwords changed through it work only on it. This is a USN rollback, and its main danger is that it's silent: replication formally succeeds, there are no errors, and the directory has diverged.

The controller can detect it. When a partner requests changes and its vector shows it has already received from this controller USNs that the controller now hands out again under the same invocationID, the controller logs 2095 in Directory Service, pauses Netlogon so that no more changes that will never leave are created on it, and sets Dsa Not Writable = 4 in the registry — a quarantine that the documentation forbids lifting by hand. Event 2103 is "The Active Directory Domain Services database has been restored using an unsupported restoration procedure" and the paused Netlogon. The events themselves may get overwritten, the registry value stays. There are three ways out: forcibly demote the controller, clean up the metadata and promote it again (part 19), restore the system state from a proper backup, during which the directory changes its invocationID by itself (part 20), or make the directory take a new invocationID following the procedure for snapshots without a backup.

Why a restore from backup doesn't cause a rollback: the directory gets a new invocationID, and for the partners it's a new source — the up-to-dateness vector for the old invocationID stays, one for the new one is created, and everything the new source hands out will be requested. Reverting a snapshot doesn't change the invocationID, and that's the whole difference.

Since Windows Server 2012 there's a safeguard — VM-GenerationID: a hypervisor that supports it (Hyper-V since Windows Server 2012, and others) changes this identifier inside the virtual machine when a snapshot is applied and when a copy of the machine is imported. The controller keeps its value in the msDS-GenerationId attribute of its own account and compares it with the current one at boot and before committing every change. If they differ — a new invocationID, the current RID pool discarded, then non-authoritative inbound replication of changes and a non-authoritative synchronization of SYSVOL; the log shows event 2170 "A Generation ID change has been detected". Such a rollback is safe for the directory and SYSVOL but not for everything else: files, application databases and settings on the server itself, anything that doesn't come back through replication, still jump into the past. Controller snapshots remain a way to bring a machine back within the hour after a failed update, not a way to back up.

A controller that missed half a year

The third scenario is related to the first two. dc3 in the branch hasn't replicated for 200 days: the link was down and nobody looked. All that time event 1864 on the other controllers showed a growing count of "more than a month", "more than two months", and once 180 days had passed — "more than a tombstone lifetime". When dc3 comes back, replication between it and its partners stops with event 2042 "It has been too long since this machine last replicated with the named source machine" and error 8614 in repadmin. The block protects against lingering objects, which dc3 now almost certainly has plenty of.

There is one right way out: don't bring dc3 back. Forcibly demote it, clean up the metadata, reinstall and promote it again — all of that is in the next part. The documented workaround — find and remove the lingering objects (Lingering Object Liquidator or repadmin /removelingeringobjects for all partitions), temporarily set Allow Replication With Divergent and Corrupt Partner = 1 on the destination (repadmin /regkey <controller> +allowDivergent), let replication go through and revert the value (-allowDivergent). It exists for cases where reinstalling is impossible and takes longer than reinstalling.

The tombstone lifetime everything is counted from:

PS C:\> (Get-ADObject 'CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,DC=corp,DC=example,DC=com' -Properties tombstoneLifetime).tombstoneLifetime
180

An empty value here means 60 days: 180 is set when a forest is created starting with Windows Server 2003 SP1, while in forests created earlier and never fixed by hand the attribute is empty. They should set 180 explicitly. With the AD Recycle Bin enabled, msDS-DeletedObjectLifetime is added — the period during which a deleted object can still be restored in full (the subject of part 20).

Hands-on

All forest diagnostics in one script, worth running on a schedule:

$dcs = (Get-ADDomainController -Filter *).HostName
$report = [ordered]@{}
$report.dcdiag = dcdiag /s:$((Get-ADDomain).PDCEmulator) /e /q /skip:SystemLog /skip:DFSREvent /skip:KccEvent 2>&1 | Out-String
$report.repl   = repadmin /showrepl * /csv | ConvertFrom-Csv | Where-Object 'Number of Failures' -gt 0 | Out-String
$report.time   = w32tm /monitor 2>&1 | Select-String 'offset' | Out-String
$report.events = Invoke-Command $dcs {
    Get-WinEvent -FilterHashtable @{LogName='Directory Service','DNS Server','DFS Replication'; Level=1,2,3; StartTime=(Get-Date).AddDays(-1)} -ErrorAction SilentlyContinue |
        Select-Object LogName, Id
} | Group-Object PSComputerName, LogName, Id | Select-Object Count, Name | Out-String
$report.sysvol = Invoke-Command $dcs {
    Get-CimInstance -Namespace root\microsoftdfs -ClassName DfsrReplicatedFolderInfo -Filter "ReplicatedFolderName='SYSVOL Share'"
} | Select-Object PSComputerName, State | Out-String
$report | Format-List

The logs are read on the controllers themselves through Invoke-Command, not with Get-WinEvent -ComputerName: that one goes over the same RPC as the dcdiag tests and hits the same firewall. The report in our forest:

Name  : dcdiag
Value :             Invalid service type: DnsCache on DC3, current value

                    WIN32_SHARE_PROCESS, expected value WIN32_OWN_PROCESS

                 ......................... DC3 failed test Services



Name  : repl
Value :

Name  : time
Value :
            NTP: -0.0002433s offset from dc1.corp.example.com
            NTP: +0.0000000s offset from dc1.corp.example.com
            NTP: -0.0008230s offset from dc1.corp.example.com
            NTP: -0.0007763s offset from dc1.corp.example.com



Name  : events
Value :
        Count Name
        ----- ----
            2 dc1.corp.example.com, Directory Service, 3079
            5 dc2.corp.example.com, Directory Service, 2089
            1 dc3.corp.example.com, Directory Service, 3041
            5 dc4.corp.example.com, Directory Service, 2089
            3 dc4.corp.example.com, Directory Service, 3079



Name  : sysvol
Value :
        PSComputerName       State
        --------------       -----
        dc3.corp.example.com     4
        dc1.corp.example.com     4
        dc2.corp.example.com     4
        dc4.corp.example.com     4

This is what a healthy forest looks like: in dcdiag only the known false failure of the Services test on the 2019 controller, repl is empty, time offsets are fractions of a millisecond (one line per controller, each relative to the PDC dc1, zero for dc1 itself), SYSVOL is in state 4 everywhere. The logs have warnings, not errors: 2089 on dc2 and dc4 — the partitions haven't been backed up for a long time (one event for each of the five partitions), 3079 — a Windows Server 2025 controller withheld confidential attributes from a client reading them over unencrypted LDAP, 3041 — advice to require channel binding for LDAPS. None of this breaks the domain now, but the first is a direct reason for part 20.

What a lingering object looks like before it's found: on one controller the object exists, on another there's nothing at all — neither the live object nor a tombstone. repadmin /showobjmeta with the object GUID answers like this on the second one:

PS C:\> repadmin /showobjmeta dc1 "<GUID=5d4c3b2a-1f0e-4d9c-8b7a-6f5e4d3c2b1a>"
DsReplicaGetInfo() failed with status 8439 (0x20f7):
    The distinguished name specified for this replication operation is invalid.

With a DN instead of a GUID the answer is different — "failed with status 8333 (0x208d): Directory object not found.". On the controller where the object is alive, the same command shows the ordinary metadata of its attributes. An object that exists on one controller and isn't present on another even as a tombstone is the very definition of a lingering object.

A USN rollback before the controller has detected it — the controller's own USN next to what a partner remembers about it:

PS C:\> (Get-ADRootDSE -Server dc2).highestCommittedUSN
53424732
PS C:\> (Get-ADReplicationUpToDatenessVectorTable -Target dc1 -Partition 'DC=corp,DC=example,DC=com' | Where-Object Partner -like '*CN=DC2,*').UsnFilter
53424720

The partner remembers dc2 at a USN no higher than what dc2 has issued itself: all is well (the difference is changes from the last seconds that haven't reached dc1 yet). If the partner's UsnFilter is greater than the controller's own highestCommittedUSN, dc2 is in the past, and at the next change request it will log 2095. The filter on CN=DC2, is needed because the vector also keeps former invocationIDs of the same controller, and their Partner is empty. If the controller has already detected the rollback — in the registry:

PS C:\> Invoke-Command dc2 { Get-ItemProperty 'HKLM:\SYSTEM\CurrentControlSet\Services\NTDS\Parameters' -Name 'Dsa Not Writable' -ErrorAction SilentlyContinue }

A healthy controller has no such value, and the command prints nothing; 4 means "restored with an unsupported method", and from there it's the next part.

And the safeguard against rollback. msDS-GenerationId doesn't replicate, so it has to be read from the controller itself — just like rIDNextRID in part 17:

PS C:\> (Get-ADComputer dc1 -Server dc1 -Properties 'msDS-GenerationId').'msDS-GenerationId'.Length
8
PS C:\> (Get-ADComputer dc1 -Server dc2 -Properties 'msDS-GenerationId').'msDS-GenerationId'.Length
0

At every boot a virtual controller notes in the log that it has compared the identifier:

PS C:\> Invoke-Command dc1 { Get-WinEvent -FilterHashtable @{LogName='Directory Service'; Id=2168} -MaxEvents 2; Get-WinEvent -FilterHashtable @{LogName='Directory Service'; Id=2172} -MaxEvents 2 } |
    Sort-Object TimeCreated | Format-Table TimeCreated, Id, @{n='Message'; e={($_.Message -split "`r?`n")[0]}}

TimeCreated           Id Message
-----------           -- -------
19.10.2026 0:29:06  2168 The DC is running on a supported hypervisor. VM Generation ID is detected.
19.10.2026 0:29:06  2172 Read the msDS-GenerationId attribute of the Domain Controller's computer object.
26.10.2026 13:40:52 2168 The DC is running on a supported hypervisor. VM Generation ID is detected.
26.10.2026 13:40:52 2172 Read the msDS-GenerationId attribute of the Domain Controller's computer object.

2168 — the hypervisor provides a VM-GenerationID, 2172 — the stored value was read; after a rollback to a snapshot, 2170 would appear instead of the usual pair.

Sources

dcdiag — switches, tests and what they check, /fix, /c, DNS subtests, CheckSecurityError, CutoffServers and Topology;
Errors when you run DCDIAG commands — error 0x6ba in the log tests, the Remote Event Log Management rules, the false Services failure on a controller of another version;
Repadmin and Repadmin /showobjmeta — repadmin commands;
Get-ADReplicationFailure, Get-ADReplicationPartnerMetadata, Get-ADReplicationUpToDatenessVectorTable and Get-ADRootDSE — the same from the ActiveDirectory module, -Target and -Scope;
Troubleshooting Active Directory Replication Problems, Guidance for troubleshooting Active Directory replication and Troubleshoot common AD replication errors — the general procedure and error codes;
Replication error 8524, 8453, 1722 and 1753 — DNS, replication permissions, RPC and the endpoint mapper;
Replication error -2146893022 — the Replications test output template, the controller password on the KDC, disabling the KDC and netdom resetpwd, finding the source by the <GUID>._msdcs CNAME;
Event ID 1388 or 1988 and Replication error 8606 — lingering objects, strict and loose mode, the default value, repadmin /removelingeringobjects and /regkey +strict;
Description of the Lingering Object Liquidator tool — a tool for removing lingering objects across all pairs;
Event ID 2042 and Replication error 8614 — a controller that hasn't replicated for longer than the tombstone lifetime, Allow Replication With Divergent and Corrupt Partner;
How to detect and recover from a USN rollback — event 2095, Dsa Not Writable, the paused Netlogon and three ways to recover;
Virtualized Domain Controller Architecture and ms-DS-Generation-Id — VM-GenerationID, when it's compared and what the controller does on a mismatch;
Shelf life of a system-state backup of AD and Tombstone-Lifetime — the tombstone lifetime, 60 and 180 days;
Nltest — /sc_query, /sc_verify, /sc_reset, /dsquerydns, /dbflag;
Enable debug logging for Netlogon service — nltest /dbflag, no restart needed, the size of netlogon.log;
Troubleshoot domain controller location issues and Verify that SRV DNS records have been created — the Locator, netlogon.dns and the controller's SRV records;
Test-ComputerSecureChannel — checking and repairing the secure channel;
Test-NetConnection and Using the PortQry command-line tool — checking ports and the RPC endpoint mapper;
Troubleshoot missing SYSVOL and Netlogon shares and DfsrReplicatedFolderInfo class — states of the SYSVOL folder, events 2213, 4012, 4614, 4604, ResumeReplication, MaxOfflineTimeInDays;
Get-DfsrBacklog — the DFSR backlog from PowerShell;
4740, 4771 and 4776 — fields of the lockout and failed authentication events;
System Audit Policy recommendations — default auditing and the recommended one for controllers;
How to enable LDAP signing — events 2886, 2887 and 2889;
Windows Time service tools and settings — w32tm /monitor and /resync;
Get-WinEvent — -FilterHashtable with LogName and Level arrays.

Now we have an order of checks, we can tell a real dcdiag failure from a false one, and we know three directory states after which a controller isn't repaired but replaced. How that's done is the next part.

Next — part 19: replacing and retiring controllers. Regular demotion with Uninstall-ADDSDomainController and forced demotion when the controller is already dead; metadata cleanup — what a vanished controller leaves in the directory and DNS and how to remove it; moving DHCP and DNS client settings; raising the forest and domain functional levels; and why a new Windows Server version on a controller is always a new controller rather than an upgrade of the old one.