Kubernetes for beginners. Part 40: How do Windows pods join Active Directory with gMSA?
All parts of the series “Kubernetes for beginners”In the previous part ordinary workloads started running on the test cluster's Windows worker: hello is built for Windows, csi-driver-smb mounts file shares on the node, and logs and metrics are collected by the test cluster's Loki and Prometheus. But Windows applications have a need that Linux applications usually do not: they live in an Active Directory domain. An IIS site lets users in with Windows authentication, a service connects to SQL Server with Integrated Security=SSPI instead of a password in its connection string, reports are written to a file share whose permissions are granted to an account rather than to an address. All of this rests on Kerberos and the domain account of the process. And a container does not join the domain. Today: how a Windows pod gets a domain identity — group managed service accounts (gMSA), the credential spec, the GMSACredentialSpec CRD and the webhook that ties them together, the use permission in RBAC, a domain-joined node versus the CCG plugin for a node outside the domain, checks with nltest and klist, and common errors.
How Active Directory works is covered in part 3 of the Windows Server series, the Kerberos protocol in part 6, and DNS in the domain in part 7. The domain is the same — corp.example.com with the NetBIOS name CORP and the domain controller dc1.corp.example.com. In the lab network it sits at 192.168.10.53 — the very DNS server for the corp.example.com zone that served as the stub-zone example in part 16. Two more domain servers are the file server srv1.corp.example.com and sql1.corp.example.com with SQL Server, whose SPN MSSQLSvc/sql1.corp.example.com:1433 is registered on the svc-sql account, as in part 6. The cluster side, like this whole block, is in the test cluster from part 38: kubectl and helm commands go to the lab context, and the Windows node is k8s-lab-win-1 (192.168.10.25).
Why a pod needs a domain account
Integrated Windows Authentication is designed so that the application knows no passwords at all. The process runs in a logon session of a particular account, and LSA — the Windows security subsystem, the lsass.exe process — keeps Kerberos tickets for that session. When the process opens \\srv1.corp.example.com\reports or connects to SQL Server, the SSPI library takes a ticket for the required service from the session cache (or gets one from the domain controller) and embeds it into the application protocol through the SPNEGO negotiation mechanism: client and server choose Kerberos, and if that fails, NTLM. The server decrypts the ticket with its key and learns who has come and which groups they belong to. The AS, TGS and AP exchanges field by field are in part 6 of the Windows Server series.
Hence the requirement: the process must run as an account the domain knows. On an ordinary server this is either the computer account (for services running as Local System or Network Service) or a service account with a password.
The options without gMSA and why they are bad:
- A login and password in a Secret (part 12), with the application authenticating with them explicitly: a SQL login in the connection string,
net usewith a password. The password lives in the cluster in reversible form, is changed by hand in all places at once, and many programs cannot work this way at all — an IIS application pool, for example, keeps the password of its account in its own configuration. - csi-driver-smb volumes (part 24, on Windows nodes — the previous part): the node mounts the share with an account from a Secret. This solves only file access, all pods with the volume run as one account, and the password still sits in the cluster.
- Joining the container to the domain is impossible. Joining a domain creates a computer account in the directory with its own password, which the machine keeps in its registry and changes every 30 days, and a Netlogon secure channel with the domain controller. A container, however, is created from an image, lives for minutes, exists in ten copies and disappears without a trace: every pod would need its own computer account, created before start and deleted afterwards, and its password would somehow have to be delivered inside. Windows containers cannot be joined to a domain in principle.
gMSA removes all three problems: one domain account for many machines, whose password Active Directory itself generates and changes and hands out only to permitted hosts. When starting a container, the host gets the gMSA password and passes it to the container, and on the network the container looks like a domain computer named after the gMSA.
How a gMSA works
Managed service accounts appeared in Windows Server 2008 R2 as MSA (standalone MSA): an account for the services of a single computer, with a password that this computer changes itself. Windows Server 2012 added group ones, gMSA: the same idea, but the account is used by many servers at once — a web farm behind a load balancer, container hosts.
The object in the directory. A gMSA is an object of class msDS-GroupManagedServiceAccount, a subclass of computer. So it has everything a computer account has: a sAMAccountName ending with a dollar sign (gmsa-reports$), the WORKSTATION_TRUST_ACCOUNT flag (4096) in userAccountControl, SPNs, the dNSHostName attribute. The name is limited to 15 characters — the limit of a computer's NetBIOS name. By default such objects are created in the CN=Managed Service Accounts,DC=corp,DC=example,DC=com container.
Nobody sets the password. The domain controller computes it. Every domain controller starting with Windows Server 2012 runs the key distribution service — KDS (Microsoft Key Distribution Service, the kdssvc.dll library). It holds the KDS root key: an object in the Configuration partition (CN=Master Root Keys,CN=Group Key Distribution Service,CN=Services,CN=Configuration,DC=corp,DC=example,DC=com), one per forest and replicated to all domain controllers of the forest. From the root key, the account's SID and the number of the current time interval, the domain controller deterministically derives the gMSA password. So any domain controller computes the same value at any moment, and the password need not be stored, replicated or changed by a command: a new interval begins — a new password results. The interval length is the msDS-ManagedPasswordInterval attribute, 30 days by default; it is set only when the account is created.
Who can get the password. The host reads the password from the msDS-ManagedPassword attribute over LDAP. The attribute is constructed: it is not in the database; the domain controller builds the value at the moment of the request and returns it only to those listed in the msDS-GroupMSAMembership attribute (in PowerShell, the PrincipalsAllowedToRetrieveManagedPassword parameter), and only over an encrypted connection. The reply contains the current password, the previous one and the time until the next change: the previous one is needed so that services on other hosts that have not yet switched to the new password can decrypt tickets. From the password the host derives the Kerberos keys itself — AES salted with the realm and the name, as in part 6.
The attribute is not returned to anyone else, domain administrators included. This is protection by an access list, not magic: a domain administrator can add themselves to msDS-GroupMSAMembership and then read the password, so changes to this attribute are worth tracking with auditing.
dc1.corp.example.com k8s-lab-win-1
┌──────────────────────────────────────┐ (K8S-LAB-WIN-1$ in group k8s-win-nodes)
│ KDS: │ │
│ root key (Configuration │ │
│ partition, one per forest) │ │
│ + SID of gmsa-reports$ │ │
│ + number of the 30-day interval │ │
│ = password of gmsa-reports$ │ │
│ (computed, not stored) │ │
└──────────────────────────────────────┘ │
│ │
│◄──── 1. Kerberos: TGT for K8S-LAB-WIN-1$ ────────┤
│◄──── 2. encrypted LDAP: msDS-ManagedPassword ────┤
│ │
│ 3. is K8S-LAB-WIN-1$ (via group k8s-win-nodes) │
│ in msDS-GroupMSAMembership? │
│ │
├──── 4. yes: current and previous password ──────►│
│ no: the attribute is not returned │
Having got the password, the host can do everything a domain computer with that name can: get a TGT as gmsa-reports$, establish a Netlogon secure channel, decrypt tickets issued for this account's SPNs.
How a pod becomes a gMSA
The chain is long, and both Kubernetes and Windows take part in it:
administrator kube-apiserver
│ kubectl apply: Deployment reports, │
│ windowsOptions.gmsaCredentialSpecName: │
│ gmsa-reports │
└────────────────────────────────────────────►│
│ 1. gMSA mutating webhook:
│ GMSACredentialSpec gmsa-reports
│ → JSON into the pod's gmsaCredentialSpec
│ 2. gMSA validating webhook:
│ SubjectAccessReview: may
│ SA win-demo/reports use
│ gmsa-reports?
▼
etcd ──► scheduler: k8s-lab-win-1
┌─ k8s-lab-win-1 ────────────────────────────────────────────────────────────┐
│ kubelet ── CRI, credential_spec (JSON) ──► containerd ──► hcsshim ──► HCS │
│ │ │
│ 3. HCS: container + CCG instance ◄────────────┘ │
│ │ │
│ 4. CCG: gmsa-reports$ password ◄───┼──── LDAP ──── dc1 │
│ as K8S-LAB-WIN-1$ (domain-joined node) │
│ or as the plugin's account (node outside the domain) │
│ ▼ │
│ 5. container: SYSTEM and NETWORK SERVICE go out as CORP\gmsa-reports$ │
│ ──── Kerberos AS-REQ gmsa-reports$ ────► dc1 (KDC) │
└────────────────────────────────────────────────────────────────────────────┘
- The mutating webhook. The pod manifest gives only the name of the spec — the
securityContext.windowsOptions.gmsaCredentialSpecNamefield. An admission webhook (an external HTTPS service to which kube-apiserver sends an object to modify or validate before writing it; the mechanism is covered in part 28) finds theGMSACredentialSpecobject with that name and writes its content — the spec's JSON — into the neighbouringgmsaCredentialSpecfield. The name field stays. - The validating webhook. The same service checks that the pod's ServiceAccount has the
usepermission on that spec, and that the JSON ingmsaCredentialSpecmatches the object's content — you cannot slip in a spec of your own bypassing the name. - kubelet and containerd. kubelet passes the JSON in the CRI request to create the container (the
credential_specfield in the Windows container's security context), containerd puts it into the OCI spec, and hcsshim — containerd's shim for Windows (part 36) — asks HCS (Host Compute Service, the Windows service that creates containers) to create the container together with an instance of Container Credential Guard (CCG). - CCG is a Windows component (the
ccg.exeprocess) that gets the gMSA password for the container. If the node is domain-joined, it reads the password as the node's computer account; if not, as a domain user whose credentials a plugin gives it (more on that below). The password is handed to the container's security subsystem; it is not in the image, in the cluster API or in the pod's files. - Inside the container the gMSA becomes the account of the "computer" that the container is. Everything that goes onto the network as the computer — processes running as Local System and Network Service, the Netlogon service — authenticates in the domain as
CORP\gmsa-reports$. The container's local usersContainerAdministratorandContainerUserremain local and do not go onto the network as the gMSA.
The last point is the main thing to remember when building an image: the application must run as Network Service or Local System. whoami inside will then show nt authority\network service, not the gMSA name — the process's local identity does not change; the computer's identity on the network does.
Active Directory: the KDS root key and the gMSA
Everything in this section runs on dc1 or on a machine with the ActiveDirectory module, as a domain administrator (or an account delegated the right to create msDS-GroupManagedServiceAccount objects). More on the module is in part 13 of the Windows Server series.
The KDS root key is created once per forest, and first we check whether it already exists. The Get-KdsRootKey cmdlet is not always good for this: in our lab, under a domain administrator account it returned nothing, although the forest does have keys. It is more reliable to look at the key objects themselves in the Configuration partition:
PS C:\> Get-ADObject -SearchBase 'CN=Master Root Keys,CN=Group Key Distribution Service,CN=Services,CN=Configuration,DC=corp,DC=example,DC=com' `
-Filter 'objectClass -eq "msKds-ProvRootKey"' -Properties whenCreated | Select-Object Name, whenCreated
Name whenCreated
---- -----------
feb72f87-81f1-5918-f9a4-99db486b8d34 15.05.2019 22:38:10
1ace02ad-d162-1a3f-dd4d-8818bb898217 20.02.2015 13:59:56
The lab domain is older than the series, and it has two keys — gMSAs work with them, there is no need to create a new one. A new forest has no key, and it is created with Add-KdsRootKey -EffectiveImmediately. Despite the parameter name, the key comes into use only after 10 hours: this time is given to replication, so that the key reaches all domain controllers of the forest. If a domain controller that does not have the key yet receives a gMSA password request, it cannot compute the password. In a learning domain with a single domain controller you can skip the wait by setting the effective time in the past — Add-KdsRootKey -EffectiveTime (Get-Date).AddHours(-10) — but in a domain with several domain controllers you must not do this. An attempt to create a gMSA before that time ends with the error New-ADServiceAccount : Key does not exist — not a typo in the name, but a key that has not taken effect yet.
The node group. The gMSA password will be read by the cluster's Windows nodes. Granting the right to each node separately is inconvenient, so we create a domain local group (why domain local — the AGDLP rule from part 3) and an OU for the cluster nodes, so that group policies meant for ordinary servers do not apply to them:
PS C:\> New-ADOrganizationalUnit -Name Kubernetes -Path 'OU=Servers,DC=corp,DC=example,DC=com'
PS C:\> New-ADGroup -Name k8s-win-nodes -GroupScope DomainLocal -GroupCategory Security `
-Path 'OU=Kubernetes,OU=Servers,DC=corp,DC=example,DC=com' -Description 'Kubernetes Windows nodes: reading gMSA passwords'
The gMSA. One account per application, not one per cluster: all pods allowed to use it get its rights in the domain.
PS C:\> New-ADServiceAccount -Name gmsa-reports -DNSHostName gmsa-reports.corp.example.com `
-ServicePrincipalNames 'host/gmsa-reports', 'host/gmsa-reports.corp.example.com' `
-PrincipalsAllowedToRetrieveManagedPassword k8s-win-nodes `
-KerberosEncryptionType AES128, AES256 -ManagedPasswordIntervalInDays 30
-DNSHostNameis mandatory: a "computer" must have a DNS name, even though nobody creates a DNS record for it.-ServicePrincipalNames— containers do not register SPNs themselves, so at leasthost/is set at creation. If the application in the pod accepts incoming Windows authentication, it will need SPNs for the name clients use to reach it — more on that in the section on names.-KerberosEncryptionType AES128, AES256writes the value 24 tomsDS-SupportedEncryptionTypes: tickets for this account will be AES only. Without an explicit value the domain controller may assume the account supports only RC4, and in a domain where RC4 is disabled (part 6) authentication will fail.
PS C:\> Get-ADServiceAccount gmsa-reports -Properties DNSHostName, ServicePrincipalNames, userAccountControl, `
PrincipalsAllowedToRetrieveManagedPassword, msDS-ManagedPasswordInterval, msDS-SupportedEncryptionTypes
DNSHostName : gmsa-reports.corp.example.com
msDS-ManagedPasswordInterval : 30
msDS-SupportedEncryptionTypes : 24
ObjectClass : msDS-GroupManagedServiceAccount
PrincipalsAllowedToRetrieveManagedPassword : {CN=k8s-win-nodes,OU=Kubernetes,OU=Servers,DC=corp,DC=example,DC=com}
SamAccountName : gmsa-reports$
ServicePrincipalNames : {host/gmsa-reports.corp.example.com, host/gmsa-reports}
SID : S-1-5-21-3623811015-3361044348-30300820-1187
userAccountControl : 4096
ObjectClass and userAccountControl: 4096 are those very signs of a "computer". But the domain administrator does not get the password:
PS C:\> (Get-ADServiceAccount gmsa-reports -Properties msDS-ManagedPassword).'msDS-ManagedPassword'
PS C:\>
Empty: the administrator is not in msDS-GroupMSAMembership.
A domain-joined node
There are two ways to give a node the right to get the password. The first, the original one described in the Kubernetes documentation, is to join the Windows node to the domain and add its computer account to the k8s-win-nodes group. The second is to leave the node outside the domain and install a CCG plugin; it is covered below, along with why the test cluster uses the first.
The k8s-lab-win-1 node from part 38 is currently in a workgroup and resolves names through the router. To join the domain it needs a DNS server that knows the corp.example.com zone and its SRV records (how a client finds a domain controller — part 3). Since Calico was installed, the node's address lives not on the physical Ethernet adapter but on the virtual vEthernet (Ethernet), which HNS created when it attached the adapter to the virtual switch (part 37); we set DNS on it:
PS C:\> Get-DnsClientServerAddress -AddressFamily IPv4 | Where-Object ServerAddresses
InterfaceAlias Interface Address ServerAddresses
Index Family
-------------- --------- ------- ---------------
vEthernet (Ethernet) 14 IPv4 {192.168.10.1}
PS C:\> Set-DnsClientServerAddress -InterfaceAlias 'vEthernet (Ethernet)' -ServerAddresses 192.168.10.53
dc1 resolves external names through its forwarders (part 7), so the node will still see ghcr.io and the rest of the internet.
Joining the domain requires a restart, so the node is drained first (drain — part 35):
$ kubectl drain k8s-lab-win-1 --ignore-daemonsets --delete-emptydir-data --timeout=10m
node/k8s-lab-win-1 cordoned
Warning: ignoring DaemonSet-managed Pods: calico-system/calico-node-windows-q54h2, kube-system/csi-smb-node-win-tmwv5, kube-system/kube-proxy-windows-z9qdq, monitoring/alloy-windows-f6ns9, monitoring/kps-prometheus-windows-exporter-l45kh
evicting pod win-demo/reports-writer-67b96b59b-g9wrq
evicting pod win-demo/aspnetapp-769559d8ff-vk445
evicting pod win-demo/aspnetapp-769559d8ff-dkp9z
evicting pod win-demo/hello-win-759c899954-9g74w
evicting pod win-demo/hello-win-759c899954-vwdq6
...
node/k8s-lab-win-1 drained
PS C:\> Add-Computer -DomainName corp.example.com -OUPath 'OU=Kubernetes,OU=Servers,DC=corp,DC=example,DC=com' `
-Credential (Get-Credential CORP\adm-join) -Restart
What changes on the node after joining the domain:
- the name: the short name stays
k8s-lab-win-1, and the Node object in the cluster is the same — kubelet registers the node by the short host name. But the primary DNS suffix becomescorp.example.com: the node's full name is nowk8s-lab-win-1.corp.example.com, and the node registers its A record in the domain zone itself. Thek8s-lab-win-1.example.comrecord in theexample.comzone stays as it was; - time: joining the domain does not change the time source that we set manually in part 38 (
/syncfromflags:manual), so we switch the time service to the domain hierarchy withw32tm /config /syncfromflags:domhier /updateand a restart ofw32time— after that it takes time from the domain controller. For Kerberos this is good: the clock skew tolerance is five minutes, and pods live by the node's clock. How time works in a domain is the topic of part 12 of the Windows Server series; - group policies: domain policies start to apply to the node. Firewall rules, update settings with automatic restarts, restrictions on services meant for ordinary servers can break a cluster node — for example, restart it without a
drain. So the nodes live in their own OU, and policy inheritance on it is blocked, linking only what the node really needs (on group policies — part 24 of the same series):
PS C:\> Set-GPInheritance -Target 'OU=Kubernetes,OU=Servers,DC=corp,DC=example,DC=com' -IsBlocked Yes
Account policies (passwords, lockout, Kerberos ticket lifetimes) are set at the domain level and apply anyway; blocking inheritance does not turn them off.
After the restart we add the node to the group and return it to service:
PS C:\> Add-ADGroupMember k8s-win-nodes -Members 'K8S-LAB-WIN-1$'
$ kubectl uncordon k8s-lab-win-1
node/k8s-lab-win-1 uncordoned
A subtlety with the group: group membership gets into the ticket's PAC when the TGT is issued (part 6), and the node got its TGT at boot, when it was not yet in k8s-win-nodes. Until a restart or a TGT renewal the domain controller will consider that the node has no right. Instead of a restart it is enough to purge the tickets of the computer account — the SYSTEM session with the ID 0x3e7 — and check access to the password with Test-ADServiceAccount from the ActiveDirectory module:
PS C:\> Install-WindowsFeature RSAT-AD-PowerShell
PS C:\> klist -li 0x3e7 purge
Current LogonId is 0:0x5c1a2
Targeted LogonId is 0:0x3e7
Deleting all tickets:
Ticket(s) purged!
PS C:\> Test-ADServiceAccount gmsa-reports
True
True means the node got the gMSA password. False means the node is not in the group (or the group is not in PrincipalsAllowedToRetrieveManagedPassword), or a ticket with the old set of groups is still in the cache. Test-ADServiceAccount is run on the node itself — it checks the rights of the computer it runs on — at its console or over RDP. From an SSH session with key-based login or from Enter-PSSession it fails with Unable to contact the server. This may be because this server does not exist, it is currently down, or it does not have the Active Directory Web Services running: such a session has no network credentials of the user (the "double hop" problem), and the ActiveDirectory module has nothing to log on to AD Web Services on the domain controller with. Where there is no console, it is run as a scheduled task as SYSTEM — that is, as the node itself.
A rule from the Microsoft documentation: when the node is domain-joined, the node and the gMSA must be in the same domain, otherwise the node will not get the password.
The credential spec
A credential spec is a JSON document that tells CCG which gMSA to give the container and which domain it is in. It contains no secrets: only names, SIDs and the domain GUID. It is generated by the New-CredentialSpec cmdlet from the CredentialSpec module (PowerShell Gallery) on a domain-joined machine with the ActiveDirectory module — for example, on k8s-lab-win-1 itself:
PS C:\> Install-PackageProvider NuGet -Force
PS C:\> Install-Module CredentialSpec -Force
PS C:\> New-CredentialSpec -AccountName gmsa-reports -Path C:\credspecs\gmsa-reports.json
PS C:\> Get-Content C:\credspecs\gmsa-reports.json | ConvertFrom-Json | ConvertTo-Json -Depth 4 -Compress
{"CmsPlugins":["ActiveDirectory"],"DomainJoinConfig":{"Sid":"S-1-5-21-3623811015-3361044348-30300820","MachineAccountName":"gmsa-reports","Guid":"7a1b2c3d-4e5f-6071-8293-a4b5c6d7e8f9","DnsTreeName":"corp.example.com","DnsName":"corp.example.com","NetBiosName":"CORP"},"ActiveDirectoryConfig":{"GroupManagedServiceAccounts":[{"Name":"gmsa-reports","Scope":"corp.example.com"},{"Name":"gmsa-reports","Scope":"CORP"}]}}
Without -Path the file goes to Docker's CredentialSpecs directory (C:\ProgramData\Docker\CredentialSpecs), which does not exist on a node with containerd. Here the JSON is printed on one line; the same structure in readable form is in the YAML of the cluster object below. The fields:
| Field | What it is |
|---|---|
CmsPlugins |
the credential source; for gMSA always ActiveDirectory |
DomainJoinConfig.Sid |
the domain SID (without a RID) |
DomainJoinConfig.MachineAccountName |
the gMSA name without $ — under this name the container will be a "computer" in the domain |
DomainJoinConfig.Guid |
the domain GUID — the same one nltest /dsgetdc shows |
DnsTreeName, DnsName, NetBiosName |
the forest name, the DNS name and the NetBIOS name of the gMSA's domain |
GroupManagedServiceAccounts |
the list of gMSAs, one entry for the domain's DNS name and one for its NetBIOS name |
If a service in the container must run as a different gMSA than the container's "computer", that gMSA is added with the -AdditionalAccounts parameter: it ends up in GroupManagedServiceAccounts, and the service is configured to use it as its service account. For a node outside the domain a HostAccountConfig section is added to ActiveDirectoryConfig — more on it below.
The CRD and the webhook in the cluster
The external components for gMSA are developed by SIG Windows in the kubernetes-sigs/windows-gmsa repository: the GMSACredentialSpec CRD, the admission webhook and a Helm chart that installs them together. gMSA support in the pod API itself (the windowsOptions fields) has been stable since Kubernetes 1.18, while the CRD and the webhook remain external — they are not in the apiserver.
You can install them with the deploy-gmsa-webhook.sh script from the repository, which issues the webhook certificate through the cluster's CSR API and applies the manifests (this is what the Configure GMSA task of the Kubernetes documentation describes), or with the chart. The chart is more convenient: it is upgraded like any Helm release (part 27), and by default the webhook certificate is issued by cert-manager — it was installed into the test cluster in part 38 the same way as into the main one in part 19. For this the chart creates its own Issuer and Certificate and asks cert-manager to inject the CA into the webhook configurations.
# gmsa-values.yaml
replicaCount: 2
nodeSelector:
kubernetes.io/os: linux
podDisruptionBudget:
enabled: true
minAvailable: 1
randomHostname: true
nodeSelector— the webhook image is built for Linux; the pod would not land on the Windows node anyway because of the taint, but the lab's rule is to pin Linux workloads to Linux nodes explicitly;- two replicas and a PDB — the webhook sits on the path of creating every pod in the cluster, more on this below;
randomHostname— the webhook gives pods with a gMSA a random 15-character host name; why is explained in the section on names.
$ helm repo add windows-gmsa https://raw.githubusercontent.com/kubernetes-sigs/windows-gmsa/master/charts/repo
$ helm repo update
$ helm search repo windows-gmsa/gmsa
NAME CHART VERSION APP VERSION DESCRIPTION
windows-gmsa/gmsa 0.14.1 0.14.1 Windows GMSA Configuration
$ helm install gmsa windows-gmsa/gmsa -n kube-system --version 0.14.1 -f gmsa-values.yaml
NAME: gmsa
LAST DEPLOYED: Sat Nov 7 13:15:18 2026
NAMESPACE: kube-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
What appeared in the cluster:
- The
gmsacredentialspecs.windows.k8s.ioCRD (from the chart'scrds/directory): thewindows.k8s.iogroup, versionv1(stored) and the oldv1alpha1, a cluster-scoped resource (scope: Cluster) — a spec is one per cluster, not per namespace. The schema describes acredspecfield with the same keys as in the JSON above. The CRD has anapi-approved.kubernetes.ioannotation with a link to a KEP:*.k8s.iogroups are reserved for the project and require an approved proposal (part 28); - the webhook Deployment (image
registry.k8s.io/gmsa-webhook/k8s-gmsa-webhook:v0.14.1), a Service in front of it and a ServiceAccount with rights to readgmsacredentialspecsand createlocalsubjectaccessreviews; - MutatingWebhookConfiguration and ValidatingWebhookConfiguration: apiserver sends the webhook pod creation and the addition of ephemeral containers (the
pods/ephemeralcontainerssubresource —kubectl debugadds a debug container through it, and without this rule such a container would bypass the check).
The configurations have failurePolicy: Fail: if the webhook is unavailable, apiserver rejects the request. The webhook does not know in advance whether a Windows or a Linux pod is in front of it, so it receives the creation of every pod in the cluster, except pods in the release's own namespace (kube-system) and in namespaces labelled windows.k8s.io/disabled: "true". If both webhook replicas go down, not a single pod outside kube-system is created in the cluster: neither a new one nor one recreated by a ReplicaSet. Hence two replicas, a PDB and the habit of looking at the webhook first when pods "suddenly stopped being created". The risks of webhooks with failurePolicy: Fail are covered in part 28.
GMSACredentialSpec and the use permission
The JSON from New-CredentialSpec turns into a cluster object: the same keys under the credspec field.
apiVersion: windows.k8s.io/v1
kind: GMSACredentialSpec
metadata:
name: gmsa-reports
credspec:
CmsPlugins:
- ActiveDirectory
DomainJoinConfig:
Sid: S-1-5-21-3623811015-3361044348-30300820
MachineAccountName: gmsa-reports
Guid: 7a1b2c3d-4e5f-6071-8293-a4b5c6d7e8f9
DnsTreeName: corp.example.com
DnsName: corp.example.com
NetBiosName: CORP
ActiveDirectoryConfig:
GroupManagedServiceAccounts:
- Name: gmsa-reports
Scope: corp.example.com
- Name: gmsa-reports
Scope: CORP
The object name is arbitrary but, like the name of any Kubernetes object, consists of lowercase letters, digits and hyphens. Documentation examples include gmsa-WebApp1 in mixed case; such a name will not be created in a cluster, and gmsaCredentialSpecName in the pod must match the object name letter for letter. It is convenient to name the object after the gMSA itself.
The object contains no secrets, so it can be kept in Git next to the other manifests. The secret — the gMSA password — stays in Active Directory, and the right to get it is decided by the domain, not by the cluster. The cluster decides something else: which pods may ask the node to use that right. For this the webhook checks the use verb for the pod's ServiceAccount — one of the special RBAC verbs that exist not for API operations but for checks like this one (part 29):
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: gmsa-reports-user
rules:
- apiGroups: ["windows.k8s.io"]
resources: ["gmsacredentialspecs"]
resourceNames: ["gmsa-reports"]
verbs: ["use"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: reports
namespace: win-demo
automountServiceAccountToken: false
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: reports-gmsa
namespace: win-demo
subjects:
- kind: ServiceAccount
name: reports
namespace: win-demo
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: gmsa-reports-user
How the check works. The webhook creates a LocalSubjectAccessReview — a request to the authorization API asking "is such-and-such subject allowed such-and-such action" — in the pod's namespace, for system:serviceaccount:win-demo:reports, with the attributes: verb use, group windows.k8s.io, resource gmsacredentialspecs, name gmsa-reports. RBAC answers yes or no, and the webhook admits or rejects the pod. Since the check runs in the pod's namespace, an ordinary RoleBinding works even though the resource itself is cluster-scoped: the ClusterRole here is a template "the right to this gMSA", and the RoleBinding grants it in one namespace. If the gMSA is needed in a second namespace, that takes a second RoleBinding to the same ClusterRole.
Two consequences are worth understanding.
It is the pod's ServiceAccount that is checked, not the person creating the pod. A user with the create pods right in win-demo can create a pod with serviceAccountName: reports and get the gmsa-reports identity on the network, even without having the use right themselves. The boundary is the namespace: the right to create pods in a namespace that has a ServiceAccount with the right to a gMSA equals the right to act in the domain as that gMSA. A namespace with a gMSA and the rights in it are granted with this in mind.
A node can get the passwords of all gMSAs allowed to its group. Kubernetes decides which pod gets a gMSA, but Windows on the node does not know about it: whoever takes over the node (for example, through a HostProcess pod with administrator rights — part 36) gets any of them. So gMSAs are given minimal rights in the domain, and for sensitive accounts a separate node group is created and such pods are pinned to those nodes with a taint and a nodeSelector (part 13).
$ kubectl apply -f gmsa-reports-credspec.yaml -f gmsa-reports-rbac.yaml
gmsacredentialspec.windows.k8s.io/gmsa-reports created
clusterrole.rbac.authorization.k8s.io/gmsa-reports-user created
serviceaccount/reports created
rolebinding.rbac.authorization.k8s.io/reports-gmsa created
$ kubectl auth can-i use gmsacredentialspecs.windows.k8s.io/gmsa-reports -n win-demo --as=system:serviceaccount:win-demo:reports
Warning: resource 'gmsacredentialspecs' is not namespace scoped in group 'windows.k8s.io'
yes
$ kubectl auth can-i use gmsacredentialspecs.windows.k8s.io/gmsa-reports -n win-demo --as=system:serviceaccount:win-demo:default
Warning: resource 'gmsacredentialspecs' is not namespace scoped in group 'windows.k8s.io'
no
The kubectl warning is honest — the resource is cluster-scoped — but the check still runs with the win-demo namespace, exactly the way the webhook does it.
DNS and network: the path from the pod to the domain controller
A pod with a gMSA finds the domain controller itself: the Netlogon service in the container runs the DC Locator — it queries DNS for the SRV record _ldap._tcp.dc._msdcs.corp.example.com, gets a list of domain controllers and sends them an LDAP ping (part 3). The pod's DNS server is the test cluster's CoreDNS (10.96.0.10), and it knows nothing about corp.example.com: everything that is not cluster.local it forwards to the router 192.168.10.1. We reconfigured the node, but pods ask CoreDNS, not the node. So we add a server block for the domain zone to the test cluster's Corefile — exactly the one that was the stub-zone example in part 16:
corp.example.com:53 {
errors
cache 30
forward . 192.168.10.53
}
kubectl -n kube-system edit configmap coredns, the block goes next to .:53; the reload plugin applies it in about a minute. CoreDNS picks a block by the longest matching zone, and the domain's SRV records now come from dc1. If there are several domain controllers, all of them are listed in forward. And one more rule from part 37: partial names like hello.demo do not resolve from Windows pods, so both cluster services (<service>.<namespace>.svc.cluster.local) and domain resources (srv1.corp.example.com, not srv1) are written as full names in the configuration of Windows applications.
The network. A packet from the pod to 192.168.10.53 leaves the pod address pool, and Calico on the Windows node replaces the source address with the node's address (the pool's natOutgoing, part 37). The domain controller, the file server and SQL Server see all pods of the node as 192.168.10.25. For Kerberos this does not matter: Windows does not write the client address into the ticket. It matters for firewalls: a rule "allow only from node addresses" on the domain servers works, while a rule by pod address does not.
Which ports the pod needs (the full table of domain controller ports is in part 3):
| Where to | Port | Why |
|---|---|---|
dc1 |
88 TCP and UDP | Kerberos: TGT and service tickets |
dc1 |
389 TCP and UDP | LDAP, and LDAP ping when locating the domain controller |
dc1 |
135 TCP and 49152–65535 TCP | RPC — the Windows remote procedure call mechanism: on 135 the Endpoint Mapper reports the service's dynamic port; Netlogon builds the secure channel over it |
srv1 |
445 TCP | SMB |
sql1 |
1433 TCP | SQL Server |
DNS is not on this list: pods ask CoreDNS, and it is the CoreDNS pods on Linux nodes that go to dc1 on port 53. If a default-deny egress policy is in effect in win-demo (part 20), the pod with a gMSA needs a permission:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: reports-to-domain
namespace: win-demo
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: reports
policyTypes: [Egress]
egress:
- to:
- ipBlock: {cidr: 192.168.10.53/32}
ports:
- {protocol: TCP, port: 88}
- {protocol: UDP, port: 88}
- {protocol: TCP, port: 389}
- {protocol: UDP, port: 389}
- {protocol: TCP, port: 135}
- {protocol: TCP, port: 49152, endPort: 65535}
- to:
- ipBlock: {cidr: 192.168.10.54/32} # srv1
ports:
- {protocol: TCP, port: 445}
- to:
- ipBlock: {cidr: 192.168.10.55/32} # sql1
ports:
- {protocol: TCP, port: 1433}
Calico enforces policies on Windows nodes too, through VFP rules.
A pod running as the gMSA
The example is a reports service: once a minute it reads the list of files in the \\srv1.corp.example.com\reports share and asks SQL Server on sql1 under which name it sees the service. There is no login or password in the manifests.
Rights to resources are granted to the gMSA account as to any domain computer — with the dollar at the end. On srv1:
PS C:\> New-Item -ItemType Directory D:\Shares\reports
PS C:\> icacls D:\Shares\reports /grant 'CORP\gmsa-reports$:(OI)(CI)RX'
PS C:\> New-SmbShare -Name reports -Path D:\Shares\reports -ReadAccess 'CORP\gmsa-reports$' -EncryptData $true
On sql1, a login for the gMSA. The example needs no rights in databases: the query SELECT SUSER_SNAME(), DB_NAME() runs in master too. A real application is given rights in its own database — CREATE USER ... FOR LOGIN and a role such as db_datareader.
CREATE LOGIN [CORP\gmsa-reports$] FROM WINDOWS WITH DEFAULT_DATABASE = master;
In a real domain, gMSAs, like people, are put into a global group by role and given rights through the resource's domain local groups (AGDLP, part 3); here the rights are granted directly to keep the example short. One caveat: a gMSA's group membership, as with any computer, gets into its tickets when the TGT is obtained, and the pod will see a new group only with a new TGT.
The application. A script in a ConfigMap and a Deployment:
apiVersion: v1
kind: ConfigMap
metadata:
name: reports-script
namespace: win-demo
data:
check.ps1: |
while ($true) {
try {
$files = (Get-ChildItem '\\srv1.corp.example.com\reports').Count
$conn = New-Object System.Data.SqlClient.SqlConnection `
'Server=sql1.corp.example.com;Database=master;Integrated Security=SSPI;Encrypt=True;TrustServerCertificate=True'
$conn.Open()
$cmd = $conn.CreateCommand()
$cmd.CommandText = 'SELECT SUSER_SNAME(), DB_NAME()'
$r = $cmd.ExecuteReader(); $null = $r.Read()
'{0:s} user={1} files={2} sql_login={3} db={4}' -f (Get-Date),
[Security.Principal.WindowsIdentity]::GetCurrent().Name, $files, $r.GetString(0), $r.GetString(1)
$conn.Close()
} catch {
'{0:s} error: {1}' -f (Get-Date), $_.Exception.Message
}
Start-Sleep -Seconds 60
}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: reports
namespace: win-demo
spec:
replicas: 2
selector:
matchLabels:
app.kubernetes.io/name: reports
template:
metadata:
labels:
app.kubernetes.io/name: reports
spec:
runtimeClassName: windows-2025
os:
name: windows
serviceAccountName: reports
automountServiceAccountToken: false
securityContext:
runAsNonRoot: true
windowsOptions:
gmsaCredentialSpecName: gmsa-reports
runAsUserName: "NT AUTHORITY\\NETWORK SERVICE"
containers:
- name: reports
image: mcr.microsoft.com/windows/servercore:ltsc2025
command: ["powershell.exe", "-NoLogo", "-NoProfile", "-ExecutionPolicy", "Bypass",
"-File", 'C:\scripts\check.ps1']
resources:
requests: {cpu: 100m, memory: 256Mi}
limits: {cpu: "1", memory: 512Mi}
volumeMounts:
- name: scripts
mountPath: 'C:\scripts'
readOnly: true
volumes:
- name: scripts
configMap:
name: reports-script
Let us go through what relates to gMSA:
runtimeClassName: windows-2025— the pod lands onk8s-lab-win-1with the right OS build and tolerates the Windows node's taint (part 36); theservercore:ltsc2025image is needed because nanoserver has neither PowerShell 5.1 nor .NET Framework withSqlClient;gmsaCredentialSpecNameat the pod level — all containers of the pod get the gMSA. It can also be set for an individual container in itssecurityContext.windowsOptions;runAsUserName: "NT AUTHORITY\\NETWORK SERVICE"— without it the process would run as the image's user: forservercorethat isContainerAdministrator, a local account that does not go onto the network as the gMSA. A backslash in a double-quoted YAML string is doubled. In your own images the same is set in the Dockerfile with the lineUSER "NT AUTHORITY\NETWORK SERVICE"; for IIS the application pool is given theNetworkServiceidentity (appcmd set AppPool DefaultAppPool -processModel.identityType:NetworkService), for a Windows service it issc.exe config <service> obj= "NT AUTHORITY\NETWORK SERVICE";os.name: windowsandrunAsNonRoot: true— thewin-demonamespace for the Windows examples from part 38 runs at therestrictedlevel (Pod Security labels — part 30), and thereportspod passes it: for pods withos.name: windowsPod Security does not require the Linux fields (allowPrivilegeEscalation, seccomp, capabilities), andrunAsNonRootis checked by kubelet — on Windows "root" meansContainerAdministrator, and Network Service passes the check;automountServiceAccountToken: false— the pod does not need a ServiceAccount token: the ServiceAccount here only carries theuseright.
The host name and the gMSA name
A pod with a gMSA has two names, and they should not be confused.
The host name is what hostname returns inside the container. In Kubernetes it is the pod name (reports-6c9f7d8b54-k2x9q) or the value of spec.hostname, if set. It is a local name: the container does not present itself to the domain with it.
The network name is the gMSA name. Since Windows Server 2019 a container always goes onto the network under the name of its gMSA, whatever its host name. In Windows Server 2016 and versions 1709 and 1803 it was different: the host name had to match the gMSA name, otherwise incoming NTLM authentication and name-to-SID translation broke. Hence the old recipes with hostname: gmsa-reports in the manifest — on Windows Server 2019 and newer this is unnecessary and harmful.
Here is why it is harmful. Microsoft describes a known issue: if several containers with the same gMSA run at the same time with the same host name, they interfere with each other when talking to the domain controller — a new container breaks the secure channel of the previous one, and authentication fails intermittently, including nltest /sc_verify inside the container. The remedy is unique host names. In Kubernetes pod names are unique, but a computer's NetBIOS name in Windows is limited to 15 characters, and the names of pods of one ReplicaSet differ only in the last five: reports-6c9f7d8b54-k2x9q and reports-6c9f7d8b54-p7m3t have the same first 15 characters. For this case the webhook has the randomHostname parameter, which we enabled: for pods with a gMSA that have no spec.hostname, the webhook sets a host name of 15 random characters. The alternative is short Deployment names, but it is more reliable not to depend on name length.
Incoming authentication. If a pod accepts Windows authentication — an IIS site with Negotiate, a service with Kerberos — clients reach it by a Service or Ingress name, for example reports.corp.example.com. The client builds the SPN from the name it typed (part 6), so the gMSA needs an SPN for that name, otherwise the client gets KDC_ERR_S_PRINCIPAL_UNKNOWN and silently falls back to NTLM:
PS C:\> setspn -S HTTP/reports.corp.example.com gmsa-reports$
Reaching a pod or a service by IP address never gives Kerberos: nobody has an SPN with an address registered, and the client switches to NTLM — a challenge–response protocol in which the server forwards the verification to the domain controller over the secure channel. NTLM works with gMSA, but without mutual authentication and with extra load on the domain controller.
Nodes outside the domain: CCG and plugins
Joining a node to the domain is extra work for every new node: a computer account, a group, a restart, and when the node is recreated, cleaning up the old account. So Windows Server 2019 introduced a second mode: gMSA for a non-domain-joined node. The node stays in a workgroup, and CCG gets the gMSA password as an ordinary domain user whose credentials are kept in a secret store. They are fetched from there by a CCG plugin — a COM object (a Windows component loaded by its class identifier, CLSID) that implements the ICcgDomainAuthCredentials interface with a single method, GetPasswordCredentials: take a string from the spec as input and return the domain, the user name and the password.
Windows node outside the domain
HCS: container, credential spec with a HostAccountConfig section
│
▼
CCG ── 1. COM class PluginGUID ──► plugin ── 2. PluginInput ──► secret store:
▲ │ Secret, Key Vault,
└─── 3. CORP\svc-ccg-reports ◄──────┘ Vault ...
and its password
CCG ── 4. Kerberos: TGT as svc-ccg-reports ──────────────────────► dc1
CCG ── 5. LDAP: msDS-ManagedPassword gmsa-reports ───────────────► dc1
CCG ◄─ 6. gmsa-reports$ password ───────────────────────────────── dc1
│
▼
container: on the network it is CORP\gmsa-reports$
The spec gets a HostAccountConfig section with three fields:
"ActiveDirectoryConfig": {
"GroupManagedServiceAccounts": [ ... ],
"HostAccountConfig": {
"PortableCcgVersion": "1",
"PluginGUID": "{e4781092-f116-4b79-b55e-28eb6a224e26}",
"PluginInput": "gmsa-system:ccg-reports-credentials"
}
}
PortableCcgVersion— the version of the mechanism, currently always"1";PluginGUID— the plugin's CLSID, different for each implementation; the example uses the Rancher plugin;PluginInput— a string whose meaning only the plugin knows: for the Rancher plugin it is<namespace>:<Secret name>, for the Azure Key Vault plugin it is the managed identity's identifier and the secret's address.
In Active Directory one thing changes: instead of the group with the nodes' computer accounts, PrincipalsAllowedToRetrieveManagedPassword gets a group with this user. The user's rights must be exactly these: membership in a group allowed to read the passwords of the required gMSAs, a long random password, no interactive logons. Its credentials allow getting gMSA passwords from any machine that can see the domain controller, so they must be protected no worse than the passwords themselves.
Windows has no ready-made plugin. Microsoft states plainly that Windows does not include a built-in plugin and that you need to get one from a vendor or write your own. What exists:
| Implementation | Where the credentials are stored | Status |
|---|---|---|
| AKS (Azure) | Azure Key Vault; the plugin is open source: microsoft/Azure-Key-Vault-Plugin-gMSA | built into AKS Windows nodes |
| AKS on Windows Server (Arc) | a Kubernetes Secret | built into the distribution, installed by its cmdlets |
| Amazon EKS | AWS Secrets Manager, SSM | built into the Windows AMI for EKS |
| rancher/Rancher-Plugin-gMSA | a Kubernetes Secret that an "account provider" HostProcess pod serves to the plugin on the node | experimental; the requirements list Windows Server 2019 and 2022 |
| briantist/CcgVault | HashiCorp Vault | a minimal implementation |
In this mode CCG has its own log — Microsoft-Windows-Containers-CCG/Admin on the node — and it shows at which step a failure happened:
| Event | What it means |
|---|---|
| 1 | the plugin from the spec was loaded |
| 2 | the gMSA password was obtained; the gMSA and the plugin are named |
| 3 | the spec could not be parsed: wrong GUID or fields |
| 4 | the plugin did not load: not installed or not registered |
| 6 | the plugin loaded but returned no credentials: wrong PluginInput or no access to the store |
| 7 | the gMSA password changed, CCG fetches the credentials again |
| 8 | the credentials were obtained, but the gMSA password is not issued with them: the user is not allowed to read it |
PS C:\> Get-WinEvent -LogName 'Microsoft-Windows-Containers-CCG/Admin' -MaxEvents 5 | Format-Table TimeCreated, Id, Message -Wrap
One more difference: a node outside the domain does not get DNS and time settings from the domain. You have to help it resolve domain names and keep its clock within five minutes of the domain controller yourself.
What the test cluster uses. There the node is domain-joined, and here is why:
| Domain-joined node | Non-domain-joined node with a plugin | |
|---|---|---|
| Components | only Windows and the webhook from kubernetes-sigs | plus the plugin and its delivery of credentials |
| Secret in the cluster | none: the node keeps its own password | the user's credentials in a secret store |
| New node | domain join, group, restart | the plugin is installed on the node automatically |
| Time and DNS | from the domain | configure yourself |
| Domain policies | apply, an OU of its own is needed | do not apply |
| Maturity on kubeadm | the main scenario of the Kubernetes documentation | implementations are tied to clouds or experimental |
For one or two permanent Windows nodes, joining the domain is simpler and needs no third-party code on the node. In a cloud where Windows nodes are created and deleted by autoscaling, the domainless mode wins — there the provider already has the plugin built in.
Looking at it by hand
The components in the cluster:
$ kubectl get crd gmsacredentialspecs.windows.k8s.io
NAME CREATED AT
gmsacredentialspecs.windows.k8s.io 2026-11-07T10:15:18Z
$ kubectl api-resources --api-group=windows.k8s.io
NAME SHORTNAMES APIVERSION NAMESPACED KIND
gmsacredentialspecs windows.k8s.io/v1 false GMSACredentialSpec
$ kubectl get gmsacredentialspecs
NAME AGE
gmsa-reports 38m
$ kubectl -n kube-system get pods -o wide | grep gmsa
gmsa-5bfdf5ff88-kf4sc 1/1 Running 0 39m 172.21.172.16 k8s-lab-node-2 <none> <none>
gmsa-5bfdf5ff88-kzcf7 1/1 Running 0 39m 172.21.211.28 k8s-lab-node-1 <none> <none>
$ kubectl get mutatingwebhookconfigurations,validatingwebhookconfigurations | grep gmsa
mutatingwebhookconfiguration.admissionregistration.k8s.io/gmsa 1 39m
validatingwebhookconfiguration.admissionregistration.k8s.io/gmsa 1 39m
$ kubectl -n kube-system get certificates,issuers | grep gmsa
certificate.cert-manager.io/gmsa True gmsa-server-cert 39m
certificate.cert-manager.io/gmsa-ca True gmsa-root-ca 39m
issuer.cert-manager.io/gmsa True 39m
The webhook replicas spread over the two Linux workers, and the taint keeps them off the control plane: the webhook will survive the failure of one node, and the PDB will not let drain evict both replicas at once. The webhook certificate was issued by cert-manager — for this the chart created its own CA (gmsa-root-ca) and Issuer. We apply reports and look at what the mutating webhook did:
$ kubectl apply -f reports.yaml
configmap/reports-script created
deployment.apps/reports created
$ kubectl -n win-demo get pods -l app.kubernetes.io/name=reports -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
reports-56c8c6dc7f-8ccbj 1/1 Running 0 7m32s 172.21.41.47 k8s-lab-win-1 <none> <none>
reports-56c8c6dc7f-skpdb 1/1 Running 0 7m29s 172.21.41.48 k8s-lab-win-1 <none> <none>
$ kubectl -n win-demo get pod reports-56c8c6dc7f-8ccbj -o jsonpath='{.spec.hostname}{"\n"}{.spec.securityContext.windowsOptions}{"\n"}'
aa0a06c0-caeb-4
{"gmsaCredentialSpec":"{\"ActiveDirectoryConfig\":{\"GroupManagedServiceAccounts\":[{\"Name\":\"gmsa-reports\",\"Scope\":\"corp.example.com\"},…]},\"CmsPlugins\":[\"ActiveDirectory\"],\"DomainJoinConfig\":{…}}","gmsaCredentialSpecName":"gmsa-reports","runAsUserName":"NT AUTHORITY\\NETWORK SERVICE"}
The manifest had only the name, while the pod object has the whole spec JSON: that is what kubelet will get. spec.hostname is 15 random characters from randomHostname. The application log:
$ kubectl -n win-demo logs reports-56c8c6dc7f-8ccbj --tail=2
2026-11-07T13:47:32 user=NT AUTHORITY\NETWORK SERVICE files=3 sql_login=CORP\gmsa-reports$ db=master
2026-11-07T13:48:32 user=NT AUTHORITY\NETWORK SERVICE files=3 sql_login=CORP\gmsa-reports$ db=master
This is the essence of the mechanism in one line: locally the process is NETWORK SERVICE, while SQL Server sees the login CORP\gmsa-reports$. Let us look inside. kubectl exec starts a process as the same user as the container — that is, as Network Service too:
$ kubectl -n win-demo exec -it reports-56c8c6dc7f-8ccbj -- powershell
PS C:\> whoami
nt authority\network service
PS C:\> hostname
aa0a06c0-caeb-4
PS C:\> nltest /dsgetdc:corp.example.com
DC: \\dc1.corp.example.com
Address: \\192.168.10.53
Dom Guid: 7a1b2c3d-4e5f-6071-8293-a4b5c6d7e8f9
Dom Name: corp.example.com
Forest Name: corp.example.com
Dc Site Name: Office
Our Site Name: Office
Flags: PDC GC DS LDAP KDC TIMESERV GTIMESERV WRITABLE DNS_DC DNS_DOMAIN DNS_FOREST CLOSE_SITE FULL_SECRET WS DS_8 DS_9 DS_10 KEYLIST DS_13
The command completed successfully
The DC Locator in the pod found the domain controller — so the corp.example.com block in CoreDNS and the path to dc1 work. The tickets of the Network Service session (0x3e4), shared by the application and our shell:
PS C:\> klist
Current LogonId is 0:0x3e4
Cached Tickets: (7)
#0> Client: gmsa-reports$ @ CORP.EXAMPLE.COM
Server: krbtgt/CORP.EXAMPLE.COM @ CORP.EXAMPLE.COM
KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
Ticket Flags 0x60a10000 -> forwardable forwarded renewable pre_authent name_canonicalize
...
#1> Client: gmsa-reports$ @ CORP.EXAMPLE.COM
Server: krbtgt/CORP.EXAMPLE.COM @ CORP.EXAMPLE.COM
KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
Ticket Flags 0x40e10000 -> forwardable renewable initial pre_authent name_canonicalize
Start Time: 11/7/2026 13:47:29 (local)
End Time: 11/7/2026 23:47:29 (local)
Renew Time: 11/14/2026 13:47:29 (local)
Session Key Type: AES-256-CTS-HMAC-SHA1-96
Cache Flags: 0x1 -> PRIMARY
Kdc Called: dc1.corp.example.com
...
#3> Client: gmsa-reports$ @ CORP.EXAMPLE.COM
Server: MSSQLSvc/sql1.corp.example.com:1433 @ CORP.EXAMPLE.COM
KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
...
#4> Client: gmsa-reports$ @ CORP.EXAMPLE.COM
Server: cifs/srv1.corp.example.com @ CORP.EXAMPLE.COM
KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
...
Client: gmsa-reports$ in all tickets: the TGT was obtained as the gMSA, and with it the tickets for SMB on srv1 and for SQL Server. The other tickets in the cache (ldap/, GC/ and cifs/ of the domain controller) are needed by Windows itself in the container — by the Netlogon service and directory lookups. The MSSQLSvc/... ticket proves that the SQL Server login went over Kerberos, not NTLM: with a fallback to NTLM there would be no service ticket in the cache. The initial flag on the TGT means it was obtained by an AS exchange, that is, the container authenticated with the gMSA password that CCG passed to it. klist get krbtgt requests a TGT again and answers A ticket to krbtgt has been retrieved successfully if the domain controller is reachable.
The secure channel — a persistent authenticated RPC connection from the Netlogon service to the domain controller, through which the "computer" verifies NTLM logons and looks up names and SIDs — is checked with nltest /sc_verify. This and other Netlogon control commands need administrator rights in the container, and reports runs as Network Service. Diagnostics need a temporary pod with the same gMSA but running as the image's default user, ContainerAdministrator. In win-demo at the restricted level Pod Security will not let it in:
Error from server (Forbidden): error when creating "gmsa-debug.yaml": pods "gmsa-debug" is forbidden: violates PodSecurity "restricted:v1.37": runAsNonRoot != true (pod or container "shell" must set securityContext.runAsNonRoot=true)
So the debug pod lives in a separate namespace at the baseline level, with its own ServiceAccount and its own RoleBinding to the same ClusterRole — exactly the "gMSA in a second namespace" case mentioned above:
apiVersion: v1
kind: Namespace
metadata:
name: win-debug
labels:
pod-security.kubernetes.io/enforce: baseline
pod-security.kubernetes.io/enforce-version: v1.37
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: gmsa-debug
namespace: win-debug
automountServiceAccountToken: false
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: gmsa-debug
namespace: win-debug
subjects:
- kind: ServiceAccount
name: gmsa-debug
namespace: win-debug
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: gmsa-reports-user
---
apiVersion: v1
kind: Pod
metadata:
name: gmsa-debug
namespace: win-debug
spec:
runtimeClassName: windows-2025
os: {name: windows}
serviceAccountName: gmsa-debug
securityContext:
windowsOptions:
gmsaCredentialSpecName: gmsa-reports
containers:
- name: shell
image: mcr.microsoft.com/windows/servercore:ltsc2025
command: ["powershell.exe", "-Command", "Start-Sleep -Seconds 86400"]
$ kubectl apply -f gmsa-debug.yaml
namespace/win-debug created
serviceaccount/gmsa-debug created
rolebinding.rbac.authorization.k8s.io/gmsa-debug created
pod/gmsa-debug created
$ kubectl -n win-debug exec gmsa-debug -- whoami
user manager\containeradministrator
$ kubectl -n win-debug exec gmsa-debug -- nltest /sc_verify:corp.example.com
Flags: 800000b0 HAS_IP HAS_TIMESERV Authentication Service: Kerberos
Trusted DC Name \\dc1.corp.example.com
Trusted DC Connection Status Status = 0 0x0 NERR_Success
Trust Verification Status = 0 0x0 NERR_Success
The command completed successfully
$ kubectl delete namespace win-debug
namespace "win-debug" deleted
NERR_Success in both lines: the secure channel with dc1 is established, the trust to the domain is confirmed, and the check went over Kerberos. This namespace also illustrates what was said about the rights boundary: any pod with a ServiceAccount that has been given the RoleBinding got the gmsa-reports identity on the network — nobody asked whether whoever created the pod had the use right.
From the file server's side the session looks like this:
PS C:\> Get-SmbSession | Where-Object ClientUserName -like '*gmsa*' | Select-Object ClientComputerName, ClientUserName, Dialect, NumOpens
ClientComputerName ClientUserName Dialect NumOpens
------------------ -------------- ------- --------
192.168.10.25 CORP\gmsa-reports$ 3.1.1 1
192.168.10.25 CORP\gmsa-reports$ 3.1.1 1
The user is the gMSA, the address is the node's: both reports pods come from 192.168.10.25 after NAT, each with its own session. SMB 3.1.1, and the share is encrypted.
And a webhook refusal — a pod in win-demo with the default ServiceAccount, which has no use right:
$ kubectl -n win-demo apply -f gmsa-test.yaml
Error from server: error when creating "gmsa-test.yaml": admission webhook "admission-webhook.windows-gmsa.sigs.k8s.io" denied the request: service account "default" is not authorized to `use` GMSA cred spec "gmsa-reports"
Common errors
Pods are not created, Linux pods included. A ReplicaSet FailedCreate event with the text failed calling webhook "admission-webhook.windows-gmsa.sigs.k8s.io" — the webhook is unavailable, and failurePolicy: Fail does not let pods through. Look at the webhook pods in kube-system, its Service and its certificate (cert-manager should have issued the Certificate and injected the CA into the webhook configurations).
service account "…" is not authorized to `use` GMSA cred spec "…". There is no RoleBinding in the pod's namespace, the manifest has the wrong ServiceAccount (default by default), or the ClusterRole has a typo in resourceNames. The check is kubectl auth can-i use gmsacredentialspecs.windows.k8s.io/<name> -n <namespace> --as=system:serviceaccount:<namespace>:<sa>.
admission webhook "admission-webhook.windows-gmsa.sigs.k8s.io" denied the request: cred spec gmsa-missing does not exist. The name in gmsaCredentialSpecName does not match the name of the GMSACredentialSpec object — case counts too.
The Pod "…" is invalid: spec: Forbidden: pod updates may not change fields other than spec.containers[*].image, … when trying to change the gMSA of a running pod. The gMSA settings, like almost all of the pod spec, cannot be changed after creation — the API itself refuses, the request never gets to the webhook. They are changed in the Deployment template, and the pods are recreated.
Login failed for user 'CORP\gmsa-reports$' from SQL Server. Kerberos did its part — the pod's klist has a ticket for MSSQLSvc/sql1.corp.example.com:1433 — but SQL Server has no login for the gMSA. Do not confuse this message with ANONYMOUS LOGON below: here the server recognised the gMSA but did not let it in.
The pod hangs in ContainerCreating, the events show an hcs::CreateComputeSystem error. CCG did not get the gMSA password. On a domain-joined node, run Test-ADServiceAccount <gmsa> on the node itself: False means that the node is not in the group from PrincipalsAllowedToRetrieveManagedPassword, the group is the wrong one, or the node still holds a TGT with the old list of groups (klist -li 0x3e7 purge). On a node outside the domain, check the Microsoft-Windows-Containers-CCG/Admin log, events 3–8 from the table above.
Getting DC name failed: Status = 1355 0x54b ERROR_NO_SUCH_DOMAIN from nltest /dsgetdc in the pod. The pod does not find the domain controller: CoreDNS has no block for corp.example.com, or dc1 is unreachable from the pod (NetworkPolicy, the domain controller's firewall, routing). Check step by step: Resolve-DnsName -Type SRV _ldap._tcp.dc._msdcs.corp.example.com and Test-NetConnection dc1.corp.example.com -Port 389 from inside the pod.
I_NetLogonControl failed: Status = 1722 0x6ba RPC_S_SERVER_UNAVAILABLE from nltest /sc_verify or nltest /query. If the gMSA was not applied to the pod at all (no gmsaCredentialSpecName, or the webhook did not add the JSON), the container is an ordinary computer outside the domain, and every Netlogon command answers like this; first look at the pod's .spec.securityContext.windowsOptions. If it was applied, RPC does not get through to the domain controller: port 135 and the dynamic range. When the secure channel fails, the Kubernetes documentation suggests nltest /sc_reset:corp.example.com inside the container, and if only that helps, deal with the network rather than automating Netlogon restarts.
Intermittent authentication failures on some replicas but not on others. Identical host names on pods with the same gMSA — see the section on names; enable randomHostname or check that the manifest does not set spec.hostname.
Access is denied to the share or Login failed for user 'NT AUTHORITY\ANONYMOUS LOGON' in SQL Server. The application runs as ContainerUser or ContainerAdministrator, not as Network Service or Local System, and does not go onto the network as the gMSA. Or the rights were granted without the dollar — CORP\gmsa-reports instead of CORP\gmsa-reports$: such an account does not exist or, worse, there is a user with the same name without the dollar. Microsoft separately advises checking that the domain has no other object whose name matches the gMSA: Get-ADObject -Filter 'sAMAccountName -like "gmsa-reports*"'.
KRB_AP_ERR_SKEW in the Kerberos log. The node's clock has drifted from the domain controller by more than five minutes. Pods live by the node's clock; on a domain-joined node w32tm /query /source must show the domain controller.
The share is reachable by IP but not by name. The Kubernetes documentation describes this case for pods with a gMSA and suggests an HNS registry value on the nodes, after which the pods have to be recreated:
PS C:\> reg add "HKLM\SYSTEM\CurrentControlSet\Services\hns\State" /v EnableCompartmentNamespace /t REG_DWORD /d 1
Windows pods now go to the domain as a gMSA: Active Directory keeps and changes the password, the node fetches it, and the cluster decides through RBAC which pod to trust with it. This concludes the block on the hybrid cluster — from how a Windows node and its network work to workloads and domain authentication. But the GMSACredentialSpec, the RoleBinding and the reports Deployment, like everything before them, got into the cluster with kubectl apply from a workstation.
Sources
- Kubernetes: Configure GMSA for Windows Pods and containers, Configure RunAsUserName for Windows pods and containers, Pod Security Standards — the section on
spec.os.name. - kubernetes-sigs/windows-gmsa: the CRD, the webhook, the
gmsaHelm chart and itsvalues.yaml. - Microsoft: Create gMSAs for Windows containers, Configure your app to use a gMSA, Troubleshoot gMSAs for Windows containers, Group Managed Service Accounts overview, ICcgDomainAuthCredentials, gMSA in AKS on Windows Server.
- CCG plugins: Azure Key Vault Plugin, Rancher-Plugin-gMSA, CcgVault, domainless Windows authentication in Amazon EKS.
Next is part 41, the last of the main parts: GitOps with Argo CD — how to make a Git repository the single source of truth about the cluster, with changes pulled into the cluster by a controller running inside it. Application objects, sync and detection of manual drift, sync waves and hooks, ApplicationSet and projects, what to do with secrets that do not belong in Git, and the hello stack with overlays for dev and prod — back in the main cluster.
Only registered users can leave a comment.
Sign in or register to leave a comment.