Windows containers by hand. Part 7: Building a Windows image without Docker and pushing it to a registry

All parts of the series “Windows containers by hand”

In the fourth part we took apart an image downloaded from a registry: the index, the manifest, the configuration and the layers. In the fifth we ran it without containerd, through HCS. One last step remains: to walk the same path in the opposite direction, building our own image without Docker and without BuildKit, putting a folder of files into it, writing a manifest and an index with os.version, and pushing the image to a registry. And then pulling it and running it, both with nerdctl and with our scripts.

The scripts for this part are in the 07-build folder of the repository.


Our own registry

We will push the image to our own registry on the test machine, so that nothing gets published outside. The reference implementation of the OCI Distribution protocol is the Distribution project, the same registry that is usually run from the registry:2 image. It has no ready-made builds for Windows, but we have had Go since the fourth part, so we will build it from source:

Set-Location C:\build
curl.exe -sSLo distribution.zip https://github.com/distribution/distribution/archive/refs/tags/v3.1.2.zip
Expand-Archive distribution.zip C:\build\src
Set-Location C:\build\src\distribution-3.1.2
C:\build\tools\go\bin\go.exe build -o C:\registry\ ./cmd/registry
C:\registry\registry.exe --version
C:\registry\registry.exe github.com/distribution/distribution/v3 v3.1.2+unknown

The registry's configuration is a YAML file. At first I gave it ordinary filesystem storage:

version: 0.1
storage:
  filesystem:
    rootdirectory: C:/registry/data
http:
  addr: 127.0.0.1:5000

The registry started, but the very first push attempt ended with a 500 error:

{"errors":[{"code":"UNKNOWN","message":"unknown error","detail":"filesystem: rename C:\\registry\\data\\docker\\registry\\v2\\repositories\\hello\\_uploads\\01a1ba9f-dc98-7dd3-8de9-b2e13b0e9dd4\\startedat.01a1ba9f-dc98-7dd4-b3a7-c9b6a61fabea.tmp C:\\registry\\data\\docker\\registry\\v2\\repositories\\hello\\_uploads\\01a1ba9f-dc98-7dd3-8de9-b2e13b0e9dd4\\startedat: The process cannot access the file because it is being used by another process.\nunlinkat C:\\registry\\data\\docker\\registry\\v2\\repositories\\hello\\_uploads\\01a1ba9f-dc98-7dd3-8de9-b2e13b0e9dd4\\startedat.01a1ba9f-dc98-7dd4-b3a7-c9b6a61fabea.tmp: The process cannot access the file because it is being used by another process."}]}

Distribution's filesystem driver renames a file that is still open. Linux allows this, Windows does not. In-memory storage is enough for this experiment: images live as long as the registry process runs. The registry listens only on 127.0.0.1 and runs as a scheduled task under SYSTEM, like CoreDNS in the second part:

@'
version: 0.1
storage:
  inmemory: {}
http:
  addr: 127.0.0.1:5000
'@ | Set-Content C:\registry\config.yml -Encoding ascii
$action = New-ScheduledTaskAction -Execute C:\registry\registry.exe -Argument 'serve C:\registry\config.yml' -WorkingDirectory C:\registry
$settings = New-ScheduledTaskSettingsSet -ExecutionTimeLimit 0 -MultipleInstances IgnoreNew
Register-ScheduledTask -TaskName registry -Action $action -Settings $settings -Trigger (New-ScheduledTaskTrigger -AtStartup) -User SYSTEM -RunLevel Highest -Force | Out-Null
Start-ScheduledTask registry
curl.exe -si http://127.0.0.1:5000/v2/ | Select-String '^(HTTP|Docker-Distribution)'
curl.exe -s http://127.0.0.1:5000/v2/_catalog
HTTP/1.1 200 OK
Docker-Distribution-Api-Version: registry/2.0
{"repositories":[]}

The same response on /v2/ as from mcr.microsoft.com in the fourth part, and an empty catalog: /v2/_catalog lists the repositories. One caution: the registry process is named registry, and Windows has a system process named Registry (it holds the registry hives in memory), so Get-Process registry finds both. Stop the registry with Stop-ScheduledTask registry, not with Stop-Process by name.


A layer from a folder

We will use Nano Server as the base image and download it with the pull-image.ps1 script. The application is a single batch file that shows where it is running:

pull-image.ps1 mcr.microsoft.com/windows/nanoserver:ltsc2025 C:\images\nanoserver
New-Item -ItemType Directory C:\lab\app -Force | Out-Null
@'
@echo off
echo Hello from an image built by hand
ver
echo user: %USERNAME%, computer: %COMPUTERNAME%
'@ | Set-Content C:\lab\app\hello.cmd -Encoding ascii

We saw how a Windows image layer is structured in the fourth part: a tar archive with a Files folder, the files added by the layer, and Hives, the registry changes. Packing such an archive from a folder with tar is not enough: Windows files have permissions and attributes that the archive must preserve, and the delta registry hives have to be prepared by the system itself. So we do what Docker does: a writable layer on top of the base image, the files go into it, and the changes are exported with wclayer export:

$base = 'C:\images\nanoserver\layers\9a4c0752c6bda5fc00f886a9acf0c0afed0750082f7999c331df978fb9f97d2a'
New-Item -ItemType Directory C:\lab\build\mnt -Force | Out-Null; Set-Location C:\lab\build
wclayer create -l $base C:\lab\build\scratch
wclayer mount -l $base C:\lab\build\scratch C:\lab\build\mnt
New-Item -ItemType Directory C:\lab\build\mnt\app | Out-Null
Copy-Item C:\lab\app\hello.cmd C:\lab\build\mnt\app\
mountvol C:\lab\build\mnt /D
wclayer unmount C:\lab\build\scratch
wclayer export -l $base -o C:\lab\build\layer.tar C:\lab\build\scratch
tar -tvf layer.tar
C:\lab\build\mnt
d---------  0 0      0           0 Nov 08 13:11 Files
d---------  0 0      0           0 Nov 08 13:11 Files/app
----------  0 0      0         102 Nov 08 13:03 Files/app/hello.cmd
d---------  0 0      0           0 Nov 08 13:11 Hives
----------  0 0      0        8192 Nov 08 13:11 Hives/DefaultUser_Delta
----------  0 0      0        8192 Nov 08 13:11 Hives/Sam_Delta
----------  0 0      0        8192 Nov 08 13:11 Hives/Security_Delta
----------  0 0      0        8192 Nov 08 13:11 Hives/Software_Delta
----------  0 0      0        8192 Nov 08 13:11 Hives/System_Delta

These are exactly the layers we saw in the ASP.NET example: Files holds only what we added, and Hives holds five delta hives, empty here because we did not change the registry. Export works only on an unmounted layer: while it is mounted, wclayer export fails with "The process cannot access the file because it is being used by another process".

wclayer export has a -z switch that compresses the archive right away. But in hcsshim 0.15.0 it produces an empty stub:

wclayer export -l $base -z -o C:\lab\build\layer-z.tar.gz C:\lab\build\scratch
(Get-Item layer-z.tar.gz).Length
10

Ten bytes: a gzip header with no data. We will compress it ourselves and compute both hashes, as in the fourth part: diff_id from the uncompressed archive, digest from the compressed one:

'diff_id sha256:' + (Get-FileHash layer.tar).Hash.ToLower()
$in = [IO.File]::OpenRead("$PWD\layer.tar"); $out = [IO.File]::Create("$PWD\layer.tar.gz")
$gz = [IO.Compression.GZipStream]::new($out, [IO.Compression.CompressionMode]::Compress)
$in.CopyTo($gz); $gz.Dispose(); $out.Dispose(); $in.Dispose()
'digest  sha256:' + (Get-FileHash layer.tar.gz).Hash.ToLower()
(Get-Item layer.tar.gz).Length
diff_id sha256:3c5247cdaeb19693665ee83ea365b6ec0de41c71cd6bdb77f98fefffdc015c4b
digest  sha256:4c77c9f11e137fd528a5acb58f2f5ecd1e6454b05d8444e956b02ce06c5b9a2c
1420

The layer is 1420 bytes. The hashes differ from build to build: the archive records the modification times of files and folders, and the same folder built a minute later gives a different diff_id. We are about to see this.


Configuration, manifest, index

The rest is JSON. The steps are collected in build-image.ps1: it creates the layer as above and writes the image in the OCI image layout format, a standard folder holding an image: an oci-layout file, an index.json index and all the objects in blobs\sha256\<hash>, exactly as they are stored in a registry. The base image is the folder created by pull-image.ps1: the layers, the manifest and the configuration come from there. We will build the image twice, on Nano Server 2025 and on Nano Server 2022, which we first download with pull-image.ps1 using the -Build 20348 switch:

foreach ($f in 'build-image.ps1', 'push-image.ps1') {
    curl.exe -sSLo "C:\hcs\$f" "https://raw.githubusercontent.com/vitaliy-leschenko/windows-containers-by-hand/main/07-build/$f"
}
pull-image.ps1 mcr.microsoft.com/windows/nanoserver:ltsc2022 C:\images\nanoserver2022 -Build 20348
build-image.ps1 C:\images\nanoserver C:\lab\app C:\lab\hello -Cmd cmd, /c, C:\app\hello.cmd
build-image.ps1 C:\images\nanoserver2022 C:\lab\app C:\lab\hello -Cmd cmd, /c, C:\app\hello.cmd
index sha256:62d28cb7771b0cf1b37db2662e468dad76e7ef15ca4ef52b2ab73c102a23da98 -> 10.0.20348.5773 sha256:808d344f215d2cf5327474d4b8857881862956c3950aab220f72e8cf946ce480
layer 1/1 sha256:aab3f83a63eabc8139cb3ee24a60957ef6c22bbaf638bf849d57bd4c8aab3e6d imported, 122077144 bytes
mcr.microsoft.com/windows/nanoserver:ltsc2022 -> C:\images\nanoserver2022, 1 layers
layer  sha256:3212d1316b9e369c8f9645adebd44a8a4da789e45b9b7b47567c0fe7e89dced8  diff_id sha256:7c9fa2ad4016d142f1dbf3382d538a941590b97b598483f2137368bdb00746b1
config sha256:a502c8c4b023d5354cb562047c7651bacd41853927f10a6e0c8670446dd3020f
manifest sha256:9d61883f0fb4339521b610da4c68a7dfeac211b22f927202e98049f5c7744169
index  10.0.26100.33674 added, 1 manifests
layer  sha256:3f706888073bdac142fa61647800062d597fcb906288d068cb4e5efb1c7566d3  diff_id sha256:4e60cf5dec4904cf0fb0896f7a2eb35fc51aee05f710b3ff599cec5a0faaa7a7
config sha256:800c0b032d98b263578bc834260bb1fafd02a01d547bbe4ff5cd14079b6e5097
manifest sha256:d5b420e7398204228546758238bd1d5fd8c01d67ae613630729e15358f7383b7
index  10.0.20348.5773 added, 2 manifests

The diff_id of the first layer is 7c9fa2ad…, not 3c5247cd… as for the layer built by hand shortly before: a different time in the archive. What we got:

Get-ChildItem C:\lab\hello -Recurse -File | Format-Table Length, Name
   Length Name
   ------ ----
      558 index.json
       30 oci-layout
     1416 3212d1316b9e369c8f9645adebd44a8a4da789e45b9b7b47567c0fe7e89dced8
     1406 3f706888073bdac142fa61647800062d597fcb906288d068cb4e5efb1c7566d3
      806 800c0b032d98b263578bc834260bb1fafd02a01d547bbe4ff5cd14079b6e5097
      562 9d61883f0fb4339521b610da4c68a7dfeac211b22f927202e98049f5c7744169
      808 a502c8c4b023d5354cb562047c7651bacd41853927f10a6e0c8670446dd3020f
194222443 a7eba7a59d48cb7e97e4eeb9fc22cf6a070f73a398171e5da84ae4aecc15cbde
      562 d5b420e7398204228546758238bd1d5fd8c01d67ae613630729e15358f7383b7
122077144 f402befcc0a67b05a1ef1e3681a5025acbd688a95ae05b89f0c529897b13d42b

Two base layers of 194 and 122 MB, the same compressed Nano Server 2025 and 2022 archives from Microsoft's registry, two layers of ours at about one and a half kilobytes each, two configurations, two manifests and an index. Let's go through the JSON from the top down.

Index

{
    "schemaVersion": 2,
    "mediaType": "application/vnd.oci.image.index.v1+json",
    "manifests": [
        {
            "mediaType": "application/vnd.oci.image.manifest.v1+json",
            "digest": "sha256:9d61883f0fb4339521b610da4c68a7dfeac211b22f927202e98049f5c7744169",
            "size": 562,
            "platform": { "architecture": "amd64", "os": "windows", "os.version": "10.0.26100.33674" }
        },
        {
            "mediaType": "application/vnd.oci.image.manifest.v1+json",
            "digest": "sha256:d5b420e7398204228546758238bd1d5fd8c01d67ae613630729e15358f7383b7",
            "size": 562,
            "platform": { "architecture": "amd64", "os": "windows", "os.version": "10.0.20348.5773" }
        }
    ]
}

One manifest per platform. For Windows images the platform is defined not only by os and architecture but also by os.version, the build and update number of the base image. The runtime uses it to choose a variant: on a build 26100 host, the first one. The script takes os.version from the base image's configuration: it must match what the layer was actually built on, otherwise the runtime will choose a variant that will not start on this host.

Manifest

{
    "schemaVersion": 2,
    "mediaType": "application/vnd.oci.image.manifest.v1+json",
    "config": {
        "mediaType": "application/vnd.oci.image.config.v1+json",
        "digest": "sha256:a502c8c4b023d5354cb562047c7651bacd41853927f10a6e0c8670446dd3020f",
        "size": 808
    },
    "layers": [
        {
            "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
            "digest": "sha256:a7eba7a59d48cb7e97e4eeb9fc22cf6a070f73a398171e5da84ae4aecc15cbde",
            "size": 194222443
        },
        {
            "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
            "digest": "sha256:3212d1316b9e369c8f9645adebd44a8a4da789e45b9b7b47567c0fe7e89dced8",
            "size": 1416
        }
    ]
}

Layers go from the bottom up, the base one first. The base layer is the same compressed archive a7eba7a5… as in Microsoft's registry; in Microsoft's image it has the Docker type application/vnd.docker.image.rootfs.diff.tar.gzip, in ours the OCI type, and the bytes are the same. The layer did not even have to be copied: in an OCI manifest a layer can have a urls field with the addresses to download it from. But runtimes and registries treat such "external" layers differently, so it is simpler to include a copy.

Configuration

{
    "architecture": "amd64",
    "config": {
        "Hostname": "",
        "Domainname": "",
        "User": "ContainerUser",
        "AttachStdin": false,
        "AttachStdout": false,
        "AttachStderr": false,
        "Tty": false,
        "OpenStdin": false,
        "StdinOnce": false,
        "Env": null,
        "Cmd": [ "cmd", "/c", "C:\\app\\hello.cmd" ],
        "Image": "",
        "Volumes": null,
        "WorkingDir": "C:\\app",
        "Entrypoint": null,
        "OnBuild": null,
        "Labels": null
    },
    "created": "2026-11-08T10:11:32Z",
    "history": [
        { "created": "2026-10-24T23:26:15.8743218Z", "created_by": "Apply image 10.0.26100.33674" },
        { "created": "2026-11-08T10:11:32Z", "created_by": "build-image.ps1: C:\\lab\\app -\u003e C:\\app" }
    ],
    "os": "windows",
    "os.version": "10.0.26100.33674",
    "rootfs": {
        "type": "layers",
        "diff_ids": [
            "sha256:9a4c0752c6bda5fc00f886a9acf0c0afed0750082f7999c331df978fb9f97d2a",
            "sha256:7c9fa2ad4016d142f1dbf3382d538a941590b97b598483f2137368bdb00746b1"
        ]
    }
}

This is the Nano Server configuration from the fourth part with four changes: the Cmd command and the WorkingDir working folder, a second diff_id, a second history entry and a new created time. \u003e in the history is the > character, which is how ConvertTo-Json in Windows PowerShell escapes it; the JSON means the same either way. The fields Hostname, AttachStdin and the like come from the Docker format and are not in the OCI specification. Runtimes do not read them, and we left them as they are in the base image.


Pushing to the registry

A push uses the same OCI Distribution protocol as a pull, only with different HTTP methods:

  1. HEAD /v2/<repository>/blobs/<digest>: does such a blob exist; 404 means it does not.
  2. POST /v2/<repository>/blobs/uploads/: open an upload. The registry responds with 202 Accepted and the upload address in the Location header.
  3. PUT <upload address> with the digest=<digest> query parameter and the content: upload the whole blob and state its hash. An address from Distribution already has the _state parameter, so digest is appended with &. The registry checks the hash and responds with 201 Created.
  4. When all the blobs are in place: PUT /v2/<repository>/manifests/<digest> for each manifest and PUT /v2/<repository>/manifests/<tag> for the index. The document type goes in the Content-Type header.

The first two steps by hand:

$r = 'http://127.0.0.1:5000/v2/hello'
$d = 'sha256:a502c8c4b023d5354cb562047c7651bacd41853927f10a6e0c8670446dd3020f'
curl.exe -s -o NUL -w "%{http_code}`n" -I "$r/blobs/$d"
curl.exe -s -o NUL -D - -X POST "$r/blobs/uploads/" | Select-String '^(HTTP|Location)'
404
HTTP/1.1 202 Accepted
Location: http://127.0.0.1:5000/v2/hello/blobs/uploads/01a1bc6d-0e5b-7245-a6ae-b6b4cd271e59?_state=7eL31B3lNkX2ym1XswZ2Xstak5dhMsfEtXZJaV98Gtl7Ik5hbWUiOiJoZWxsbyIsIlVVSUQiOiIwMWExYmM2ZC0wZTViLTcyNDUtYTZhZS1iNmI0Y2QyNzFlNTkiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjYtMTEtMDhUMTg6MzA6MjYuMTM5MTQ4OVoifQ%3D%3D

The hello repository does not exist yet, but there is no need to create it before uploading: the registry creates it by itself. The _state parameter in the upload address is the state of the upload, which the registry keeps on the client side. The rest is done by push-image.ps1:

push-image.ps1 C:\lab\hello 127.0.0.1:5000/hello:1.0
curl.exe -s http://127.0.0.1:5000/v2/_catalog
curl.exe -s http://127.0.0.1:5000/v2/hello/tags/list
curl.exe -s -o NUL -w "%{http_code} %{content_type}`n" -H 'Accept: application/vnd.oci.image.index.v1+json' http://127.0.0.1:5000/v2/hello/manifests/1.0
blob     sha256:a502c8c4b023d5354cb562047c7651bacd41853927f10a6e0c8670446dd3020f pushed, 808 bytes
blob     sha256:a7eba7a59d48cb7e97e4eeb9fc22cf6a070f73a398171e5da84ae4aecc15cbde pushed, 194222443 bytes
blob     sha256:3212d1316b9e369c8f9645adebd44a8a4da789e45b9b7b47567c0fe7e89dced8 pushed, 1416 bytes
manifest sha256:9d61883f0fb4339521b610da4c68a7dfeac211b22f927202e98049f5c7744169
blob     sha256:800c0b032d98b263578bc834260bb1fafd02a01d547bbe4ff5cd14079b6e5097 pushed, 806 bytes
blob     sha256:f402befcc0a67b05a1ef1e3681a5025acbd688a95ae05b89f0c529897b13d42b pushed, 122077144 bytes
blob     sha256:3f706888073bdac142fa61647800062d597fcb906288d068cb4e5efb1c7566d3 pushed, 1406 bytes
manifest sha256:d5b420e7398204228546758238bd1d5fd8c01d67ae613630729e15358f7383b7
manifest 1.0
{"repositories":["hello"]}
{"name":"hello","tags":["1.0"]}
200 application/vnd.oci.image.index.v1+json

The 1.0 tag points to the index, and the index points to the two manifests.


Pulling and running

First, nerdctl. The registry works over HTTP, without TLS, so nerdctl has to be allowed to do that with the --insecure-registry switch. We will download the base images from Microsoft's registry in advance, to see what comes from ours:

nerdctl pull mcr.microsoft.com/windows/nanoserver:ltsc2025
nerdctl pull mcr.microsoft.com/windows/nanoserver:ltsc2022
nerdctl pull --insecure-registry 127.0.0.1:5000/hello:1.0
nerdctl run --rm 127.0.0.1:5000/hello:1.0
nerdctl run --rm --platform 'windows(10.0.20348)/amd64' 127.0.0.1:5000/hello:1.0
Completed pull from OCI Registry (127.0.0.1:5000/hello:1.0)	elapsed: 0.4 s	total:  6.0 Ki	(13.3 KiB/s)
Hello from an image built by hand

Microsoft Windows [Version 10.0.26100.33674]
user: ContainerUser, computer: MINWINPC
Hello from an image built by hand

Microsoft Windows [Version 10.0.20348.5773]
user: ContainerUser, computer: MINWINPC

Only 6 KiB were downloaded: containerd already had the base layers, which have the same digests as in the Microsoft images we have just downloaded, and it took from the registry only the manifests, the configurations and our small layers. By default the variant for the host's build, 26100, started, and with the windows(10.0.20348)/amd64 platform, the variant on Nano Server 2022; Windows Server 2025 runs it with process isolation, which is nerdctl's default mode.

Now without containerd: we pull the image with pull-image.ps1 from the fourth part and run it with the script from the fifth. The -Attach switch of run-container.ps1 waits for the process and shows its output:

pull-image.ps1 127.0.0.1:5000/hello:1.0 C:\images\hello-back
run-container.ps1 hello C:\images\hello-back -Ip 172.30.0.60 -Attach
remove-container.ps1 hello
index sha256:ef26ca9d244f36b074c64aad467373a0dce20fd9804d1154a9ea0e74163d584d -> 10.0.26100.33674 sha256:9d61883f0fb4339521b610da4c68a7dfeac211b22f927202e98049f5c7744169
layer 1/2 sha256:9a4c0752c6bda5fc00f886a9acf0c0afed0750082f7999c331df978fb9f97d2a imported, 194222443 bytes
layer 2/2 sha256:7c9fa2ad4016d142f1dbf3382d538a941590b97b598483f2137368bdb00746b1 imported, 1416 bytes
127.0.0.1:5000/hello:1.0 -> C:\images\hello-back, 2 layers
volume \\?\Volume{e88cf2d6-c88f-436f-8ae1-4b49e1e76c5b}
endpoint 172.30.0.60 in nat, namespace db9a5373-c501-4de5-b88e-5b1fca1d5302
compute system hello
Hello from an image built by hand

Microsoft Windows [Version 10.0.26100.33674]
user: ContainerUser, computer: HELLO
exit code 0

pull-image.ps1 chose the variant from our index by os.version, verified the hashes and imported two layers, and run-container.ps1 assembled a container from them through HCS. The computer name is HELLO, from GuestOs.HostName in the HCS document. There is no Docker, containerd or BuildKit in this chain: the image was built, pushed, pulled and run by our own scripts on top of wclayer, hcsctl and curl.exe.


Cleaning up

We no longer need the images, layers, working folders and registry from this part. The registry keeps images only in memory and was needed only here, so we remove its task and folder too:

Set-Location C:\
nerdctl rmi 127.0.0.1:5000/hello:1.0 mcr.microsoft.com/windows/nanoserver:ltsc2025 mcr.microsoft.com/windows/nanoserver:ltsc2022
wclayer remove C:\lab\build\scratch
Get-ChildItem C:\images\*\layers -Directory | ForEach-Object { wclayer remove $_.FullName }
Remove-Item C:\images, C:\lab, C:\containers -Recurse -Force
Stop-ScheduledTask registry
Unregister-ScheduledTask registry -Confirm:$false
Remove-Item C:\registry -Recurse -Force

Part summary

  • A Windows image layer is the changes of a writable layer exported with wclayer export: Files with the new files and Hives with the delta registry hives. Export an unmounted layer; -z in hcsshim 0.15.0 produces an empty gzip, so you have to compress it yourself.
  • The configuration is the base image's configuration plus a diff_id, a history entry and a command. The manifest is the configuration and the layers from the bottom up. The index holds one manifest per os.version.
  • A push is HEAD, POST and PUT on the same /v2/ addresses as a pull; manifests go after the blobs, and the index goes under a tag.
  • Layer hashes change from build to build: the archive records times.
  • Distribution builds for Windows, but its filesystem storage does not work on the test machine; in-memory storage is enough for the experiment.

Sources


Next — the last isolation level, or rather its absence: HostProcess containers. Who creates them, what is left of a container in them and why their base image weighs 20 KB.

Only registered users can leave a comment.

Sign in or register to leave a comment.