Hyper-V. Part 5: Allocating processors and memory
All parts of the series “Hyper-V”In the first part the VM got two virtual processors and 4 GB of memory, and that was enough to install the system. Here we'll look at how Hyper-V divides processor time and memory between VMs: what the processor reserve, limit and weight mean, how dynamic memory works and what to do when the host runs short of memory, which of these settings change on a running VM, and how to give a VM its own Hyper-V.
Virtual processors
The number of virtual processors is set by Set-VMProcessor -Count, and it changes only on a VM that is turned off (Set-VMProcessor):
Stop-VM -Name app1
Set-VMProcessor -VMName app1 -Count 8
Start-VM -Name app1
Get-VMProcessor -VMName app1 | Format-List Count, Reserve, Maximum, RelativeWeight, CompatibilityForMigrationEnabled
A virtual processor is a thread that the hypervisor schedules on the host's logical processors. Hyper-V does not limit the ratio of virtual processors to logical ones, but every extra vCPU is an extra thread to schedule, and a VM with eight processors that uses two gains nothing from them while getting in its neighbours' way. A single VM cannot be given more than the host's number of logical processors (limits).
Reserve, limit and weight
How processor time is divided between VMs when there isn't enough for all of them is set by three parameters, and they change on the fly (same source):
-Reserve— the percentage of the VM's resources guaranteed to be reserved for it (0–100);-Maximum— the percentage the VM will not go above, even if the host is idle (0–100);-RelativeWeight— the VM's weight in the competition for the processor (1–10000, 100 by default): when processor time is short, a VM with weight 200 gets twice as much as one with weight 100.
Set-VMProcessor -VMName sql1 -Reserve 25 -RelativeWeight 200
Set-VMProcessor -VMName test1 -Maximum 50 -RelativeWeight 50
Get-VM | Get-VMProcessor | Format-Table VMName, Count, Reserve, Maximum, RelativeWeight
The percentages are of the VM's own resources, not the host's: -Maximum 50 on a VM with four processors means no more than two processors' worth of time. A reserve reduces what is left for everyone else: the host will not start a VM if its reserve cannot be provided.
SMT and NUMA
SMT (Hyper-Threading inside the guest): -HwThreadCountPerCore sets how many threads per core the VM sees; 0 means the same as the host, and this value is not supported on Windows Server 2016 (Set-VMProcessor). The guest needs this knowledge to lay out its threads across cores correctly.
NUMA. On a multiprocessor server memory is divided between sockets, and access to the memory of "its own" socket is faster than to another socket's. Hyper-V gives a large VM a virtual NUMA topology similar to the host's so that the guest also takes into account where the memory lives. -MaximumCountPerNumaNode limits the number of virtual processors in one virtual NUMA node, -MaximumCountPerNumaSocket the number of nodes in a socket; without a specific reason they are left as Hyper-V computed them from the host.
Get-VMHostNumaNode | Format-Table NodeId, ProcessorsAvailability, MemoryAvailable, MemoryTotal
Get-VMProcessor -VMName sql1 | Format-List HwThreadCountPerCore, MaximumCountPerNumaNode, MaximumCountPerNumaSocket
If a VM needs more memory than one NUMA node of the host has, it gets memory from several nodes. Set-VMHost -NumaSpanningEnabled $false forbids this, and then such a VM simply won't start — an option for those to whom predictable speed matters more (Set-VMHost).
Static memory
Without dynamic memory a VM gets exactly StartupBytes and holds it the whole time it runs. Starting with Windows Server 2016 the amount of static memory changes on a running VM of both generations if the guest is Windows Server 2016 or Windows 10 and later (what's new in Windows Server 2016). It can be reduced only down to the amount the guest manages to free:
Set-VMMemory -VMName app1 -StartupBytes 12GB
For SQL Server, Exchange and other services that manage memory themselves and take all they can see, static memory is usually the choice.
Dynamic memory
With dynamic memory a VM starts with StartupBytes, and from then on Hyper-V, through a driver in the guest, takes away unused memory down to MinimumBytes and adds memory when it runs short, up to MaximumBytes (Dynamic Memory overview). It is turned on and off on a VM that is turned off; on a running one you can only raise the maximum and lower the minimum:
Stop-VM -Name app2
Set-VMMemory -VMName app2 -DynamicMemoryEnabled $true -MinimumBytes 1GB -StartupBytes 2GB -MaximumBytes 8GB
Start-VM -Name app2
Set-VMMemory -VMName app2 -MaximumBytes 16GB # on the fly
Two more parameters (Set-VMMemory):
-Buffer— how many percent above the current demand to keep in reserve (from 5 to 2000, 20 by default). The buffer is there so that a spike in demand doesn't wait for Hyper-V to add memory;-Priority— the VM's weight when the host doesn't have enough memory for everyone (from 0 to 100, 50 by default). Important VMs get a higher weight, test ones a lower one.
To see how much a VM has been assigned and how much it asks for:
Get-VM | Format-Table Name, DynamicMemoryEnabled,
@{n='AssignedGB'; e={[math]::Round($_.MemoryAssigned/1GB, 1)}},
@{n='DemandGB'; e={[math]::Round($_.MemoryDemand/1GB, 1)}}, MemoryStatus
Get-VMMemory -VMName app2
MemoryDemand is how much the guest actually uses, MemoryAssigned is how much it has been given right now, that is, demand plus buffer. MemoryStatus shows OK, Low or Warning when the buffer is exhausted and the VM is short of memory.
Smart Paging
Dynamic memory creates an unpleasant case: a VM was running at its minimum, the host handed out the freed memory to neighbours, and then the VM was restarted — and it again needs its startup amount, which the host doesn't have. Then Hyper-V temporarily makes up the shortfall with a file on disk — this is Smart Paging; after boot the guest gives back the excess, and the file is no longer used (Dynamic Memory overview). It kicks in only when restarting a VM whose minimum is below its startup amount. The folder for the file is set by Set-VM -SmartPagingFilePath; the volume for it needs headroom:
Set-VM -Name app2 -SmartPagingFilePath 'D:\Hyper-V\SmartPaging'
Get-VM | Format-Table Name, SmartPagingFilePath, SmartPagingFileInUse
What changes on a running VM
The first question before any change is whether the VM will have to be turned off. A summary table for the whole series:
| Change | On the fly | Condition |
|---|---|---|
| Attach or detach a disk | yes | only on a SCSI controller |
| Expand a disk | yes | disk on a SCSI controller |
| Add a SCSI controller | no | |
| Add or remove a network adapter | yes | generation 2 only |
| Change the switch, VLAN, bandwidth | yes | |
| Change the amount of static memory | yes | guest Windows Server 2016 / Windows 10 and later |
| Raise the maximum or lower the minimum of dynamic memory | yes | |
| Turn dynamic memory on or off | no | |
| Number of virtual processors | no | |
| Processor reserve, limit and weight | yes | |
| Processor compatibility for migration | no | |
| Configuration version | no | |
| Generation | never |
Sources: hot add of network adapters and memory — in the feature compatibility table and in what's new in Windows Server 2016, dynamic memory — in the Dynamic Memory overview, disks — in Resize-VHD.
Nested virtualization
To run its own Hyper-V inside a VM — a lab for a cluster or containers with Hyper-V isolation — the VM is given the processor's virtualization extensions. The VM must be turned off for this (Set-VMProcessor, nested virtualization):
Stop-VM -Name lab1
Set-VMProcessor -VMName lab1 -ExposeVirtualizationExtensions $true
Set-VMNetworkAdapter -VMName lab1 -MacAddressSpoofing On
Start-VM -Name lab1
The second command is needed so that the VMs inside lab1 can reach the network with their own MAC addresses through an external switch — this is the switch from the part about networking. While Hyper-V runs inside a VM, its memory does not change on the fly, even if dynamic memory is enabled (nested virtualization).
Resource metering
Resource metering accumulates for a VM its average processor load, memory, disk usage and network traffic since metering was turned on (Measure-VM):
Enable-VMResourceMetering -VMName app1
Measure-VM -VMName app1
Reset-VMResourceMetering -VMName app1
This is not monitoring but a counter: it is read once per period — for example, once a month for a consumption report — and reset.
Looking at it by hand
VMs that are short of memory, and the difference between assigned and requested memory:
Get-VM | Where-Object State -eq 'Running' | Sort-Object MemoryDemand -Descending |
Format-Table Name, MemoryStatus,
@{n='DemandGB'; e={[math]::Round($_.MemoryDemand/1GB, 1)}},
@{n='AssignedGB'; e={[math]::Round($_.MemoryAssigned/1GB, 1)}},
@{n='MaxGB'; e={[math]::Round($_.MemoryMaximum/1GB, 1)}}
How many virtual processors are handed out against the host's logical processors:
$vcpu = (Get-VM | Where-Object State -eq 'Running' | Measure-Object ProcessorCount -Sum).Sum
$lp = (Get-VMHost).LogicalProcessorCount
"{0} vCPU on {1} logical processors, {2:N1} : 1" -f $vcpu, $lp, ($vcpu / $lp)
Sources
- Set-VMProcessor — number of processors, reserve, limit, weight, SMT, NUMA, nested virtualization;
- Plan for Hyper-V scalability — processor and memory limits;
- Set-VMHost — NUMA spanning;
- What's new in Windows Server 2016 — changing memory and network adapters on the fly;
- Hyper-V Dynamic Memory Overview — dynamic memory and Smart Paging;
- Set-VMMemory — memory parameters, buffer and priority;
- Hyper-V feature compatibility by generation and guest — what is supported in which generation and guest;
- Resize-VHD — changing disks on the fly;
- Nested virtualization — Hyper-V inside a VM;
- Measure-VM — resource metering.
A VM's processor is threads that the hypervisor divides by reserve, limit and weight, and its memory is either a fixed amount or a range within which Hyper-V redistributes it between VMs by itself.
Next — part 6: how checkpoints work. Standard and production checkpoints, the checkpoint tree, why deleting a checkpoint is a disk merge, what to do with orphaned .avhdx files and why a checkpoint is no substitute for a backup.
Only registered users can leave a comment.
Sign in or register to leave a comment.