Compute Scheduling

Overview

The Xloud Compute scheduler determines which hypervisor node receives every new instance, resize, live migration, evacuate, and unshelve request. It runs a two-phase pipeline: filters eliminate hosts that don’t meet requirements, then weighers rank the survivors and the top-scoring host wins. Administrators control this pipeline via nova.conf, flavor extra specs, host aggregate metadata, and scheduler hints.

Administrator Access Required — This operation requires the admin role. Contact your Xloud administrator if you do not have sufficient permissions.

Prerequisites

Admin credentials sourced from admin-openrc.sh
XDeploy admin access for nova.conf changes (requires service redeploy)
Understanding of host aggregates and flavors

How the Scheduler Works

Prefilters query the Placement service to reduce the candidate set before the main pipeline runs
Filters evaluate each remaining host — a host is eliminated if it fails any single filter
Weighers assign a numeric score to each surviving host
The scheduler randomly picks one host from the top host_subset_size hosts (default: 1)

Prefilters

Prefilters run before the filter chain and reduce candidate hosts via the Placement service. They are faster than regular filters because they cut candidates early.

Prefilter	Enabled By	Description
Compute Disabled Status	Always active	Excludes hosts with `COMPUTE_STATUS_DISABLED` trait — mirrors the enabled/disabled state of nova-compute services
Image Type Support	`[scheduler] query_placement_for_image_type_support = True`	Excludes hosts that don’t support the image’s disk format (e.g., hosts using Ceph backend cannot use QCOW2 images without expensive conversion)
Isolate Aggregates	`[scheduler] query_placement_for_traits_in_extra_specs = True`	Flavor and image traits must match aggregate metadata requirements before entering the main filter chain

Scheduler Filters

Filters are configured in nova.conf under [filter_scheduler]:

nova.conf — filter configuration

[filter_scheduler]
# All available filter classes (add custom filters here)
available_filters = nova.scheduler.filters.all_filters

# Filters that are actually applied (order matters — cheap filters first)
enabled_filters = ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,ServerGroupAntiAffinityFilter,ServerGroupAffinityFilter,NUMATopologyFilter

Filter order matters for performance. Place low-cost filters (e.g., ComputeFilter) before expensive ones (e.g., NUMATopologyFilter). A filter that eliminates many hosts early saves CPU on all subsequent filters.

Core Filters

ComputeFilter

Passes only hosts where the nova-compute service is operational and enabled. This filter should always be in enabled_filters — without it, instances may be scheduled to unreachable or disabled nodes.

ComputeCapabilitiesFilter

Matches flavor extra specs against host capabilities. Use the capabilities: namespace prefix in extra specs:

Require a specific hypervisor type via extra spec

openstack flavor set my-flavor \
  --property capabilities:hypervisor_type=kvm

Supported operators in extra spec values:

Operator	Meaning	Example
`=`	Greater than or equal (numeric)	`=4`
`==`	Exact match	`==kvm`
`!=`	Not equal	`!=xen`
`>=`	Greater than or equal	`>=8`
`<=`	Less than or equal	`<=16`
`s==`	String equal (default)	`s==kvm`
`<in>`	Substring match	`<in>intel`
`<all-in>`	All values present	`<all-in>avx,sse4`
`<or>`	One of a list	`<or>kvm <or>qemu`

Matchable host attributes: free_ram_mb, free_disk_mb, host, hypervisor_type, hypervisor_version, num_instances, num_io_ops, vcpus_total, vcpus_used

ImagePropertiesFilter

Filters hosts based on image metadata properties. Set these on the image to restrict which hosts can run it:

Image Property	Description	Example Value
`hw_architecture`	Required CPU architecture	`x86_64`, `aarch64`, `ppc64le`
`img_hv_type`	Required hypervisor type	`qemu` (covers KVM and QEMU)
`img_hv_requested_version`	Minimum hypervisor version	`>=6000`
`hw_vm_mode`	Required VM ABI mode	`hvm`, `xen`, `exe`

Require x86_64 architecture on an image

openstack image set my-image --property hw_architecture=x86_64

NUMATopologyFilter

Validates NUMA topology constraints. Required for CPU-pinned instances and instances with NUMA affinity requirements. If an instance has no NUMA requirement, any host passes. If a NUMA topology is defined via flavor extra specs, only hosts that can satisfy it pass.

Enable NUMA topology on a flavor

openstack flavor set pinned-flavor \
  --property hw:numa_nodes=2 \
  --property hw:cpu_policy=dedicated \
  --property hw:cpu_thread_policy=prefer

PciPassthroughFilter

Ensures the target host has available PCI devices matching the instance’s PCI device requests. Required for GPU passthrough and SR-IOV networking. Configure PCI device aliases in nova.conf and request them via flavor extra specs.

Server Group Filters

ServerGroupAffinityFilter

Enforces co-location — all instances in the server group land on the same host.

Create affinity group and launch instances

openstack server group create --policy affinity web-cluster

openstack server create \
  --image my-image --flavor m1.medium \
  --hint group=$(openstack server group show web-cluster -f value -c id) \
  web-1

ServerGroupAntiAffinityFilter

Enforces separation — instances in the group must be on different hosts. Essential for high-availability deployments.

Create anti-affinity group and launch instances

openstack server group create --policy anti-affinity ha-cluster

openstack server create \
  --image my-image --flavor m1.medium \
  --hint group=$(openstack server group show ha-cluster -f value -c id) \
  app-1

openstack server create \
  --image my-image --flavor m1.medium \
  --hint group=$(openstack server group show ha-cluster -f value -c id) \
  app-2

Verify placement: openstack server show app-1 -f value -c OS-EXT-SRV-ATTR:host and openstack server show app-2 -f value -c OS-EXT-SRV-ATTR:host should show different hosts.

Aggregate-Based Filters

AggregateInstanceExtraSpecsFilter

Matches flavor extra specs scoped with aggregate_instance_extra_specs: prefix against host aggregate metadata. The host must belong to an aggregate with matching metadata.

Create GPU aggregate and steer instances to it

openstack aggregate create gpu-hosts
openstack aggregate add host gpu-hosts compute-node-3
openstack aggregate set --property gpu=true gpu-hosts

openstack flavor set gpu-large \
  --property aggregate_instance_extra_specs:gpu=true

AggregateIoOpsFilter

Filters hosts exceeding the maximum concurrent I/O operations. The limit can be set globally in nova.conf or per-aggregate via metadata:

Set per-aggregate I/O limit

openstack aggregate set --property max_io_ops_per_host=4 storage-heavy-agg

Falls back to filter_scheduler.max_io_ops_per_host (default: 8). Uses the minimum value if the host belongs to multiple aggregates.

AggregateNumInstancesFilter

Filters hosts that have reached their instance count ceiling. Set per-aggregate or globally:

Limit aggregate to 20 instances per host

openstack aggregate set --property max_instances_per_host=20 small-agg

Falls back to filter_scheduler.max_instances_per_host (default: 50).

AggregateMultiTenancyIsolation

Isolates specific aggregates to specific tenants (projects). Hosts in an aggregate tagged with filter_tenant_id are reserved for those tenants only. Hosts outside such aggregates serve all tenants.

Restrict an aggregate to a specific project

openstack aggregate set \
  --property filter_tenant_id=<project-id> \
  dedicated-agg

AggregateTypeAffinityFilter

Restricts which flavors can launch on hosts in an aggregate. Set instance_type metadata on the aggregate to a flavor name or comma-separated list:

Restrict aggregate to specific flavors

openstack aggregate set \
  --property instance_type=m1.large,m1.xlarge \
  large-flavor-agg

Hosts not in any aggregate with this metadata accept all flavors.

Placement Hint Filters

SameHostFilter

Schedules the instance on the same host as one or more specified instances. Pass instance UUIDs via the same_host scheduler hint:

Launch on the same host as an existing instance

openstack server create \
  --image my-image --flavor m1.medium \
  --hint same_host=<existing-instance-uuid> \
  collocated-server

DifferentHostFilter

Schedules the instance on a different host from specified instances. Pass one or more UUIDs via different_host:

Avoid the same host as existing instances

openstack server create \
  --image my-image --flavor m1.medium \
  --hint different_host=<instance-uuid-1> \
  --hint different_host=<instance-uuid-2> \
  isolated-server

SimpleCIDRAffinityFilter

Places the instance on a host whose IP falls within a given CIDR range. Requires two hints:

Place near a specific subnet

openstack server create \
  --image my-image --flavor m1.medium \
  --hint build_near_host_ip=192.168.1.1 \
  --hint cidr=/24 \
  nearby-server

IsolatedHostsFilter

Defines isolated images that can only run on designated isolated hosts. Configure in nova.conf:

nova.conf — isolated hosts and images

[filter_scheduler]
isolated_hosts = compute-isolated-1,compute-isolated-2
isolated_images = <image-uuid-1>,<image-uuid-2>
restrict_isolated_hosts_to_isolated_images = true

Other Filters

IoOpsFilter

Filters hosts that exceed filter_scheduler.max_io_ops_per_host (default: 8) concurrent I/O-intensive tasks. Counted task states: build, resize, snapshot, migrate, rescue, unshelve.

NumInstancesFilter

Filters hosts that have reached filter_scheduler.max_instances_per_host (default: 50) running instances.

MetricsFilter

Works in conjunction with MetricsWeigher. Filters hosts that are not reporting the metrics specified in [metrics] weight_setting, preventing weigher failures from missing data.

AllHostsFilter

No-op filter — passes all hosts. Useful for testing or when all filtering is done by weighers.

Scheduler Weighers

Weighers assign a numeric score to each host that passed the filter chain. The final weight is the sum of all weighted scores:

final_weight = (multiplier_1 × normalized_score_1) + (multiplier_2 × normalized_score_2) + ...

Configure weighers in nova.conf:

nova.conf — weigher configuration

[filter_scheduler]
weight_classes = nova.scheduler.weights.all_weighers

# Pick randomly from the top N hosts (1 = always pick the best)
host_subset_size = 1

# Shuffle hosts with identical final weights
shuffle_best_same_weighed_hosts = false

Weigher Reference

Weigher	Default Multiplier	Default Behavior	Negative Multiplier Effect
`RAMWeigher`	`1.0`	Spread — prefer hosts with most free RAM	Stack — fill hosts before using new ones
`CPUWeigher`	`1.0`	Spread — prefer hosts with most free vCPUs	Stack — maximize vCPU density
`DiskWeigher`	`1.0`	Spread — prefer hosts with most free disk	Stack — fill disk before moving on
`IoOpsWeigher`	`-1.0`	Prefer lighter workload hosts	Prefer heavier workload hosts
`PCIWeigher`	`1.0`	Prefer hosts with matching PCI devices	(positive only — do not set negative)
`MetricsWeigher`	Configured via `[metrics]`	Custom metric-based ranking	Inverts metric preference
`ServerGroupSoftAffinityWeigher`	`1.0`	Pack group members on same host	(positive only)
`ServerGroupSoftAntiAffinityWeigher`	`1.0`	Spread group members across hosts	(positive only)
`BuildFailureWeigher`	`1000000.0`	Heavily penalizes hosts with recent build failures	—
`CrossCellWeigher`	`1000000.0`	Prefer same cell for resize/migrate	Prefer different cell
`HypervisorVersionWeigher`	`1.0`	Prefer newer hypervisor versions	Prefer older versions
`NumInstancesWeigher`	`0.0` (inactive)	Inactive — set positive to pack, negative to spread	—

BuildFailureWeigher and CrossCellWeigher have multipliers of 1,000,000 by default — intentionally enormous to override all resource-based weighers. Lower these values if hosts with transient failures are being overly penalized.

Per-Aggregate Weigher Overrides

Most weighers support per-aggregate multiplier overrides (set as aggregate metadata). The scheduler uses the minimum value when a host belongs to multiple aggregates with different values:

Set per-aggregate RAM weigher multiplier

openstack aggregate set \
  --property ram_weight_multiplier=-1.0 \
  pack-agg

Weigher	Aggregate Metadata Key
RAMWeigher	`ram_weight_multiplier`
CPUWeigher	`cpu_weight_multiplier`
DiskWeigher	`disk_weight_multiplier`
IoOpsWeigher	`io_ops_weight_multiplier`
PCIWeigher	`pci_weight_multiplier`
BuildFailureWeigher	`build_failure_weight_multiplier`
CrossCellWeigher	`cross_cell_move_weight_multiplier`
ServerGroupSoftAffinityWeigher	`soft_affinity_weight_multiplier`
ServerGroupSoftAntiAffinityWeigher	`soft_anti_affinity_weight_multiplier`

Allocation Ratios (Overcommit)

Allocation ratios control resource overcommit — how many virtual resources can be assigned per physical resource unit. These are applied by the Placement service and affect available resource inventory.

nova.conf — allocation ratios

[DEFAULT]
# vCPU overcommit (4 virtual CPUs per physical core)
cpu_allocation_ratio = 4.0

# RAM overcommit (1.0 = no overcommit)
ram_allocation_ratio = 1.0

# Disk overcommit (1.0 = no overcommit)
disk_allocation_ratio = 1.0

# Reserve these resources from the host (not available to instances)
reserved_host_cpus = 2
reserved_host_memory_mb = 2048
reserved_host_disk_mb = 10240

Ratio	Xloud Default	Description
`cpu_allocation_ratio`	`4.0`	Virtual CPUs per physical core
`ram_allocation_ratio`	`1.0`	Virtual RAM per physical GB
`disk_allocation_ratio`	`1.0`	Virtual disk per physical GB

For workloads with bursty but not sustained CPU demand (web servers, dev environments), cpu_allocation_ratio = 8.0 or higher is common. For memory-intensive databases, keep ram_allocation_ratio = 1.0 to avoid swap pressure.

Manage Ratios via Placement API

Manage allocation ratios per compute node without a nova.conf change using the osc-placement plugin:

Set per-node allocation ratio via Placement

# Get the resource provider UUID for a compute node
openstack resource provider list --name compute-node-1

# Update CPU allocation ratio for that node
openstack resource provider inventory set \
  --resource VCPU:allocation_ratio=8.0 \
  --amend <resource-provider-uuid>

# Update RAM allocation ratio
openstack resource provider inventory set \
  --resource MEMORY_MB:allocation_ratio=1.5 \
  --amend <resource-provider-uuid>

Host Aggregates

Host aggregates group compute nodes by shared hardware characteristics, location, or policy. The scheduler uses aggregate metadata to steer workloads to specific host groups.

Dashboard
CLI

Create a host aggregate

Navigate to Admin → Compute → Host Aggregates and click Create Host Aggregate. Enter a name and optionally assign it to an availability zone.

Add hosts

In the aggregate detail view, click Manage Hosts and select the compute nodes to include.

Set metadata

Click Set Metadata and add key-value pairs. Then set the matching extra spec on flavors so the AggregateInstanceExtraSpecsFilter routes instances here.

Set matching flavor extra spec

openstack flavor set gpu-large \
  --property aggregate_instance_extra_specs:gpu=true

openstack aggregate create \
  --zone production \
  high-memory-nodes

Availability Zones

Availability zones are user-visible logical groupings of hosts. Users select a zone at instance launch. All hosts default to the nova zone.

Create zone via aggregate

openstack aggregate create --zone rack-b rack-b-agg
openstack aggregate add host rack-b-agg <hostname1>
openstack aggregate add host rack-b-agg <hostname2>

List availability zones

openstack availability zone list --long

Use availability zones for user-visible placement (e.g., rack, room, region). Use host aggregates for internal scheduling policies (e.g., GPU nodes, high-memory nodes). Zones are visible to tenants; aggregates are not.

Compute Capabilities as Traits

nova-compute automatically reports COMPUTE_* traits to the Placement service based on the virt driver’s capabilities. These traits can be used in flavor or image requirements to steer scheduling. Common traits reported automatically:

Trait	Meaning
`COMPUTE_STATUS_DISABLED`	Host is disabled (always-active prefilter excludes these)
`COMPUTE_NET_ATTACH_INTERFACE`	Host supports attach/detach of network interfaces
`COMPUTE_VOLUME_MULTI_ATTACH`	Host supports multi-attach volumes
`COMPUTE_TRUSTED_CERTS`	Host supports trusted image certificates
`HW_CPU_X86_SVM`	AMD-V hardware virtualization present
`CUSTOM_IMAGE_TYPE_RBD`	Host supports native Ceph RBD image backend

List traits for a compute node

openstack resource provider list --name <hostname>
openstack resource provider trait list <resource-provider-uuid>

Require a trait via flavor

openstack flavor set my-flavor \
  --property trait:HW_CPU_X86_SVM=required

Configuration Reference

Option	Default	Description
`filter_scheduler.enabled_filters`	(see above)	Active filters in the pipeline
`filter_scheduler.host_subset_size`	`1`	Top N hosts to randomly choose from
`filter_scheduler.max_io_ops_per_host`	`8`	Max concurrent I/O tasks before filtering
`filter_scheduler.max_instances_per_host`	`50`	Max instances before filtering
`filter_scheduler.track_instance_changes`	`true`	Enable scheduler cache for instance changes
`filter_scheduler.shuffle_best_same_weighed_hosts`	`false`	Randomize among equal-weight hosts
`filter_scheduler.ram_weight_multiplier`	`1.0`	RAM weigher multiplier
`filter_scheduler.cpu_weight_multiplier`	`1.0`	CPU weigher multiplier
`filter_scheduler.disk_weight_multiplier`	`1.0`	Disk weigher multiplier
`filter_scheduler.io_ops_weight_multiplier`	`-1.0`	I/O ops weigher multiplier
`filter_scheduler.build_failure_weight_multiplier`	`1000000.0`	Build failure penalty weigher
`scheduler.query_placement_for_image_type_support`	`false`	Enable image type prefilter

Troubleshooting

No valid host was found (NoValidHost)

The most common scheduling failure. Check:

Available hosts: openstack compute service list — are all compute services up?
Quotas: openstack quota show --project <id> — has the project hit a limit?
Flavor requirements: Does the requested flavor fit on any available host?
Aggregate filters: Is the AggregateInstanceExtraSpecsFilter blocking all hosts?
Scheduler logs: docker logs nova_scheduler | grep "Filter.*rejected\|NoValidHost"

Check scheduler decision logs

docker exec nova_scheduler journalctl -u nova-scheduler | grep -i "filter\|no valid"

CPU microarchitecture mismatch during live migration

Error: guest CPU doesn't match specification: missing featuresCause: cpu_mode = host-model or host-passthrough exposes native CPU features. Different physical CPUs on different hosts (e.g., Intel Icelake vs. Cascadelake) expose different feature sets.Fix: Configure a common CPU baseline across all compute nodes:

nova.conf on all compute nodes

[libvirt]
cpu_mode = custom
cpu_model = Cascadelake-Server-noTSX

Use the lowest-common-denominator CPU model across your cluster. See Live Migration for details.

Instances always land on the same host

Cause: host_subset_size = 1 means the highest-scoring host always wins.Fix: Increase host_subset_size to pick randomly from the top N hosts, or adjust weigher multipliers to produce more differentiated scores:

nova.conf

[filter_scheduler]
host_subset_size = 3
shuffle_best_same_weighed_hosts = true

Aggregate filter blocking all hosts unexpectedly

If AggregateInstanceExtraSpecsFilter is enabled and a flavor has aggregate_instance_extra_specs:key=value, only hosts in aggregates with that metadata can receive the instance.

Check which hosts match a flavor's aggregate requirement

openstack aggregate list --long
# Verify the aggregate has the matching metadata key
openstack aggregate show <aggregate-name>

If you want certain flavors to run anywhere (not restricted to an aggregate), remove the aggregate_instance_extra_specs: prefix from the extra spec.

Next Steps

Compute Hosts

Enable, disable, and inspect the compute hosts the scheduler selects from

Flavor Management

Add extra specs and traits to flavors to steer workloads to matching aggregates

Live Migration

Configure CPU baselines and migration policies for cross-host moves

Advanced Features

CPU pinning, NUMA topology, huge pages, and PCI passthrough configuration

Resource Optimizer

Automate workload consolidation and rebalancing with the Xloud Resource Optimizer

Admin Guide

Return to the Compute Administration Guide index

Core Services

Other Services

Compute Scheduling

Overview

How the Scheduler Works

Prefilters

Scheduler Filters

Core Filters

Server Group Filters

Aggregate-Based Filters

Placement Hint Filters

Other Filters

Scheduler Weighers

Weigher Reference

Per-Aggregate Weigher Overrides

Allocation Ratios (Overcommit)

Manage Ratios via Placement API

Host Aggregates

Create a host aggregate

Add hosts

Set metadata

Availability Zones

Compute Capabilities as Traits

Configuration Reference

Troubleshooting

Next Steps

Compute Hosts

Flavor Management

Live Migration

Advanced Features

Resource Optimizer

Admin Guide

Core Services

Other Services

​Overview

​How the Scheduler Works

​Prefilters

​Scheduler Filters

​Core Filters

​Server Group Filters

​Aggregate-Based Filters

​Placement Hint Filters

​Other Filters

​Scheduler Weighers

​Weigher Reference

​Per-Aggregate Weigher Overrides

​Allocation Ratios (Overcommit)

​Manage Ratios via Placement API

​Host Aggregates

Create a host aggregate

Add hosts

Set metadata

​Availability Zones

​Compute Capabilities as Traits

​Configuration Reference

​Troubleshooting

​Next Steps

Compute Hosts

Flavor Management

Live Migration

Advanced Features

Resource Optimizer

Admin Guide

Overview

How the Scheduler Works

Prefilters

Scheduler Filters

Core Filters

Server Group Filters

Aggregate-Based Filters

Placement Hint Filters

Other Filters

Scheduler Weighers

Weigher Reference

Per-Aggregate Weigher Overrides

Allocation Ratios (Overcommit)

Manage Ratios via Placement API

Host Aggregates

Availability Zones

Compute Capabilities as Traits

Configuration Reference

Troubleshooting

Next Steps