Can This VPS Run Nested KVM? Execute One Guest Instruction

A VPS can show vmx or svm in /proc/cpuinfo and still be unable to run a nested KVM guest. The CPU flag is only the first layer. A usable result requires the provider to expose /dev/kvm, the current identity to open it, the device to implement the expected KVM API, and KVM_RUN to return from code executed by a virtual CPU.

This guide tests that whole path without downloading an image, booting Linux inside Linux, changing module parameters, or installing a hypervisor stack. A 100-line C program maps one page of guest memory containing the x86 HLT instruction. A pass is the exact exit KVM_EXIT_HLT. Two negative controls separately prove what permission denial and a missing device look like.

Use the result as a technical admission test before buying or committing a workload, not as a promise of production performance. The experiment proves one guest instruction can execute through KVM on this VPS at this moment. It does not benchmark CPU scheduling, approve a particular orchestration product, or replace a provider’s written support policy.

Define the Nested KVM Purchase Threshold

Nested virtualization means a virtual machine runs another virtual machine with hardware assistance. The Linux kernel nested-KVM guide names the physical hypervisor L0, the first guest L1, and a guest created inside L1 L2. In this buying test, the provider owns L0, the VPS you rent is L1, and your build worker, emulator, lab machine, or tenant guest would be L2.

A viable purchase threshold is therefore stronger than “the processor supports virtualization.” Require this evidence ladder:

1. L1 sees vmx on Intel or svm on AMD.
2. The KVM modules are active inside the kernel serving L1.
3. /dev/kvm exists and the workload identity can open it read/write.
4. KVM_GET_API_VERSION returns the userspace contract expected by current Linux headers.
5. The API can create a VM, register userspace memory, create a vCPU and enter KVM_RUN.
6. The recorded exit reason matches code placed in guest memory.

Those first three checks are prerequisites. The last three turn passive inventory into execution evidence. ArchWiki’s KVM guidance and the OpenStack KVM configuration reference both cover flags, modules and device permissions well. This experiment adds the small execution receipt a buyer needs before accepting the VPS.

If you only need isolated application processes rather than a second kernel, compare containers and virtual machines as different isolation and recovery boundaries first. Nested KVM adds another scheduler, memory boundary and operating system; it should solve a real requirement, not merely prove that it is possible.

Separate L0, L1 and L2 Ownership

Layer ownership prevents a common support dispute. You may have root in L1, but root there does not control the physical host’s BIOS, L0 module parameters, CPU model masking or device exposure. Commands such as modprobe kvm_intel nested=1 or modprobe kvm_amd nested=1 belong to the provider’s host unless L1 actually owns its kernel. Do not copy host-enablement steps into a production VPS and assume they can change L0.

Providers can expose the CPU extension while withholding /dev/kvm. They can expose the device to one group while your service runs as another identity. A platform may also allow nested execution but classify it as unsupported, best-effort or incompatible with live migration. Google Cloud’s nested virtualization overview is provider-specific, but its separation of supported machine types, restrictions and performance impact is a useful model for questions to ask any seller.

Inside L1, you own the userspace program and its configuration. Inside L2, you own the guest image and workload. Passing this article’s probe assigns one narrow fact to the right boundary: L1 successfully asked L0’s KVM interface to run the byte placed at L2 address zero. It does not transfer ownership of L0 behavior to you.

Inventory the Current VPS Without Changing It

Run the inventory as the same identity that will operate the nested workload when possible. id matters because /dev/kvm is normally controlled by a Unix group or ACL. The command stops when the CPU extension or device is absent; it does not install packages, load modules, change groups or edit permissions.

set -euo pipefail
LAB=/tmp/voxfor-kvm-admission
MARKER=voxfor-kvm-admission-v1

printf 'tested_utc=%s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
printf 'kernel=%s\n' "$(uname -r)"
printf 'virtualization=%s\n' "$(systemd-detect-virt)"
printf 'identity='; id
printf 'cpu_extension='; grep -m1 -Eo 'vmx|svm' /proc/cpuinfo
printf 'loaded_modules='
awk '$1 ~ /^kvm(_intel|_amd)?$/ {printf "%s%s", separator, $1; separator=","} END {print ""}' /proc/modules
stat -c 'device=%n mode=%a owner=%U group=%G major_minor=%t:%T' /dev/kvm

if test -e "$LAB"; then
  printf 'refusing_existing_path=%s\n' "$LAB" >&2
  exit 30
fi

Our reproduced environment reported Debian 13, Linux 6.12.96+deb13-amd64, virtualization type kvm, CPU extension svm, modules kvm_amd,kvm, and /dev/kvm mode 0660 owned by root:kvm. Your vendor, architecture and group can differ. The material requirement is not matching those labels; it is preserving the inventory beside the execution result.

No vmx/svm means the needed x86 virtualization extension is not visible in L1. A missing /dev/kvm means there is no device path for this program to open. A permission error means the path exists but this identity is not admitted. Those are different provider or identity conversations, so do not “fix” all three by making the device world-writable.

Build a Diskless One-Instruction Probe

This probe uses the documented KVM userspace API. It checks API version 12 from the installed Linux headers, checks userspace-memory support, creates one VM and one vCPU, maps one page, writes byte 0xf4 (HLT) at the initial instruction pointer, and enters KVM_RUN. It creates no disk, tap interface, bridge, firmware, persistent VM definition or background process.

One exact marker protects the temporary path. If it already exists, stop and inspect it rather than deleting somebody else’s files. Paste the complete block so the later hash identifies the program you actually built.

install -d -m 0755 "$LAB"
printf '%s\n' "$MARKER" > "$LAB/.owner"
cat > "$LAB/kvm_minimal_hlt.c" <<'EOF'
#define _GNU_SOURCE
#include <errno.h>
#include <fcntl.h>
#include <linux/kvm.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <unistd.h>

static int fail(const char *step) {
    printf("%s=fail errno=%d(%s)\n", step, errno, strerror(errno));
    return 20;
}

int main(int argc, char **argv) {
    const char *device = argc > 1 ? argv[1] : "/dev/kvm";
    int kvm = open(device, O_RDWR | O_CLOEXEC);
    if (kvm < 0) {
        printf("device_path=%s\n", device);
        return fail("device_open");
    }
    printf("device_path=%s\ndevice_open=ok\n", device);

    int api = ioctl(kvm, KVM_GET_API_VERSION, 0);
    if (api < 0) return fail("api_query");
    printf("api_version=%d\n", api);
    if (api != KVM_API_VERSION) {
        printf("acceptance=reject_api_version\n");
        return 21;
    }

    int user_memory = ioctl(kvm, KVM_CHECK_EXTENSION, KVM_CAP_USER_MEMORY);
    if (user_memory <= 0) {
        printf("cap_user_memory=%d\nacceptance=reject_user_memory\n", user_memory);
        return 22;
    }
    printf("cap_user_memory=%d\n", user_memory);

    int vm = ioctl(kvm, KVM_CREATE_VM, 0);
    if (vm < 0) return fail("vm_create");
    printf("vm_create=ok\n");

    size_t page_size = (size_t)sysconf(_SC_PAGESIZE);
    uint8_t *memory = mmap(NULL, page_size, PROT_READ | PROT_WRITE,
                           MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
    if (memory == MAP_FAILED) return fail("guest_memory_map");
    memory[0] = 0xf4; /* HLT */

    struct kvm_userspace_memory_region region = {
        .slot = 0,
        .guest_phys_addr = 0,
        .memory_size = page_size,
        .userspace_addr = (uint64_t)memory,
    };
    if (ioctl(vm, KVM_SET_USER_MEMORY_REGION, &region) < 0)
        return fail("guest_memory_register");

    int vcpu = ioctl(vm, KVM_CREATE_VCPU, 0);
    if (vcpu < 0) return fail("vcpu_create");
    printf("vcpu_create=ok\n");

    int run_size = ioctl(kvm, KVM_GET_VCPU_MMAP_SIZE, 0);
    if (run_size < (int)sizeof(struct kvm_run)) {
        errno = EPROTO;
        return fail("vcpu_mmap_size");
    }
    struct kvm_run *run = mmap(NULL, (size_t)run_size,
                               PROT_READ | PROT_WRITE, MAP_SHARED, vcpu, 0);
    if (run == MAP_FAILED) return fail("vcpu_run_map");

    struct kvm_sregs sregs;
    if (ioctl(vcpu, KVM_GET_SREGS, &sregs) < 0) return fail("get_sregs");
    sregs.cs.base = 0;
    sregs.cs.selector = 0;
    if (ioctl(vcpu, KVM_SET_SREGS, &sregs) < 0) return fail("set_sregs");

    struct kvm_regs regs = { .rip = 0, .rflags = 0x2 };
    if (ioctl(vcpu, KVM_SET_REGS, &regs) < 0) return fail("set_regs");
    if (ioctl(vcpu, KVM_RUN, 0) < 0) return fail("kvm_run");

    if (run->exit_reason != KVM_EXIT_HLT) {
        printf("guest_exit=unexpected:%u\nacceptance=reject_guest_execution\n",
               run->exit_reason);
        return 23;
    }
    printf("guest_exit=KVM_EXIT_HLT\n");
    printf("acceptance=hardware_kvm_execution\n");

    munmap(run, (size_t)run_size);
    close(vcpu);
    munmap(memory, page_size);
    close(vm);
    close(kvm);
    return 0;
}
EOF

gcc -std=c17 -O2 -Wall -Wextra -Werror \
  "$LAB/kvm_minimal_hlt.c" -o "$LAB/kvm_minimal_hlt"
sha256sum "$LAB/kvm_minimal_hlt.c"
printf 'compiler=%s\n' "$(gcc -dumpfullversion -dumpversion)"

Our exact published-block source hash was d76910db59b32c54a13e318ce516d9510377e24ba3e4446f1d76f428ef201843, built with GCC 14.2.0. A different hash means the source differs; it is not automatically wrong, but it must be reviewed and retained as its own artifact.

Run Both Failure Controls and the Positive Execution

First execute the same binary as UID/GID 65534 with supplementary groups cleared. On the reproduced host, /dev/kvm is group-restricted, so this deliberately unadmitted identity must fail at device_open with status 20. If it succeeds, your device access is broader than this model; inspect ACLs, udev rules and service identity before relying on the result.

set +e
setpriv --reuid=65534 --regid=65534 --clear-groups \
  "$LAB/kvm_minimal_hlt" /dev/kvm
DENIED_STATUS=$?
set -e
printf 'denied_status=%d\n' "$DENIED_STATUS"
test "$DENIED_STATUS" -eq 20

Now run as the admitted identity. Do not use sudo reflexively if the real service will not run as root: the relevant pass is the identity that will own the workload.

"$LAB/kvm_minimal_hlt" /dev/kvm

Finally point the binary at an intentionally nonexistent path. This proves that “device absent” produces a different errno from “device present but denied.”

set +e
"$LAB/kvm_minimal_hlt" /dev/voxfor-kvm-missing
MISSING_STATUS=$?
set -e
printf 'missing_status=%d\n' "$MISSING_STATUS"
test "$MISSING_STATUS" -eq 20

Here is the representative observed receipt:

=== denied identity negative control ===
device_path=/dev/kvm
device_open=fail errno=13(Permission denied)
denied_status=20
=== admitted identity execution ===
device_path=/dev/kvm
device_open=ok
api_version=12
cap_user_memory=1
vm_create=ok
vcpu_create=ok
guest_exit=KVM_EXIT_HLT
acceptance=hardware_kvm_execution
=== missing device negative control ===
device_path=/dev/voxfor-kvm-missing
device_open=fail errno=2(No such file or directory)
missing_status=20

That sequence is meaningful because the failures own different layers and the positive path crosses all of them. KVM_EXIT_HLT is not decorative output: the program accepts it only after the API, memory region, VM, vCPU and KVM_RUN calls succeed.

Verify the exact positive predicates again from a captured receipt rather than visually scanning a long terminal. This command does not hide the raw output; retain both.

POSITIVE_RECEIPT="$("$LAB/kvm_minimal_hlt" /dev/kvm)"
printf '%s\n' "$POSITIVE_RECEIPT"
grep -qx 'api_version=12' <<<"$POSITIVE_RECEIPT"
grep -qx 'cap_user_memory=1' <<<"$POSITIVE_RECEIPT"
grep -qx 'vm_create=ok' <<<"$POSITIVE_RECEIPT"
grep -qx 'vcpu_create=ok' <<<"$POSITIVE_RECEIPT"
grep -qx 'guest_exit=KVM_EXIT_HLT' <<<"$POSITIVE_RECEIPT"
grep -qx 'acceptance=hardware_kvm_execution' <<<"$POSITIVE_RECEIPT"
printf 'verification=accepted_api_device_vm_vcpu_and_guest_exit\n'

Accept nested KVM execution only when the denied identity returns errno 13/status 20, the missing path returns errno 2/status 20, and the admitted identity records API 12, userspace-memory capability, VM creation, vCPU creation, KVM_EXIT_HLT, and acceptance=hardware_kvm_execution. Any missing or contradictory predicate is a rejection, not a partial pass.

Convert the Receipt Into a Buy, Reject or Escalate Decision

A useful receipt should change the purchase decision. It should not become a screenshot that sales and engineering interpret differently.

Ask the seller to answer in writing:

• Is nested KVM supported on this exact plan, region and CPU family, or merely visible today?
• Is /dev/kvm intentionally exposed after stop/start, resize, migration and host maintenance?
• Which nested workloads are supported: CI VMs, Android emulators, libvirt, QEMU, Firecracker, Proxmox or none contractually?
• Are CPU model, nested features, overcommit and live migration stable enough for the workload?
• Which support team owns a regression when the receipt passes before maintenance and fails afterward?

If the provider cannot own those answers, a current technical pass is still useful for a disposable lab, but it is weak evidence for a durable service commitment. The broader hypervisor foundation guide helps place Type 1/Type 2 language, KVM and resource isolation around this specific buying test.

Keep Performance and Workload Fitness Outside This Pass

One HLT instruction proves execution, not throughput. Nested workloads add scheduling layers; noisy neighbors and L0 overcommit can affect L1 while L2 adds its own run queues. After functional admission, measure VPS CPU steal time under a controlled workload and define percentiles, duration and rejection thresholds appropriate to the job.

Storage is also untested. An emulator that boots or a CI guest that creates layers can be limited by tail latency even when KVM execution is perfect. Run a bounded FIO VPS disk-latency percentile benchmark on disposable data. If the design uses QCOW2 overlays, understand which files a rebase or commit changes in a backing chain before treating image operations as harmless.

Memory admission needs more than KVM_CAP_USER_MEMORY. Size L1 for the host userspace process, every L2 guest, page cache and failure headroom. If ballooning is part of the design, the KVM balloon, guest-pressure and host-swap evidence chain shows why a target value is not the same as memory actually reclaimed.

Networking, device passthrough, kernel modules, huge pages, confidential-computing modes, snapshots and live migration remain product-specific. QEMU can fall back to TCG software emulation when KVM is unavailable, but that is a different execution and performance path. Do not let “the guest booted” conceal which accelerator ran it; record the effective accelerator and reject silent emulation when hardware KVM is a requirement.

Remove Only the Marker-Owned Fixture

This lab created one source file, one binary and one ownership marker. Confirm the marker value before deletion. Do not unload KVM modules, change /dev/kvm, remove users from groups or delete unrelated temporary directories as cleanup.

test -f "$LAB/.owner"
test "$(cat "$LAB/.owner")" = "$MARKER"
find "$LAB" -mindepth 1 -maxdepth 1 -type f -delete
rmdir "$LAB"
test ! -e "$LAB"
printf 'cleanup=owned_fixture_absent\n'

If the experiment stops early, delete only /tmp/voxfor-kvm-admission after its .owner file exactly matches voxfor-kvm-admission-v1, then require the path to be absent. The probe makes no provider, module, device-permission or persistent VM changes, so rollback must not invent any. If you separately changed a group, ACL or service definition, restore that change from its own recorded baseline and rerun the workload identity check.