Testing Framework

Aegis uses a Rust-based integration test suite that boots the actual kernel in QEMU, captures serial output, and asserts on boot behavior, subsystem initialization, GUI rendering, and end-to-end installer flows. There are no unit tests for the C kernel code – all testing is full-system integration testing against QEMU.

v1 note: The test framework is v1 and growing. Contributions are welcome – file issues or propose changes at exec/aegis.

Architecture Overview

┌─────────────────────────────────────────────────────────┐
│                    cargo test                           │
│                                                         │
│  ┌──────────────┐  ┌──────────────┐  ┌───────────────┐  │
│  │ boot_oracle  │  │ login_flow   │  │ installer     │  │
│  │    .rs       │  │   _test.rs   │  │   _test.rs    │  │
│  └──────┬───────┘  └──────┬───────┘  └──────┬────────┘  │
│         │                 │                 │           │
│         ▼                 ▼                 ▼           │
│  ┌─────────────────────────────────────────────────┐    │
│  │              aegis_tests library                │    │
│  │  ┌──────────┐ ┌──────────┐ ┌────────────────┐   │    │
│  │  │ harness  │ │ assert   │ │   presets      │   │    │
│  │  │          │ │          │ │                │   │    │
│  │  │ boot()   │ │ subsys   │ │ aegis_pc()     │   │    │
│  │  │ boot_    │ │ boot_seq │ │ aegis_q35()    │   │    │
│  │  │ stream() │ │ line_    │ │ graphical()    │   │    │
│  │  │ boot_    │ │ contains │ │ installer()    │   │    │
│  │  │ disk_    │ │ wait_for │ │ installed()    │   │    │
│  │  │ only()   │ │ _line()  │ │                │   │    │
│  │  └────┬─────┘ └──────────┘ └────────────────┘   │    │
│  │       │                                         │    │
│  │  ┌────┴─────┐                                   │    │
│  │  │  image   │  PPM loader + fuzzy compare       │    │
│  │  └──────────┘                                   │    │
│  └────────────────────────┬────────────────────────┘    │
│                           │                             │
│                           ▼                             │
│                    ┌──────────────┐                     │
│                    │    Vortex    │                     │
│                    │  QEMU mgmt   │                     │
│                    └──────┬───────┘                     │
│                           │                             │
└───────────────────────────┼─────────────────────────────┘
                            ▼
                     ┌──────────────┐
                     │    QEMU      │
                     │              │
                     │ ┌──────────┐ │
                     │ │ Aegis OS │ │
                     │ └──────────┘ │
                     └──────────────┘

Dependencies

[package]
name = "aegis-tests"
version = "0.1.0"
edition = "2021"

[dependencies]
vortex = { path = "../../vortex", features = ["qemu"] }
tokio = { version = "1", features = ["full"] }
chrono = "0.4"

Vortex is the QEMU management library that handles VM lifecycle, serial capture, monitor socket communication, screendumps, and keyboard/mouse injection. The qemu feature enables the QEMU backend. All tests are async and run on the Tokio runtime.

Running Tests

Quick Start

# Build ISO first (required by all tests)
make iso

# Run all tests
make test

# Run with q35 preset (includes NVMe disk)
make test-q35

# Run a specific test
cargo test --manifest-path tests/Cargo.toml boot_oracle -- --nocapture

# Run installer tests (requires test-iso + OVMF)
make test-iso
AEGIS_INSTALLER_ISO=build/aegis-test.iso cargo test \
    --manifest-path tests/Cargo.toml --test installer_test -- --nocapture

Environment Variables

Test Harness (harness.rs)

The AegisHarness struct provides three boot modes:

AegisHarness::boot(opts, iso)

Boots QEMU, collects all serial output until QEMU exits or the boot timeout elapses, then returns the complete ConsoleOutput. QEMU is killed on timeout.

pub async fn boot(opts: QemuOpts, iso: &Path) -> Result<ConsoleOutput, HarnessError>

Used by tests that only need to verify boot output (e.g., boot_oracle).

AegisHarness::boot_stream(opts, iso)

Boots QEMU and returns a live ConsoleStream + QemuProcess handle. The caller drives the test interactively – waiting for specific output, sending keystrokes, capturing screenshots – and is responsible for killing the process.

pub async fn boot_stream(
    opts: QemuOpts,
    iso: &Path,
) -> Result<(ConsoleStream, QemuProcess), HarnessError>

Used by interactive tests: login flows, installer tests, GUI tests.

AegisHarness::boot_disk_only(opts)

Boots QEMU from a persistent disk image with no ISO. Used for the second boot in installer test sequences – verifying that the installed system boots standalone from NVMe via UEFI.

pub async fn boot_disk_only(
    opts: QemuOpts,
) -> Result<(ConsoleStream, QemuProcess), HarnessError>

Exit Code Mappings

The harness configures QEMU’s isa-debug-exit device at I/O port 0xf4. The kernel can write to this port to signal test outcomes:

Boot Timeout

Default: 30 seconds (configurable via AEGIS_BOOT_TIMEOUT). On timeout, the harness kills the QEMU process and returns whatever output was collected.

Assertion Library (assert.rs)

Collected-Output Assertions

These operate on the complete ConsoleOutput returned by AegisHarness::boot():

assert_subsystem_ok(out, subsystem)

Asserts that the kernel output contains [<subsystem>] OK. Panics with full serial capture on failure.

assert_subsystem_ok(&out, "PMM");   // passes if "[PMM] OK: ..." is in output
assert_subsystem_ok(&out, "SCHED"); // passes if "[SCHED] OK: ..." is in output

assert_subsystem_fail(out, subsystem)

Asserts that [<subsystem>] FAIL appears in kernel output. Used to verify expected error conditions.

assert_boot_subsequence(out, expected)

Asserts that all strings in expected appear in the kernel output in order, with arbitrary gaps allowed between matches. This is the primary boot-sequence oracle assertion.

// Passes: gaps between PMM and SCHED are allowed
assert_boot_subsequence(&out, &["[PMM] OK", "[SCHED] OK"]);

// Fails: wrong order
assert_boot_subsequence(&out, &["[SCHED] OK", "[PMM] OK"]);

assert_line_contains(out, substr) / assert_no_line_contains(out, substr)

Positive and negative substring assertions across all output lines (not just kernel lines).

Streaming Assertion

wait_for_line(stream, pattern, timeout)

Consumes lines from a live ConsoleStream until one containing pattern is found, or the timeout expires. Returns the matching line on success, WaitTimeout on failure.

wait_for_line(&mut stream, "[BASTION] greeter ready", Duration::from_secs(30))
    .await
    .expect("[BASTION] greeter ready never fired within 30s");

This is the primary synchronization primitive for interactive tests – it gates on specific kernel/application log lines before proceeding to the next test step.

QEMU Presets (presets.rs)

Presets configure the virtual machine hardware for different test scenarios:

aegis_pc() – Minimal PC

Machine:  pc (i440FX)
Display:  none
VGA:      std
CPU:      Broadwell
Devices:  isa-debug-exit only
Serial:   captured
Monitor:  disabled

Minimal configuration for boot-sequence oracle tests. No NVMe, no networking, no USB.

aegis_q35() – Full Q35

Machine:  q35 (ICH9)
Display:  none
VGA:      std
CPU:      Broadwell
Devices:  xHCI + USB keyboard + USB mouse + NVMe + virtio-net
Drives:   NVMe backed by build/disk.img
Network:  User-mode with port forwarding (SSH:2222, HTTP:8080)
Serial:   captured
Monitor:  disabled

Full hardware simulation matching production configuration. Used with AEGIS_PRESET=q35.

aegis_q35_graphical_mouse() – GUI Testing

Machine:  q35 (ICH9)
Display:  VNC (127.0.0.1:17, walk to :99)
VGA:      virtio-vga (required for framebuffer mapping)
CPU:      Broadwell
Devices:  virtio-vga only (no USB keyboard/mouse)
Serial:   captured
Monitor:  enabled (HMP socket)

Key design: No USB HID devices. HMP sendkey and mouse_move route to the first keyboard/mouse device on the machine. On q35, the ICH9 LPC provides PS/2 keyboard and AUX mouse by default. Adding usb-kbd would intercept HMP events, preventing them from reaching the PS/2 drivers that Aegis actually reads from (/dev/kbd, /dev/mouse).

The VNC display is required instead of -display none because QEMU’s screendump HMP command produces all-black output when no display backend renders the VGA surface.

aegis_q35_installer(disk_path) – Text Installer

Machine:  q35
Display:  none
VGA:      std
Devices:  NVMe (caller-supplied disk path)
Serial:   captured
Monitor:  enabled (for sendkey)

aegis_q35_gui_installer(disk_path) – GUI Installer

Machine:  q35
Display:  VNC (127.0.0.1:19)
VGA:      virtio-vga (required for framebuffer)
Devices:  virtio-vga + NVMe
Serial:   captured
Monitor:  enabled

aegis_q35_installed_ovmf(disk_path, ovmf_path) – Post-Install UEFI Boot

Machine:  q35
Display:  none
VGA:      std
Devices:  NVMe
Drives:   OVMF pflash firmware (read-only) + NVMe disk
Serial:   captured
Monitor:  enabled

Used for Boot 2 of installer tests. Verifies the installed system boots from the NVMe drive via UEFI (OVMF firmware).

Visual Regression Testing (image.rs)

The image module provides fuzzy PPM comparison for screendump-based GUI tests.

PPM Format

QEMU’s screendump HMP command outputs P6 PPM files (binary RGB, 8-bit per channel, maxval 255). The parser handles the format directly without external image libraries.

Fuzzy Comparison

Exact pixel comparison is too brittle for animated UI (cursor blink, clock updates, focus rings). The comparison uses two thresholds:

pub fn assert_ppm_matches(actual: &Path, reference: &Path)

Panics with detailed diagnostics (mean diff, bad pixel count/percentage) when either threshold is exceeded.

Reference Screenshot Workflow

1. First run: No reference exists; the captured screenshot becomes the reference
2. Subsequent runs: Captures are compared against the stored reference
3. Update mode: Set AEGIS_UPDATE_SCREENSHOTS=1 to regenerate all references

Reference screenshots are stored in tests/screenshots/.

Test Suite

boot_oracle – Boot Sequence Verification

File: tests/tests/boot_oracle.rs

Boots on the minimal pc machine and asserts the kernel’s subsystem initialization sequence matches the expected boot oracle. The oracle is a subsequence of 32 kernel log lines covering all major subsystems from serial init through scheduler start:

[SERIAL] OK → [VGA] OK → [PMM] OK → [VMM] OK → [KVA] OK →
[CAP] OK → [IDT] OK → [PIC] OK → [PIT] OK → [KBD] OK →
[MOUSE] OK → [GDT] OK → [TSS] OK → [SYSCALL] OK → [SMAP] OK →
[SMEP] OK → [RNG] OK → [RAMDISK] OK → [VFS] OK → [INITRD] OK →
[ACPI] OK → [LAPIC] OK → [IOAPIC] OK → [PCIE] OK → [EXT2] OK →
[CAP_POLICY] OK → [POLL] OK → [SMP] OK → [CAP] OK (baseline) →
[VMM] OK (identity map removed) → [SCHED] OK

Also includes a negative test: on pc (no virtio-net), no [NET] lines should appear.

poll_test – Kernel Poll Self-Test

File: tests/tests/poll_test.rs

Boots on pc and asserts [POLL] OK appears in kernel output, verifying the kernel’s built-in poll() self-test passed.

screendump_test – Framebuffer Capture

File: tests/tests/screendump_test.rs

Boots graphically (q35 + virtio-vga), waits for [BASTION] greeter ready, captures a screendump, and verifies the PPM file is at least 900 KB (not truncated). Validates the QEMU screendump pipeline works end-to-end.

login_flow_test – Login + Desktop Verification

File: tests/tests/login_flow_test.rs

Full graphical login flow:

1. Boot graphically, wait for [BASTION] greeter ready
2. Capture Bastion greeter screenshot, compare against reference
3. Send credentials: root<Tab>forevervigilant<Enter> via HMP sendkey
4. Wait for [LUMEN] ready
5. Capture Lumen desktop screenshot, compare against reference

Uses fuzzy PPM comparison for visual regression detection.

dock_click_test – GUI Interaction Testing

File: tests/tests/dock_click_test.rs

The most complex GUI test. After login:

1. Parse [DOCK] item=<key> cx=<x> cy=<y> lines from serial output to discover dock icon coordinates
2. For each testable dock item (terminal, widgets):
   • Home the mouse to (0,0) with multiple small negative-delta hops
   • Move to the target icon, compensating for Lumen’s 1.5x mouse speed multiplier (send 2/3 of the desired delta)
   • Click (press + 80ms + release)
   • Wait for [LUMEN] window_opened=<key> confirmation
   • Capture post-click screenshot

The mouse homing algorithm accounts for QEMU PS/2 emulation limits (~635 delta units per poll cycle) by using 4 hops of (-500, -500) with 200ms settle between each.

installer_test – Text Installer End-to-End

File: tests/tests/installer_test.rs

Two-boot test sequence:

Boot 1 (text-mode ISO + empty NVMe):

1. Boot from aegis-test.iso (text mode)
2. Log in as root
3. Launch installer
4. Drive prompts via HMP sendkey: confirm disk, set root password, create user account
5. Wait for === Installation complete! ===

Boot 2 (OVMF + installed NVMe, no ISO):

1. Boot via UEFI firmware from the NVMe disk
2. Assert [EXT2] OK: mounted nvme0p1 appears (proves UEFI boot + GRUB + ext2 mount all work)

Requires make test-iso and OVMF (apt install ovmf).

gui_installer_test – GUI Installer End-to-End

File: tests/tests/gui_installer_test.rs

Same two-boot pattern as installer_test, but drives the graphical installer wizard:

1. Navigate through wizard screens via Tab/Enter (5 screens total)
2. Fill user account form: root password, confirm, username, user password, confirm
3. Wait for [INSTALLER] done sentinel
4. Boot 2 verifies UEFI boot from installed NVMe

Uses aegis_q35_gui_installer preset (virtio-vga required for framebuffer mapping).

mouse_api_smoke_test – HMP Mouse Round-Trip

File: tests/tests/mouse_api_smoke_test.rs

Smoke test verifying Vortex’s mouse_move and mouse_button HMP methods execute without error against a live QEMU instance. Does not verify guest-side delivery – that is covered by dock_click_test.

Writing New Tests

Basic Boot Test

use aegis_tests::{aegis_pc, iso, AegisHarness, assert_subsystem_ok};

#[tokio::test]
async fn my_subsystem_boots() {
    let iso = iso();
    if !iso.exists() {
        eprintln!("SKIP: {} not found", iso.display());
        return;
    }
    let out = AegisHarness::boot(aegis_pc(), &iso)
        .await
        .expect("QEMU failed to start");
    assert_subsystem_ok(&out, "MY_SUBSYSTEM");
}

Interactive Test

use aegis_tests::{aegis_q35_graphical_mouse, iso, wait_for_line, AegisHarness};
use std::time::Duration;

#[tokio::test]
async fn my_interactive_test() {
    let iso = iso();
    if !iso.exists() { return; }

    let (mut stream, mut proc) =
        AegisHarness::boot_stream(aegis_q35_graphical_mouse(), &iso)
            .await
            .expect("QEMU failed to start");

    // Wait for a kernel/app log line
    wait_for_line(&mut stream, "[MY_APP] ready", Duration::from_secs(30))
        .await
        .expect("app never ready");

    // Send keystrokes via HMP
    proc.send_keys("hello\n").await.expect("sendkey failed");

    // Capture screenshot
    let out = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"))
        .join("screenshots/my_test.ppm");
    proc.screendump(&out).await.expect("screendump failed");

    proc.kill().await.unwrap();
}

Graceful Skip Pattern

All tests follow a consistent skip pattern for missing prerequisites:

if !iso.exists() {
    eprintln!("SKIP: {} not found — run `make iso` first", iso.display());
    return;
}

This allows cargo test to run without failing when build artifacts are absent – tests that cannot run simply print a skip message and return success.