Syscall Interface

Aegis implements a Linux-compatible syscall interface using the SYSCALL/SYSRET instructions on x86-64 and SVC/ERET on ARM64. The kernel provides over 100 syscalls covering file I/O, process management, memory management, networking, signals, and Aegis-specific extensions. User-space code (compiled against musl libc) issues standard Linux syscall numbers.

v1 maturity note: Aegis is v1 software – the first version deemed ready for public release, not a mature or production-hardened system. The syscall interface is the kernel’s primary attack surface: every syscall handler parses untrusted user input in C. While user pointer validation and kernel staging buffers are used throughout, there should be no expectation of real, battle-tested security. There are likely exploitable vulnerabilities in individual syscall implementations, as would be expected in any from-scratch C kernel of this scale. A gradual migration from C to Rust is planned starting with the kernel; the capability system (kernel/cap/) is already in Rust and represents the beginning of this path. Contributions are welcome – file issues or propose changes at exec/aegis.

Entry Mechanism

x86-64: SYSCALL/SYSRET

The syscall entry point is configured during boot by arch_syscall_init() (kernel/arch/x86_64/arch_syscall.c):

wrmsr(IA32_EFER, rdmsr(IA32_EFER) | 1UL);     /* Enable SCE bit */
wrmsr(IA32_STAR, (ARCH_KERNEL_DS << 48) | (ARCH_KERNEL_CS << 32));
wrmsr(IA32_LSTAR, (uint64_t)syscall_entry);     /* Entry point */
wrmsr(IA32_SFMASK, 0x700UL);                    /* Clear IF, TF, DF */

CPU state on SYSCALL entry:

• RAX = syscall number
• RDI = arg1, RSI = arg2, RDX = arg3, R10 = arg4, R8 = arg5, R9 = arg6
• RCX = return RIP, R11 = saved RFLAGS
• RSP = user stack (unchanged by CPU)
• IF=0 (interrupts disabled by SFMASK)

Syscall Entry Assembly (syscall_entry.asm)

1. SWAPGS                          -- switch GS to percpu_t
2. Save user RSP to gs:32          -- percpu.user_rsp_scratch
3. Load kernel RSP from gs:24      -- percpu.kernel_stack
4. Push restore frame:             -- user_rsp, rcx (RIP), r11 (RFLAGS)
5. Push callee-saved regs:         -- r15, r14, r13, r12, rbp, rbx
6. Push r8, r9, r10                -- forms syscall_frame_t
7. Shuffle to SysV 8-arg call:
     rdi=frame, rsi=num, rdx=arg1, rcx=arg2,
     r8=arg3, r9=arg4, [rsp+8]=arg5, [rsp+16]=arg6
8. call syscall_dispatch
9. Check for pending signals
10. Restore registers, SWAPGS, SYSRET

Syscall Frame

The registers pushed onto the kernel stack form a syscall_frame_t, used by sys_fork, sys_execve, and signal delivery to read or modify the user register state:

/* x86-64 */
typedef struct syscall_frame {
    uint64_t r10;       /* +0:  arg4 */
    uint64_t r9;        /* +8:  arg6 */
    uint64_t r8;        /* +16: arg5 */
    uint64_t rbx;       /* +24: callee-saved */
    uint64_t rbp;       /* +32: callee-saved */
    uint64_t r12;       /* +40: callee-saved */
    uint64_t r13;       /* +48: callee-saved */
    uint64_t r14;       /* +56: callee-saved */
    uint64_t r15;       /* +64: callee-saved */
    uint64_t rflags;    /* +72: saved RFLAGS */
    uint64_t rip;       /* +80: return RIP */
    uint64_t user_rsp;  /* +88: saved user RSP */
} syscall_frame_t;

ARM64: SVC/ERET

On ARM64, user-space issues SVC #0. The exception vector in vectors.S saves all 31 general-purpose registers plus SP_EL0, ELR_EL1, and SPSR_EL1 (34 slots), then calls syscall_dispatch. ARM64 uses different syscall numbers than x86-64; the dispatch function translates them before the main switch table.

CR3 Policy

The SYSCALL path does not switch CR3. The user PML4 remains loaded throughout syscall_dispatch so syscalls can directly dereference user virtual addresses. This is safe because:

1. The user PML4 shares the kernel higher-half (PML4[511]) for all kernel code and KVA stacks
2. Timer interrupts via isr_common_stub switch to master PML4 before ISR dispatch
3. sched_exit switches to master PML4 before context-switching away

Dispatch Table

syscall_dispatch() in kernel/syscall/syscall.c is the central dispatch function. On ARM64, it first translates ARM64 syscall numbers to x86-64 equivalents, then dispatches via a switch statement.

ARM64 Translation Layer

ARM64 musl emits ARM64-native syscall numbers. The dispatch function maps them to x86-64 numbers before the main switch. Examples:

The *at variants (openat, mkdirat, renameat2, etc.) are translated to their non-at equivalents by stripping the dirfd argument (Aegis uses absolute/CWD-relative paths).

User Pointer Validation

All syscalls validate user pointers before access:

static inline int user_ptr_valid(uint64_t addr, uint64_t len) {
    return len <= USER_ADDR_MAX && addr <= USER_ADDR_MAX - len;
}

This is overflow-safe. Data is copied between user and kernel space via copy_from_user / copy_to_user helpers. Writes use a kernel staging buffer to prevent TOCTOU attacks (user could modify or unmap the buffer between validation and use).

These are v1 mitigations – necessary but not sufficient for a hardened kernel. The validation confirms the pointer is in the canonical user address range but does not verify the page is actually mapped or that the access won’t fault. Individual syscalls may have additional validation gaps. Security audit findings to date have been hypothetical, but real exploitable bugs in argument handling almost certainly exist across the 100+ syscall implementations.

Capability Gates

Many syscalls check the calling process’s capability table before proceeding. Each entry in the capability table pairs a capability kind (CAP_KIND_*) with a rights bitfield (READ/WRITE/EXEC). Failing a capability check returns ENOCAP (errno 130). Capability checks appear at the entry of each guarded syscall, before any resource allocation.

The capability subsystem itself (kernel/cap/) is implemented in Rust – the first kernel component to be migrated from C. This makes the capability enforcement logic memory-safe by construction, even though the syscall handlers that call into it are still C. The planned kernel-wide Rust migration will extend this safety guarantee to the syscall dispatch and argument parsing layers.

Complete Syscall Reference

File I/O (sys_io.c, sys_file.c)

Directory Operations (sys_dir.c, sys_meta.c)

Memory Management (sys_memory.c)

sys_brk details: Page-aligns upward. Zeroes new pages (Linux guarantee). Rejects shrink below brk_base to prevent freeing ELF segments. On OOM, returns current break unchanged.

sys_mmap details: Supports MAP_ANONYMOUS | MAP_PRIVATE (bump allocator from mmap_base at 0x700000000000), MAP_FIXED, and file-backed MAP_SHARED (memfd). Recycles VAs from munmap freelist before bumping.

Process Management (sys_process.c, sys_exec.c)

See Processes & ELF Loading for detailed fork/clone/execve documentation.

Identity & Session (sys_identity.c)

sys_arch_prctl details:

• ARCH_SET_FS (0x1002): Set FS.base for TLS. Rejects kernel addresses. Writes IA32_FS_BASE MSR and saves to task->fs_base.
• ARCH_GET_FS (0x1003): Write current fs_base to user pointer.

Signal Handling (sys_signal.c)

Signal delivery is checked at syscall return (SYSRET path). If a signal is pending and not masked, the kernel builds a signal frame on the user stack and redirects SYSRET to the handler. rt_sigreturn restores the original user context.

SIGKILL, SIGSTOP, and SIGCONT cannot be caught or ignored.

Time (sys_time.c)

sys_nanosleep: Computes a sleep_deadline from the PIT tick counter, sets task->sleep_deadline, and calls sched_block(). sched_tick wakes the task when the deadline passes.

Networking (sys_socket.c)

See Network Stack and Socket API for details.

Synchronization (futex.c)

Supports FUTEX_WAIT (block if *addr == val) and FUTEX_WAKE (wake up to val waiters). The FUTEX_PRIVATE_FLAG (128) is accepted and ignored (single address space per process). Used internally for clear_child_tid on thread exit (pthread_join).

Random (sys_random.c)

Block Device / Framebuffer (sys_disk.c)

Capability Management (sys_cap.c)

Aegis Extensions

Syscall numbers 360-364, 500-514 are Aegis-specific extensions outside the Linux syscall number range.

Error Handling

Syscalls return negative errno values on failure (Linux convention):

All unrecognized syscall numbers return ENOSYS.

Implementation Structure

The syscall implementation is split across multiple translation units for organization:

All files include sys_impl.h which provides shared types, constants, and forward declarations.

Related Documentation

• Processes & ELF Loading – fork, clone, execve details
• Scheduler – task states and blocking
• Interrupts & Exceptions – ISR entry path (compared with SYSCALL path)
• Capability model – capability table and capability kinds
• Security policy engine – policy capabilities and baseline capabilities
• Network Stack – socket implementation details