Syscall Interface

Aegis implements a Linux-compatible syscall interface using the SYSCALL/SYSRET instructions on x86-64 and SVC/ERET on ARM64. The kernel provides over 100 syscalls covering file I/O, process management, memory management, networking, signals, and Aegis-specific extensions. User-space code (compiled against musl libc) issues standard Linux syscall numbers.

v1 maturity note: Aegis is v1 software – the first version deemed ready for public release, not a mature or production-hardened system. The syscall interface is the kernel’s primary attack surface: every syscall handler parses untrusted user input in C. While user pointer validation and kernel staging buffers are used throughout, there should be no expectation of real, battle-tested security. There are likely exploitable vulnerabilities in individual syscall implementations, as would be expected in any from-scratch C kernel of this scale. A gradual migration from C to Rust is planned starting with the kernel; the capability system (kernel/cap/) is already in Rust and represents the beginning of this path. Contributions are welcome – file issues or propose changes at exec/aegis.

Entry Mechanism

x86-64: SYSCALL/SYSRET

The syscall entry point is configured during boot by arch_syscall_init() (kernel/arch/x86_64/arch_syscall.c):

wrmsr(IA32_EFER, rdmsr(IA32_EFER) | 1UL);     /* Enable SCE bit */
wrmsr(IA32_STAR, (ARCH_KERNEL_DS << 48) | (ARCH_KERNEL_CS << 32));
wrmsr(IA32_LSTAR, (uint64_t)syscall_entry);     /* Entry point */
wrmsr(IA32_SFMASK, 0x700UL);                    /* Clear IF, TF, DF */

CPU state on SYSCALL entry:

• RAX = syscall number
• RDI = arg1, RSI = arg2, RDX = arg3, R10 = arg4, R8 = arg5, R9 = arg6
• RCX = return RIP, R11 = saved RFLAGS
• RSP = user stack (unchanged by CPU)
• IF=0 (interrupts disabled by SFMASK)

Syscall Entry Assembly (syscall_entry.asm)

1. SWAPGS                          -- switch GS to percpu_t
2. Save user RSP to gs:32          -- percpu.user_rsp_scratch
3. Load kernel RSP from gs:24      -- percpu.kernel_stack
4. Push restore frame:             -- user_rsp, rcx (RIP), r11 (RFLAGS)
5. Push callee-saved regs:         -- r15, r14, r13, r12, rbp, rbx
6. Push r8, r9, r10                -- forms syscall_frame_t
7. Shuffle to SysV 8-arg call:
     rdi=frame, rsi=num, rdx=arg1, rcx=arg2,
     r8=arg3, r9=arg4, [rsp+8]=arg5, [rsp+16]=arg6
8. call syscall_dispatch
9. Check for pending signals
10. Restore registers, SWAPGS, SYSRET

Syscall Frame

The registers pushed onto the kernel stack form a syscall_frame_t, used by sys_fork, sys_execve, and signal delivery to read or modify the user register state:

/* x86-64 */
typedef struct syscall_frame {
    uint64_t r10;       /* +0:  arg4 */
    uint64_t r9;        /* +8:  arg6 */
    uint64_t r8;        /* +16: arg5 */
    uint64_t rbx;       /* +24: callee-saved */
    uint64_t rbp;       /* +32: callee-saved */
    uint64_t r12;       /* +40: callee-saved */
    uint64_t r13;       /* +48: callee-saved */
    uint64_t r14;       /* +56: callee-saved */
    uint64_t r15;       /* +64: callee-saved */
    uint64_t rflags;    /* +72: saved RFLAGS */
    uint64_t rip;       /* +80: return RIP */
    uint64_t user_rsp;  /* +88: saved user RSP */
} syscall_frame_t;

ARM64: SVC/ERET

On ARM64, user-space issues SVC #0. The exception vector in vectors.S saves all 31 general-purpose registers plus SP_EL0, ELR_EL1, and SPSR_EL1 (34 slots), then calls syscall_dispatch. ARM64 uses different syscall numbers than x86-64; the dispatch function translates them before the main switch table.

CR3 Policy

The SYSCALL path does not switch CR3. The user PML4 remains loaded throughout syscall_dispatch so syscalls can directly dereference user virtual addresses. This is safe because:

1. The user PML4 shares the kernel higher-half (PML4[511]) for all kernel code and KVA stacks
2. Timer interrupts via isr_common_stub switch to master PML4 before ISR dispatch
3. sched_exit switches to master PML4 before context-switching away

Dispatch Table

syscall_dispatch() in kernel/syscall/syscall.c is the central dispatch function. On ARM64, it first translates ARM64 syscall numbers to x86-64 equivalents, then dispatches via a switch statement.

ARM64 Translation Layer

ARM64 musl emits ARM64-native syscall numbers. The dispatch function maps them to x86-64 numbers before the main switch. Examples:

The *at variants (openat, mkdirat, renameat2, etc.) are translated to their non-at equivalents by stripping the dirfd argument (Aegis uses absolute/CWD-relative paths).

User Pointer Validation

All syscalls validate user pointers before access. The core check (user_ptr_valid) page-walks the entire range to ensure the pages are actually mapped; an unmapped user pointer returns EFAULT rather than panicking the kernel.

Data is copied between user and kernel space via two primitive layers:

• copy_from_user / copy_to_user (kernel/mm/uaccess.h): raw copy helpers.
• COPY_FROM_USER / COPY_TO_USER (kernel/mm/uaccess_user.h): bound macros that derive the copy length from sizeof(*dst) so the validated length and the copy length can never silently diverge. At sites that have already acquired a resource (lock, kva buffer, fd) and need an explicit cleanup path on failure, the copy_from_user_checked / copy_to_user_checked variants are used instead — they return a bool rather than issuing a hidden return.

Writes use a kernel staging buffer to prevent TOCTOU attacks. All errno values are named constants from kernel/include/aegis_errno.h, encoded for the kernel ABI via SYS_ERR(e).

These are v1 mitigations – necessary but not sufficient for a hardened kernel. Individual syscalls may still have validation gaps. Real exploitable bugs in argument handling almost certainly exist across the 100+ syscall implementations.

Capability Gates

Many syscalls check the calling process’s capability table before proceeding. Each entry in the capability table pairs a capability kind (CAP_KIND_*) with a rights bitfield (READ/WRITE/EXEC). Failing a capability check returns ENOCAP (errno 130). Capability checks appear at the entry of each guarded syscall, before any resource allocation.

The capability subsystem itself (kernel/cap/) is implemented in Rust – the first kernel component to be migrated from C. This makes the capability enforcement logic memory-safe by construction, even though the syscall handlers that call into it are still C. The planned kernel-wide Rust migration will extend this safety guarantee to the syscall dispatch and argument parsing layers.

Complete Syscall Reference

File I/O (sys_io.c, sys_file.c)

Directory Operations (sys_dir.c, sys_meta.c)

Memory Management (sys_memory.c)

sys_brk details: Page-aligns upward. Zeroes new pages (Linux guarantee). Rejects shrink below brk_base to prevent freeing ELF segments. On OOM, returns current break unchanged.

sys_mmap details: Supports MAP_ANONYMOUS | MAP_PRIVATE (bump allocator from mmap_base at 0x700000000000), MAP_FIXED, and file-backed MAP_SHARED (memfd). Recycles VAs from munmap freelist before bumping.

Process Management (sys_process.c, sys_exec.c)

See Processes & ELF Loading for detailed fork/clone/execve documentation.

Identity & Session (sys_identity.c)

sys_arch_prctl details:

• ARCH_SET_FS (0x1002): Set FS.base for TLS. Rejects kernel addresses. Writes IA32_FS_BASE MSR and saves to task->fs_base.
• ARCH_GET_FS (0x1003): Write current fs_base to user pointer.

Signal Handling (sys_signal.c)

Signal delivery is checked at syscall return (SYSRET path). If a signal is pending and not masked, the kernel builds a signal frame on the user stack and redirects SYSRET to the handler. rt_sigreturn restores the original user context.

SIGKILL, SIGSTOP, and SIGCONT cannot be caught or ignored.

Time (sys_time.c)

sys_nanosleep: Computes a sleep_deadline from the PIT tick counter, sets task->sleep_deadline, and calls sched_block(). sched_tick wakes the task when the deadline passes.

Networking (sys_socket.c)

See Network Stack and Socket API for details.

Synchronization (futex.c)

Supports FUTEX_WAIT (block if *addr == val) and FUTEX_WAKE (wake up to val waiters). The FUTEX_PRIVATE_FLAG (128) is accepted and ignored (single address space per process). Used internally for clear_child_tid on thread exit (pthread_join).

Random (sys_random.c)

Block Device / Framebuffer (sys_disk.c)

Capability Management (sys_cap.c)

Aegis Extensions

Syscall numbers 169-170, 360-364, 500-514 are Aegis-specific extensions outside the Linux syscall number range.

Error Handling

Syscalls return negative errno values on failure (Linux convention). The kernel encodes them as SYS_ERR(e) defined in kernel/include/aegis_errno.h, which expands to (uint64_t)-(int64_t)(e) — the same bit pattern musl’s syscall wrapper interprets as errno + return -1. All errno symbols are defined in that single header; no scattered per-file #define EBADF 9 fragments remain in the source.

All unrecognized syscall numbers return ENOSYS.

Implementation Structure

The syscall implementation is split across multiple translation units for organization:

All files include sys_impl.h which provides shared types, constants, and forward declarations.

Related Documentation

• Processes & ELF Loading – fork, clone, execve details
• Scheduler – task states and blocking
• Interrupts & Exceptions – ISR entry path (compared with SYSCALL path)
• Capability model – capability table and capability kinds
• Security policy engine – policy capabilities and baseline capabilities
• Network Stack – socket implementation details