May 2026 - l4-hackers - example.com Site

Comprehension question: vcpu vs extended vcpu
by David Q 01 Jun '26

01 Jun '26

Hello L4 hackers, Hope y'all are doing well. I am back with another question, this time regarding vcpus and its "extended" add-on. From reading the documentation and some tinkering, it seems the default vcpu is quite capable of performing virtualization tasks as the virtualized CPU. 1. When looking at code in uvmm, it seems the "normal" vcpu is used for virtualization by default, is this correct? 2. When should we need to use the "extended" add on? a. It seems "extended" uses additional hardware assisted virtualization features, is this understanding correct? b. If the assumption in a. is correct, what pros and cons does running "extended" have? (Maybe additional safety/stability improvements?) c. Are there hardware limitations for running in "extended" mode? (For example, is A76 supported?) Thank you so much for your attention :) Best, David

2 2

NOVA multitasking
by Nate 29 May '26

29 May '26

Hi L4 hackers! I've been quite interested in the NOVA microhypervisor, and thought because it is based off L4 and has similar architecture, you guys might be able to help. Now, I've always thought NOVA was a cool kernel thing, and wanted to make an OS around it. However, the only thing online that guides you is Genode, which doesn't really help you make an OS, but rather a domain-specific wrapper around Genode. So, I did some research on the kernel itself and attempted to make my own OS without Genode. I managed to get a simple root task. Here is start.S ``` .global _start .extern root_main .section .data .align 16 stack: .space 16384 stack_top: .text _start: mov %rsp, %rdi lea stack_top(%rip), %rsp and $-16, %rsp call root_main .hang: hlt jmp .hang ``` Here is root.c ``` #include <stdint.h> #include "thread.h" #include "hypercalls.h" #include "io.h" void thread_uno(void) { while (1) { serial_print_locked("Hello from thread!\r\n"); for (volatile int i = 0; i < 5000000; i++); } } void root_main(void *hip) { mword rax, rdi; nova_call((SEL_KERNEL_OBJ_SPACE << 8) | NOVA_SYS_CTRL_PD, SEL_MAIN_OBJ_SPACE, (KSEL_HOST_PIO << 12), (LOCAL_SEL_HOST_PIO << 12) | 0xf, 0, &rax, &rdi); nova_call((SEL_KERNEL_OBJ_SPACE << 8) | NOVA_SYS_CTRL_PD, SEL_MAIN_OBJ_SPACE, (KSEL_MAIN_PIO << 12), (LOCAL_SEL_MAIN_PIO << 12) | 0xf, 0, &rax, &rdi); nova_call((LOCAL_SEL_HOST_PIO << 8) | NOVA_SYS_CTRL_PD, LOCAL_SEL_MAIN_PIO, (0x3F8 << 12) | 3, (0x3F8 << 12) | 1, 0, &rax, &rdi); Thread t = spawn_thread(thread_uno, 20); (void)t; while (1) { serial_print_locked("Hello from main!\r\n"); for (volatile int i = 0; i < 5000000; i++); }} ``` And, io.c ``` #include "hypercalls.h" #include "io.h" static inline void outb(uint16_t port, uint8_t val) { asm volatile("outb %0, %1" : : "a"(val), "Nd"(port)); } static inline uint8_t inb(uint16_t port) { uint8_t val; asm volatile("inb %1, %0" : "=a"(val) : "Nd"(port)); return val; } volatile int serial_lock = 0; void serial_putc(char c) { while (!(inb(0x3F8 + 5) & 0x20)) asm volatile("pause"); outb(0x3F8, c); } void serial_print_locked(const char *s) { while (__atomic_test_and_set(&serial_lock, __ATOMIC_ACQUIRE)) asm volatile("pause"); while (*s) serial_putc(*s++); __atomic_clear(&serial_lock, __ATOMIC_RELEASE); } void serial_print_hex_locked(mword val) { char buf[19]; buf[0] = '0'; buf[1] = 'x'; for (int i = 17; i >= 2; i--) { int nibble = val & 0xf; buf[i] = nibble < 10 ? '0' + nibble : 'a' + nibble - 10; val >>= 4; } buf[18] = 0; const char *p = buf; while (*p) serial_putc(*p++); } void show_status(const char *msg, mword rax, mword rdi) { serial_print_locked(msg); serial_print_locked(" [RAX:"); serial_print_hex_locked(rax); serial_print_locked(" RDI:"); serial_print_hex_locked(rdi); serial_print_locked("]\r\n");} ``` So, this works simple enough. But, the next thing I want is multitasking. So, I made these two files, hypercalls.c ``` #include "hypercalls.h" void nova_call(mword rdi, mword rsi, mword rdx, mword rax, mword r8, mword *out_rax, mword *out_rdi) { mword res_rax, res_rdi; asm volatile( "mov %6, %%r8\n" "syscall\n" : "=a"(res_rax), "=D"(res_rdi) : "D"(rdi), "S"(rsi), "d"(rdx), "a"(rax), "r"(r8) : "rcx", "r11", "r8", "memory" ); if (out_rax) *out_rax = res_rax; if (out_rdi) *out_rdi = res_rdi; } ``` And thread.c ``` #include "hypercalls.h" #include "thread.h" #include "io.h" #define MAX_THREADS 16 uint8_t thread_stacks[MAX_THREADS][4096] __attribute__((aligned(4096))); uint8_t thread_utcbs[MAX_THREADS][4096] __attribute__((aligned(4096))); uint8_t worker_stacks[MAX_THREADS][4096] __attribute__((aligned(4096))); uint8_t driver_utcbs[MAX_THREADS][4096] __attribute__((aligned(4096))); static mword worker_utcb_addrs[MAX_THREADS]; int thread_slot = 0; mword next_sel = 128; typedef struct { mword entry; mword stack; } StartupArgs; static StartupArgs pending_startup[MAX_THREADS]; static mword alloc_sel(mword count) { mword s = next_sel; next_sel += count; return s; } void startup_handler(mword id) { if (id >= MAX_THREADS) while(1); mword *utcb = (mword *)worker_utcb_addrs[id]; utcb[4] = pending_startup[id].stack; utcb[16] = 0x202; utcb[17] = pending_startup[id].entry; mword rax, rdi; nova_call(NOVA_SYS_IPC_REPLY, 0x1a, 0, 0, 0, &rax, &rdi); while(1) { asm volatile("pause"); } } Thread spawn_thread(void (*entry)(void), int priority) { int slot = thread_slot++; mword worker_ec = alloc_sel(1); mword driver_ec = alloc_sel(1); mword sc_sel = alloc_sel(1); mword portal_base = (next_sel + 255) & ~255UL; next_sel = portal_base + 256; mword pt_startup = portal_base + 0x1e; mword stack_top = (mword)&thread_stacks[slot][4096]; mword worker_utcb = (mword)thread_utcbs[slot]; mword driver_utcb = (mword)driver_utcbs[slot]; worker_utcb_addrs[slot] = worker_utcb; pending_startup[slot].entry = (mword)entry; pending_startup[slot].stack = stack_top; mword rax, rdi; mword worker_stack_top = (mword)&worker_stacks[slot][4096]; nova_call((worker_ec << 8) | NOVA_SYS_CREATE_EC, SEL_MAIN_PD, worker_utcb, 0, worker_stack_top, &rax, &rdi); show_status(" Worker EC:", rax, rdi); nova_call((pt_startup << 8) | NOVA_SYS_CREATE_PT, SEL_MAIN_PD, worker_ec, (mword)startup_handler, 0, &rax, &rdi); nova_call((pt_startup << 8) | 11, slot, 0, 0, 0, &rax, &rdi); nova_call((driver_ec << 8) | (2 << 4) | NOVA_SYS_CREATE_EC, SEL_MAIN_PD, driver_utcb, portal_base, stack_top, &rax, &rdi); show_status(" Driver EC:", rax, rdi); nova_call((sc_sel << 8) | NOVA_SYS_CREATE_SC, SEL_MAIN_PD, driver_ec, ((mword)priority << 16) | 10000, 0, &rax, &rdi); show_status(" SC:", rax, rdi); return (Thread){ driver_ec, sc_sel }; } ``` At this point, I have no clue what I am doing wrong, the output is just ``` [ -1] RSDT: 0x3ffe2351 OEM:BOCHS TBL:BXPC REV: 1 LEN: 56 (ok) [ -1] FACP: 0x3ffe2149 OEM:BOCHS TBL:BXPC REV: 3 LEN: 244 (ok) [ -1] APIC: 0x3ffe223d OEM:BOCHS TBL:BXPC REV: 3 LEN: 120 (ok) [ -1] HPET: 0x3ffe22b5 OEM:BOCHS TBL:BXPC REV: 1 LEN: 56 (ok) [ -1] MCFG: 0x3ffe22ed OEM:BOCHS TBL:BXPC REV: 1 LEN: 60 (ok) [ -1] WAET: 0x3ffe2329 OEM:BOCHS TBL:BXPC REV: 1 LEN: 40 (ok) [ -1] ACPI: Version 3.0 Profile 0 Features 0x84a5 Boot 0x2 [ -1] FACS: Hardware 0x0 Flags 0x0 Wake 0x0/0x0 [ -1] PCIE: 0xb0000000 Seg 0x0000 Bus 0x00-0xff [ -1] PCIE: 8086:29c0 06-00-00 D0 0000:00:00.0 [ -1] PCIE: 1234:1111 03-00-00 D0 0000:00:01.0 [ -1] PCIE: 8086:10d3 02-00-00 D0 PMI PCIE MSI MSIX 0000:00:02.0 [ -1] PCIE: 8086:2918 06-01-00 D0 0000:00:1f.0 [ -1] PCIE: 8086:2922 01-06-01 D0 MSI 0000:00:1f.2 [ -1] PCIE: 8086:2930 0c-05-00 D0 0000:00:1f.3 [ -1] HPET: 0xfed00000 [ 0] CORE: 00:000.0 6:5e:3:0 [0x1] Intel Core Processor (Skylake) [ 0] TIME: 386ms 0ms/0ms [ 0] INFO: NOVA: 0x000000003b000000-0x000000003f000000 [ 0] INFO: MBUF: 0x000000003ec8e000-0x000000003ec8f000 [ 0] INFO: ROOT: 0x0000000000102000-0x0000000000149000 [ 0] INFO: ACPI: 0xf52c0 [ 0] INFO: UEFI: 0xffffffffffffffff 0 0 0 [ 0] INFO: FREQ: 2719941700 Hz [ 0] INFO: SBW#: OBJ:18 HST:35 GST:36 DMA:0 PIO:16 MSR:13 [ 0] INFO: HST#: 32 + 2 [ 0] INFO: GST#: 256 + 2 [ 0] INFO: CPU#: 1 Worker EC: [RAX:0x0000000000000000 RDI:0x0000000000000000] Driver EC: [RAX:0x0000000000000100 RDI:0x0000000000000008] SC: [RAX:0x0000000000142710 RDI:0x0000000000000005] Hello from main! Hello from main! Hello from main! Hello from main! Hello from main! Hello from main! Hello from main! Hello from main! Hello from main!Hello from main! (forever) ``` As you can see, no second thread. I heard that the RAX and RDI values should all be `0x0000000000000000` if it worked? Would anyone be able to help me? Thanks!

1 0

Kernel allocated memory not visible in L4Re virtual device despite correct physical RAM mapping
by stephen.yang 14 May '26

14 May '26

Hi l4-hackers, I am trying to pass a kernel-allocated memory buffer to a virtual device, but I am running into an issue. I would appreciate any help or suggestions. 1.Kernel side(driver) In my platform driver, I allocate memory using devm_kzalloc, write a string into it, and pass its physical address to the device via an MMIO register: …… void* buf = devm_kzalloc(&pdev->dev, 0x200, GFP_KERNEL); if (!buf) return -ENOMEM; phys_addr_t pa = virt_to_phys(buf); char *str = "hi stephen"; memcpy(buf, str, strlen(str) + 1); wmb(); writeq((u64)pa, hb->base + 0x00); …… My understanding is: allocate kernel memory → write data → pass physical address to the device. 2. Memory layout I observed that the allocated physical address falls within the RAM regions configured via bootargs : 0x66200000 - 0x6fffffff So my assumption was that if the virtual device maps this physical memory region, it should be able to access the data written by the kernel. 3. Virtual device side (L4Re) On the virtual device side, I map the physical memory using L4Re RM attach: void write(unsigned reg, char /*size*/, l4_uint64_t value, unsigned) { if (reg == 0x00) { addr = value; auto d = L4Re::Env::env()->get_cap<L4Re::Dataspace>("lram"); l4_addr_t vaddr; l4_addr_t base = 0x66200000; l4_addr_t size = 0x09E00000; int r = L4Re::Env::env()->rm()->attach( &vaddr, size, L4Re::Rm::F::Search_addr | L4Re::Rm::F::RW | L4Re::Rm::F::Cache_uncached, L4::Ipc::make_cap_rw(d), 0, L4_PAGESHIFT); if (r) { printf("can't attach hb memory\n"); return; } char *dst = reinterpret_cast<char *>(vaddr + (addr - base)); for (int i = 0; i < 20; i++) { printf("%02x ", (unsigned char)dst[i]); } printf("\n"); printf("get msg for guest: %s\n", dst); } } 4.Problem However, I cannot read the string written by the kernel. The memory content I get appears to be uninitialized or unrelated data. 5.Question I can confirm that the lram dataspace capability does correctly correspond to the physical memory region 0x66200000 - 0x6fffffff, and this region is valid and usable RAM on the system. I am unsure where the issue lies: (1) Is this approach (kernel allocation → pass physical address → direct mapping in virtual device) actually valid? (2) Could this be a cache coherence / DMA issue? (3) Or is there something incorrect in my L4Re RM attach usage? Any insights would be greatly appreciated.

2 6