l4-hackers September 2025

l4-hackers@os.inf.tu-dresden.de

7 participants
4 discussions

Assertion ‘state() & Thread_vcpu_user' in kernel vgic interrupt processing
by yy18513676366 14 Oct '25

14 Oct '25

Hi l4-hackers, When running a virtual machine, I encounter an assertion failure after the VM has been up for some time. The kernel crashes in src/kern/arm/thread-arm-hyp.cpp, specifically in the function vcpu_vgic_upcall(unsigned virq): vcpu_vgic_upcall(unsigned virq) { ...... assert(state() & Thread_vcpu_user); ...... } Based on source code inspection and preliminary debugging, the problem seems to be related to the management of the Thread_vcpu_user state. （1）Under normal circumstances, the vcpu_resume path (transitioning from the kernel back to the guest OS) updates the vCPU state to include Thread_vcpu_user. However, if an interrupt is delivered during this transition while the receiving side is not yet ready, the vCPU frequently return to the kernel (via vcpu_return_to_kernel) and subsequently process the interrupt through guest_irq in vcpu_entries. In this situation, the expected update of Thread_vcpu_user may not yet have taken place, which seems result in the assert being triggered when a VGIC interrupt is involved. （2）A similar condition seems to occur in the vcpu_async_ipc path. At the end of IPC handling, this function explicitly clears the Thread_vcpu_user flag. If a VGIC interrupt is delivered during this phase, the absence of the expected Thread_vcpu_user state seems to lead to the same assertion failure. I would like to confirm if the two points above are correct, and what steps I should take next to further debug this issue. In addition, I have some assumptions I would like to confirm: First, for IPC between non-vcpu threads, the L4 microkernel handles message delivery and scheduling (wake/schedule) directly, without requiring any forwarding through uvmm. Similarly, interrupts bound via the interrupt controller (ICU) to a non-vcpu thread or handler are also managed by the kernel and scheduler, and therefore do not necessarily involve uvmm. Second, passthrough interrupts, when not delivered in direct-injection mode, are routed to uvmm for handling if they are bound to a vCPU. Likewise, services provided by uvmm (such as virq) are also bound to a vCPU and therefore require forwarding through uvmm. There seems to have been a similar question in the past, but it does not seem to have been resolved. Re: Assertion failure error in kernel vgic interrupt processing - l4-hackers - OS Site I wonder if my questions are related to that post, and if any solutions exist. Thanks!

2 7

L4 Thread, pthread, comprehension question
by xudino＠gmail.com 23 Sep '25

23 Sep '25

Hello L4 hackers, I hope you are all doing well today. I am hoping to discuss the topic of threads to better my understanding of it, as it seems to be a major building block in L4Re. I have the following questions (using the c++ based libraries.): 1. Is Pthread the official way to create threads in an L4Re application? 2. I have made a L4 thread (to understand how threads work in L4) without the help of pthreads by setting up the Attr, which involved in setting the exception handlers(using rm()), pager (using rm()), instruction pointers, stack pointers, binding, ex_reg() and finally running. All seems well until I try printing a message. It seems seems I am unable to use the printf() function in the instruction pointer function of said thread. The error I get is : ttest | l4re_itas[rm]: unhandled read page fault at 0x141 pc=0x100f8a0 ttest | l4re_itas: rom/thread_test: Unhandled exception: PC=0x100f8a0 PFA=0x140 LdrFlgs=0x0 However, if I use printf in a pthread, none of these errors pop up. 2a. Am I using the wrong exception handler and pager (I got both from rm)? 2b. Also, the "l4_vcon_send(L4_BASE_LOG_CAP,..,...) function seems to work fine in either case, shouldn't printf() eventually call this function as after a series of indirections? 2c. What exc_handler and pager do pthreads use? Why are they able to handle the pager faults and exceptions from printf? Can I appropriate them for the L4 threads I started? 3. It seems the Virtual Machines (VMs) that are on the hypervisor are also closely related with threads. I assume these are Vcpu threads and not pthreads. 3a. Am I correct in this assumption? 3b. How are the exception handlers and pagers set up for these Vcpus? Thank you so much for your time, David

3 3

Re: L4 Thread, pthread, comprehension question
by Marcus Haehnel 23 Sep '25

23 Sep '25

Hi David, On 2025-09-22 05:37, xudino(a)gmail.com wrote: > Hello L4 hackers, I hope you are all doing well today. > > I am hoping to discuss the topic of threads to better my understanding > of it, as it seems to be a major building block in L4Re. I have the > following questions (using the c++ based libraries.): > > 1. Is Pthread the official way to create threads in an L4Re > application? That is at least the high level primitive that you should use if you have no good reason to use the low level L4 primitives and want to use any of the POSIX / libc functionality inside your thread. > > 2. I have made a L4 thread (to understand how threads work in L4) > without the help of pthreads by setting up the Attr, which involved in > setting the exception handlers(using rm()), pager (using rm()), > instruction pointers, stack pointers, binding, ex_reg() and finally > running. All seems well until I try printing a message. > It seems seems I am unable to use the printf() function in the > instruction pointer function of said thread. The error I get is : > ttest | l4re_itas[rm]: unhandled read page fault at 0x141 > pc=0x100f8a0 > ttest | l4re_itas: rom/thread_test: Unhandled exception: > PC=0x100f8a0 PFA=0x140 LdrFlgs=0x0 > However, if I use printf in a pthread, none of these errors pop up. > 2a. Am I using the wrong exception handler and pager (I got both > from rm)? > 2b. Also, the "l4_vcon_send(L4_BASE_LOG_CAP,..,...) function seems > to work fine in either case, shouldn't printf() eventually call this > function as > after a series of indirections? > 2c. What exc_handler and pager do pthreads use? Why are they able > to handle the pager faults and exceptions from printf? Can I > appropriate > them for the L4 threads I started? What you are seeing is a side-effect of libpthread being quite tightly integrated with the libc. While it is true that printf will call l4_vcon_send somewhen down the line it will use some (potentially thread-local) internal buffers to print its output. While you can mix libpthread threads with native L4 threads you must take care to call no function from the libc that would make any use of TLS storage, unless you set this up yourself. If you want to write normal applications std::thread and pthread threads are the way to go. > > 3. It seems the Virtual Machines (VMs) that are on the hypervisor are > also closely related with threads. I assume these are Vcpu threads and > not pthreads. > 3a. Am I correct in this assumption? > 3b. How are the exception handlers and pagers set up for these > Vcpus? Yes, a vCPU is in the end only a thread with some extended state. A thread in vcpu mode provides an additional exception execution context. When a trap from avcpu occurs the kernel will switch the vCPU to the exception entry point. See entry_ip and entry_sp here https://l4re.org/doc/structl4__vcpu__state__t.html#a1f45174954aa47bb2f21b7b… These must be set as part of the vcpu state before starting the vcpu. The exception handler must take care of handling page-faults generated by threads user code. I hope this helps :) Best regards, - Marcus > > Thank you so much for your time, > David > _______________________________________________ > l4-hackers mailing list -- l4-hackers(a)os.inf.tu-dresden.de > To unsubscribe send an email to l4-hackers-leave(a)os.inf.tu-dresden.de

2 1

Help: interactive REPL I/O for user program on L4Re/Fiasco.OC (amd64-plain) — Vcon/cons nearly working; canonical approach?
by Robert A. James 07 Sep '25

07 Sep '25

Hello l4-hackers, My name is Robert (Bob) A. James I'm VERY new to the L4Re world. This is also my first foray into systems programming having come from the Medical Device and Enterprise areas during my career. I am now retired and developing a full Operating System based on L4Re/Fiasco.OC. I’m looking for guidance on the /canonical/ way to wire an interactive text console (stdin/stdout) for a user program under L4Re/Fiasco.OC on amd64-plain. Context * I have a small, headless FORTH VM (“StarForth”) running under L4Re/Fiasco.OC; its self-tests pass. What’s missing is an *interactive REPL*. * Target outcomes: a) Run via |make qemu E=starforth O=build-x86_64| b) Build GRUB2 ISO via |make grub2iso E=starforth O=build-x86_64| c) ISO runs in QEMU and VirtualBox d) Same ISO boots on commodity x86_64 hardware What I’ve tried (and where I’m stuck) 1. *Vcon + cons* (almost there) * My mental model: Vcon as a frontend; |cons| providing the console service; StarForth as the “backend” client/subscriber. * Behavior I see: • The “terminal” prints what the *logger* prints rather than the process’s normal *stdout*. • Output sometimes *flushes only on newline*; interactive prompts without |\n| don’t appear reliably. * Aside from those two issues, this was close to a working REPL. * *Questions:* Is |cons| effectively write-only (log)? If so, what’s the supported input-capable console frontend to bind both stdin and stdout for a user process? Is there a minimal example showing correct capability wiring so user stdout/stderr don’t get mixed with the system log? 2. *GOOS/Framebuffer route* * I attempted a tiny terminal server that maps the GOOS framebuffer and renders an 8×16 CP437 font. I did not get this path working end-to-end, so I’m fine dropping it if there’s a simpler, recommended console approach. What would help * The *recommended minimal pattern* (modules/services + boot script) to give a user task a real interactive stdin/stdout: o works with |make qemu …| and |make grub2iso …| o also works on typical PCs without a full graphics stack. * A *small reference* that: (1) prints text, and (2) reads keystrokes from the user, with a snippet of |modules.list| and the Lua loader snippet. Environment / details * L4Re/Fiasco.OC on amd64-plain, built with the L4Re toolchain. * Intended entry points: |make qemu E=starforth O=build-x86_64|, |make grub2iso E=starforth O=build-x86_64|. One of my simple boot variants (for reference): |modules.list| (simplified) entry starforth kernel fiasco -serial_esc roottask moe rom/quark.cfg module l4re module ned module cons module quark.cfg module starforth |quark.cfg| --> I think this is about what I had. -- rom/quark.cfg local L4 =require("L4") local ld =L4.default_loader -- Start cons; feed it to the real serial logger (fe =L4.Env.log) local cons_ctl =ld:new_channel() ld:start({ caps = { cons = cons_ctl:svr(), -- server gate for cons' factory fe = L4.Env.log, -- cons frontend goes to serial/log }, }, "rom/cons -m l4re -f fe -a -t -c StarForth") -- ← added -c StarForth -- Ask cons to create a Vcon client for StarForth and pass it in as 'tty' -- (Capabilities are first-class in Ned; calling 'create' on caps is supported.) local tty =cons_ctl:create(L4.Proto.Log, "StarForth", "show", "key=1", "bufsz=65536") -- Start StarForth with that Vcon as its TTY ld:start({ caps = { tty = tty, -- the Vcon client for StarForth }, }, "rom/starforth") I can share a minimal reproducible case (sources, build logs, QEMU output) off-list if that helps. Thanks in advance for any pointers to the “right way,” or to a maintained example I can follow. Best regards, R. A. “Bob” James (rajames440(a)gmail.com) Kubuntu 25.04 • amd64-plain • QEMU/VirtualBox • GRUB2 boot targets

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

l4-hackers September 2025