l4-hackers June 2025

l4-hackers@os.inf.tu-dresden.de

8 participants
8 discussions

L4Re: MIPS SMP cache initialisation may be broken for some CPUs
by Paul Boddie 20 Jul '25

20 Jul '25

Hello, Pushing on with my efforts to port L4Re to another MIPS-based SoC, I have been attempting to activate more than one CPU core. Although there is some support in Fiasco for the MIPS Coherence Manager (CM) and Cluster Power Controller (CPC), these technologies are not supported in this particular SoC, which is related to the JZ4780 used in the MIPS Creator CI20. However, the Platform_control abstraction (in src/kern/mips/platform_control- mips.cpp) does seem to provide some flexibility in the way secondary CPUs or cores may be started. It seems that even in situations where the CM is not available, the alloc_secondary_boot_code method will assemble a bootstrap routine from snippets including one, _tramp_mp_cache, that initialises the caches. Unfortunately, it appears that this cache initialisation code does not work on this particular SoC. The method used by_tramp_mp_cache is the familiar one where the cache tags are invalidated for all cache index entries using the Index_Store_Tag operation for each cache type. This is supposed to work on any MIPS CPU, but through some tedious troubleshooting, I discovered that it does not work for the data cache. If the data cache variant of the Index_Store_Tag operation is performed for a range of addresses starting from the base of KSEG0, I found that some kind of side-effect would occur, possibly affecting the rather critical addresses in that range. This would prevent Fiasco from continuing to start up, and things like timers and the UART would not function, although the processor was not halted and would still run code, illuminating LEDs as a last-resort debugging tool. There is some preliminary support for this SoC, as well as the JZ4780, in Linux and it seems that instead of using Index_Store_Tag, the tag records are explicitly invalidated and updated. However, it also seems to be the case that caches can be invalidated using Hit_Invalidate. Changing the cache operations in _tramp_mp_cache to use this invalidation operation in its two variants appears to work, avoiding the problems described above. Does anyone have any thoughts or recollections about this cache initialisation routine? On the topic of actually starting secondary cores, this SoC has its own set of registers for resetting cores, interrupt control, mailboxes, and so on. I have developed a fairly simple abstraction that performs similar core bootstrapping work to that of the Cm abstraction (in src/kern/mips/cm.cpp). I then extend the Platform_control abstraction so that it will call the alloc_secondary_boot_code method, followed by the appropriate method to start the secondary cores. Is there any documentation for the SMP architecture in Fiasco and L4Re? I note that interprocessor interrupts (IPI) are one element that I might need to support, probably using the mailboxes. Thanks in advance for any advice or feedback that may be given! Paul

2 2

L4Re: Why was EXTERN_C_BEGIN/END replaced?
by Paul Boddie 23 Jun '25

23 Jun '25

Hello, Taking the opportunity to use a newer snapshot of L4Re, I notice that EXTERN_C_BEGIN and EXTERN_C_END now cause errors like this: error: unknown type name ‘DO_NOT_USE_EXTERN_C_END_ANY_MORE__USE__END_DECLS’ The commit introducing this behaviour seems to be this one: https://github.com/kernkonzept/l4re-core/commit/ e04b50e0e4f7c7a3316a2d8b658db6d81bd3e88d It would appear that throughout L4Re, a mass renaming has occurred from these symbols to __BEGIN_DECLS and __END_DECLS. Was there a particular reason for what might seem like an arbitrary change? Paul

3 2

Upgrade issues. VM won't start.
by Richard Clark 23 Jun '25

23 Jun '25

Adam, So I have been using version 23.10.1, built as per the download page, and have gotten a couple VMs to ping eachother and give me a login prompt. I finally realized I was missing the virtio_switch package and grabbed it from github and put it where it is supposed to go. Of course, being a version mismatch now, it did not compile. I decide to bite the bullet and do an upgrade (always a mistake) and use the new build process given with the new website. That went smoothly! I install ham, run it, etc, everything builds, I update my local scripts and links to use the new environment variables instead of being hardcoded, and it all builds and looks good. Except that it doesn't run. The VMs get a memory exception and kick into jdb. I have not changed any of my .cfg or .list files. The VMs and ramdisks are untouched. My local (L4 native) processes start and appear to run. But the VMs crash for some reason. Even the device tree is unchanged. I also tried a newly built linux (as opposed to a prebuilt) and that failed as well. Is there some reason for this new crash between version 23.10.1 and the latest version from github? I've attached the VM startup output from both the old and new runs so you can take a look. I'm so close... I see the IP configuration parameter and how to set up lwip and virtio_switch.... Then my natives should be able to talk directly to my linuxes. Your help is greatly appreciated! Richard

6 17

Linux fails on startup looking for SMC
by Moshe Kravchik 22 Jun '25

22 Jun '25

Hi, I'm trying to set up a Linux running on uvmm with a HW passthrough. I added `zynqmp_firmware` to my DTS (it was required for the device I'm trying to pass). However, Linux fails to add the device and I see: vm-1 | [ 1.772078] ledtrig-cpu: registered to indicate activity on CPUs vm-1 | VMM[guest]: No handler for SMC call: imm=0 a0=c200003f a1=3f ip=ffff800080026884 lr=ffff800080dcdb04 vm-1 | VMM[guest]: No handler for SMC call: imm=0 a0=c2000001 a1=0 ip=ffff800080026884 lr=ffff800080dcdb04 vm-1 | [ 1.805957] zynqmp_firmware firmware:zynqmp-firmware: probe with driver zynqmp_firmware failed with error -22 >From the traces I added, the failure is in `zynqmp_pm_get_api_version`. How do I add SMC handling? Best regards, Moshe _______________________________________________ l4-hackers mailing list -- l4-hackers(a)os.inf.tu-dresden.de To unsubscribe send an email to l4-hackers-leave(a)os.inf.tu-dresden.de

2 14

Re: Issue Rebooting Guest OS in UVMM: Invalid Capability Invoke
by Philipp Eppelt 12 Jun '25

12 Jun '25

Hi Stephen, I'm happy to hear that. In the future, please take care to reply to the list too, such that others can learn as well. :-) Cheers, Philipp Am 12.06.25 um 8:34 AM schrieb yy18513676366: > Hi Philipp, > > Thank you for your reply — it was truly helpful to me. > > > I believe I have identified the root cause of the issue. I had assigned full > permissions to all capabilities using :svr(), which included the L4_CAP_FPAGE_D > right. This permission caused kernel objects to be forcibly deleted. After I > adjusted the capability permissions to m("rwncs"), the problem was resolved. > > > I do not yet have a deep enough understanding of capabilities, their permission > flags, and the mapping process via l4_task_map. > > Regarding the restart procedure — it is actually quite straightforward. > According to the documentation, the L4Re::Parent::signal interface can be used > to terminate a running program. So I terminate uvmm first, and then restart it > using the ned script, just like at the beginning. In this way, the guest OS is > relaunched. > > > Through this process, I’ve come to appreciate the stability of L4Re. It is > reassuring to see that the crash of a single application does not affect other > programs. > > > Once again, thank you for your support and assistance. > > Best regards, > > Stephen.yang > > > > > > > At 2025-06-11 17:21:32, "Philipp Eppelt" <philipp.eppelt(a)kernkonzept.com> wrote: >>Hi Stephen, >> >>I don't quite follow your description, can you please show your ned script to >>help my understanding? >> >> >>Wrt 1.: To actively delete a kernel object, you need a capability possessing >>delete rights and call `l4_task_unmap()` featuring the delete flag or >>`l4_task_delete_obj()`. >>Otherwise, the kernel object gets deleted, when the reference count drops to >>zero, meaning there are no more capabilities holding a reference to this kernel >>object. >>An application can release its reference, but still possess the capability. Or a >>capability can be mapped to another application without increasing the reference >>count. >> >> >>To answer 2 and 3 I first need to understand your reboot workflow. >> >>Cheers, >>Philipp >> >> >>Am 10.06.25 um 10:27 AM schrieb yy18513676366: >>> Hello, >>> >>> I am currently conducting research using the L4Re system, and I am working on >>> implementing a mechanism to reboot a Guest OS managed by UVMM. >>> >>> In my setup, I run two Linux instances under UVMM. My goal is to reboot one of >>> them without affecting the other. >>> >>> To achieve this, I terminate the target instance by sending a signal using | >>> L4Re::Parent::signal|, and then I restart it using a Ned script to launch a new >>> instance of the same program. >>> When restarting, I reuse the capability that was created during the first launch >>> of the UVMM instance. >>> >>> However, during execution, the restarted program encounters an *"invalid >>> capability invoke"* error. >>> >>> >>> Upon investigation, I suspect this happens because |l4_task_unmap| was called >>> during the restart process, which likely removed kernel objects associated with >>> the original capability. As a result, the reused |cap_index| no longer refers to >>> a valid object. >>> >>> Based on this, I would appreciate your insights on the following questions: >>> >>> 1. >>> >>> Under what conditions are kernel objects deleted in L4Re? >>> >>> 2. >>> >>> In this rebooting workflow, how can we ensure that capabilities remain valid >>> or that their associated kernel objects are not prematurely unmapped or deleted? >>> >>> 3. >>> >>> If UVMM crashes during operation, how can we design a mechanism to ensure >>> system-wide stability and security? >>> >>> Thank you in advance for your valuable feedback! >>> >>> Best regards, >>> stephen.yang >>> >>> _______________________________________________ >>> l4-hackers mailing list -- l4-hackers(a)os.inf.tu-dresden.de >>> To unsubscribe send an email to l4-hackers-leave(a)os.inf.tu-dresden.de >> >>-- >>philipp.eppelt(a)kernkonzept.com - Tel. 0351-41 883 221 >>http://www.kernkonzept.com >> >>Kernkonzept GmbH. Sitz: Dresden. Amtsgericht Dresden, HRB 31129. >>Geschäftsführer: Dr.-Ing. Michael Hohmuth >> -- philipp.eppelt(a)kernkonzept.com - Tel. 0351-41 883 221 http://www.kernkonzept.com Kernkonzept GmbH. Sitz: Dresden. Amtsgericht Dresden, HRB 31129. Geschäftsführer: Dr.-Ing. Michael Hohmuth

1 0

Issue Rebooting Guest OS in UVMM: Invalid Capability Invoke
by yy18513676366 11 Jun '25

11 Jun '25

Hello, I am currently conducting research using the L4Re system, and I am working on implementing a mechanism to reboot a Guest OS managed by UVMM. In my setup, I run two Linux instances under UVMM. My goal is to reboot one of them without affecting the other. To achieve this, I terminate the target instance by sending a signal using L4Re::Parent::signal, and then I restart it using a Ned script to launch a new instance of the same program. When restarting, I reuse the capability that was created during the first launch of the UVMM instance. However, during execution, the restarted program encounters an "invalid capability invoke" error. Upon investigation, I suspect this happens because l4_task_unmap was called during the restart process, which likely removed kernel objects associated with the original capability. As a result, the reused cap_index no longer refers to a valid object. Based on this, I would appreciate your insights on the following questions: Under what conditions are kernel objects deleted in L4Re? In this rebooting workflow, how can we ensure that capabilities remain valid or that their associated kernel objects are not prematurely unmapped or deleted? If UVMM crashes during operation, how can we design a mechanism to ensure system-wide stability and security? Thank you in advance for your valuable feedback! Best regards, stephen.yang

2 1

L4Re: MIPS syncICache seems to be broken
by Paul Boddie 09 Jun '25

09 Jun '25

Hello, I have been trying to get L4Re working on another MIPS-based SoC and found myself struggling to get the kernel to boot. Initially, I wondered about various exotic concurrency issues, particularly since the SoC has two cores (each with two threads, apparently), but I started to suspect that our old nemesis, cache management, was to blame. Anyway, to save you the long story, I had a closer look at the syncICache routine that resides in the bootstrap package/module: pkg/bootstrap/server/src/ARCH-mips/crt0.S It turns out that syncICache is ineffective due to an erroneous loop continuation test. Here's the offending code: 10: synci 0($a0) addu $a0, $a0, $v0 sltu $v1, $a0, $a1 bne $v1, $0, 10b nop Here, $a0 is the region start and $a1 is the region size, with $v0 being the cache line size. The above code is equivalent to the following: address = start; do { synci 0(address) address += linesize; } while (address < size); The address should, of course, be compared to start + size. So, cache entries may not be flushed, particularly later in regions and at higher addresses, omitting coverage even of the early kernel regions, making kernel execution very unpredictable and unreliable. Perhaps the larger caches make this fault more obvious on this particular SoC. In the startup routine in the bootstrap package/module... pkg/bootstrap/server/src/startup.cc ...there is a declaration for syncICache, along with code to flush the caches, introduced purely for MIPS. To test my suspicions, I wrote a C++ version of the routine with inline assembly language for the machine instructions, fixing the loop continuation test. This seemed to allow the kernel to boot and the system to start up as I originally expected. Would you like me to contribute this fix? I think that unless inline assembly is problematic for some toolchain choices, it might be clearer to maintain this code in C++ rather than entirely in assembly language, thus avoiding such faults in the first place. This problem may have been around since the introduction of MIPS support into L4Re, appearing first in r70 of the public Subversion repository from back in May 2016. Paul

2 2

Can't tcp into VM?
by Richard Clark 05 Jun '25

05 Jun '25

Hey Adam! I’ve gotten the lwip stack working, and two vms running, and the ramdisks expanded a little, and some test processes statically built so I can run them… and the results suggest a firewall in the way, but I don’t see a way to enable the port I want… So lwip stack native to l4re, ipaddr 192.168.1.100, Vm1 at 192.168.1.2 Vm2 at 192.168.1.3 The vms can ping eachother. The vms can ping the lwip stack. Vm1 can establish a tcp connection TO the lwip stack. (and send packets!) Lwip cannot establish a tcp connection TO vm1. I would have tweaked the firewall, but I don’t see one, or any firewall utilities, in your default ramdisks. Suggestions? How would I enable port 5000 from 192.168.1.100 (lwip) to be allowed into vm1? Worst case, I guess I could just start all the connections from the vm, but still, it would be nice to know why it isn’t being allowed. I think this is the last little piece I am missing, and then I can run demos for our customer! 😊 Thanks! Richard

2 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

l4-hackers June 2025