- l4-hackers - example.com Site

Compatibility of microkernel systems with legacy OSes; was L4Re with GCC 9; Repositories and Roadmaps
by Andrew Warkentin 30 Jan '20

30 Jan '20

On 1/28/20, Martin Decky <martin.decky(a)huawei.com> wrote: > > FYI: The microkernel-based OS Huawei is working on internally has Linux > compatibility (both from the syscall API and from the drivers point of view) > as one of its goals. > Is it natively Unix-like, or is the Linux compatibility layer some kind of containerized "penalty box" that doesn't really integrate into the system? Also, since you're not developing it publicly , that may limit its ability to compete with Linux, which of course has a completely open development model without copyright assignment. In addition, from one article I read, it sounded like even though it is going to be open source eventually, it was designed with the assumption that it would be running on tivoized devices with locked bootloaders, with no way for the user to get full control. > > GNU Hurd obviously also tries to be GNU/Linux compatible (not on the syscall > level, but on the glibc level). Many other microkernel-based systems (e.g. > Genode, just to name one) maintain their own adaptation layers for hosting > Linux device drivers. > >From what I understand, Hurd doesn't implement a lot of Linux-kernel-specific APIs. On 1/28/20, Paul Boddie <paul(a)boddie.org.uk> wrote: > > I actually think that most people agree with you on this, at least when > confronted with the notion for the first time. But I don't think such > considerations should overrule every other consideration: it reminds me of > people in the English-speaking world (particularly in my country of origin) > > who say that if they could be bothered to learn a foreign language, it would > > be "something like Spanish that a lot of people speak", justifying the > choice > in terms of all the millions of individual speakers with whom they could be > > having hypothetical conversations that, naturally, they wouldn't end up > having > anyway. > For my purposes, I would much rather have an OS that runs Linux programs natively rather than having to port every single application I want to run. > > Your project being this one: > > https://gitlab.com/uxrt > > Is there a summary of it anywhere? > This is the closest thing to a summary that I have at the moment: https://gitlab.com/uxrt/uxrt-toplevel/blob/master/architecture_notes > > I wonder whether people don't already rely on things like L4Linux for such > functionality. However, I don't personally think that having boxed-up driver > > frameworks is particularly elegant or plays to the strengths of > microkernel- > based paradigms. In L4Re, I found myself staring down bits of imported Linux > > kernel code and wondering whether things might not have been easier - and > most > certainly clearer - had something been developed independently. > Yes, it's true that it's maybe not the most elegant way to run drivers, but I think it's a worthwhile trade-off for broad hardware support. As far as UX/RT is concerned it's not really that big a sacrifice because it will have relatively little vertical modularization anyway (e.g. instead of separate disk filesystem, partition driver, and disk device driver processes, all three will be incorporated into a single "disk server" process, much as on QNX; this should improve performance over most other microkernel OSes while not really sacrificing much as far as security or error recovery). > > Of course, the real solution is to have a library of functionality that a > variety of systems could use. Then, we wouldn't need to pick over the > remains > of some other project. Being familiar with this industry over a couple of > decades, however, I am fully aware that people exhibit rather strong > tendencies to insist that such cooperation is not possible, usually because > of > something super-special that they insist that only they are doing. > I'd be willing to cooperate on such libraries as long as they don't place unnecessary burdens on client code.

3 9

Hardware-aided Trusted Computing devroom at FOSDEM
by Vasily A. Sartakov 12 Dec '19

12 Dec '19

Hello I would like to invite your to participate in a new devroom: Hardware-aided Trusted Computing. The devroom is devoted to open source projects on top of modern TEEs like Intel SGX, AMD SEV/SVE, Arm TrustZone. You can find the schedule here: https://fosdem.org/2020/schedule/track/hardware_aided_trusted_computing/ I hope this will be interesting for the community. See you at FOSDEM. -- Dr.-Eng. V. A. Sartakov

1 0

Filesystem Architectures with L4Re and L4-Based Systems
by Paul Boddie 02 Dec '19

02 Dec '19

Hello, As this list can attest, I previously spent a bit of time diving into the way dynamic linking is supported by L4Re to get shared mode programs working. This turned out to have a relatively simple fix related to symbols employed by GCC- generated code. But during this exercise, I gained some familiarity with the way libraries are loaded, and this appears to involve the virtual filesystem support, with the loader perusing the filesystem to access libraries, and with these being provided as modules by the "rom" filesystem. Since then, I've spent some time looking at how files are provided by filesystems and accessed by code that uses the conventional C or POSIX library interfaces. It raises a few questions about why things are done in a particular way in L4Re, and which approaches are in use for other L4-based systems, especially those that seek to provide general-purpose, multi-user solutions. What I think I now understand about the virtual filesystem support in L4Re is as follows. The virtual filesystem itself is a "client-side" construct, meaning that it resides within any given program. That programs have their own view of a filesystem is not too different from what the documentation about systems like the Hurd describes and advocates, and I suppose it is a tempting approach because it gives each program the flexibility to be customised in this regard. However, it seems that beyond the internal "mount tree" representation, configuration of the namespace hierarchy is largely done using the L4Re namespace concept, with namespaces acting as directories, and non-namespace objects within the hierarchy interpreted as particular file types. Apart from the interpretation of namespaces as directories, it seems that for the most part, dataspaces are employed to be interpreted as file objects. I will admit that I didn't really look very hard at how different systems provide file access before now, but what surprised me slightly was the apparent lack of delegation in L4Re. As far as I can tell, on systems based on kernels like Linux, the open library function will employ a system call to delegate the matter of finding the file and obtaining a way of accessing it to the familiar monolithic-kernel-plus-filesystems arrangement. On more "exotic" systems (than Linux) like Minix 3 or the Hurd, it appears that delegation to a virtual filesystem server [1] or servers [2] occurs to mediate access to specific files. Some systems like Inferno and Plan 9 employ protocols [3] to formalise the client-server relationship. I had almost expected to find similar things in L4Re, but I could only confirm their absence after reviewing the various VFS abstractions. One thing I wonder about is whether the current L4Re approach is able to satisfactorily deal with filesystem content that needs to serve different user entities, with content having different permission and ownership rights. It seems that the filesystem logic has to be available as a library within a program, which is not necessarily a problem. But to avoid filesystem content being completely exposed to the discretion of user programs (reminiscent of the problems with early Network File System implementations), the library would need to call out to other entities, which doesn't appear to be done with the current L4Re abstractions. One could implement support for, say, the ext3 filesystem in a library, operating on a dataspace provided at some kind of mountpoint in a directory (with the directory being provided by a namespace, of course) but it would be like giving a user program access to a block device in a traditional Unix-like system. Are there any articles about the design of L4Re that might explain the motivations here? And are there any accessible-but-detailed articles about other microkernel-based systems and the way in which they structure their filesystem architectures? I looked around on the TU-Dresden site for materials, but I didn't immediately find anything obviously relevant. Paul [1] https://wiki.minix3.org/doku.php?id=developersguide:vfsinternals [2] https://www.gnu.org/software/hurd/hurd-talk.html#pat [3] http://doc.cat-v.org/inferno/4th_edition/styx

1 1

Alignment checking on ARM
by Al Grant 26 Nov '19

26 Nov '19

I'm building L4 on ARM using GCC 7.4.0 on Cortex-A15. ARCH-arm/head.cc enables alignment checking (sets SCTLR.A=1), but we then see an alignment check fault in init_kip_f.cc. With the default compiler options, the GCC cross tools will assume alignment checking is _not_ enabled and will generate code that may use unaligned access. We can override this by compiling with -mno-unaligned-access. But it would be good to know what is intended. If the intention is to be able to run "mainstream" open-source code in userspace, i.e. build code using the standard Linux ABI, we ought to allow that, and not enable alignment checks in SCTLR. Some code will not be fixed by compiling -mno-unaligned-access (because it casts arbitrary unaligned pointers to word-sized types etc.) So my question is, what is the intention? (1) fault unaligned access in kernel and userspace (2) fault unaligned access in kernel but allow in userspace (3) allow unaligned access in kernel and userspace I'm assuming it can't be (3) as the bootstrap code explicitly enables alignment checking. (2) would be a good compromise, it would enable alignment checks which would pick up some possible faults in the kernel, but it would improve portability of ordinary open-source packages in userspace. IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

2 1

Kernel Helpers segmentation fault
by Sateesh K 14 Nov '19

14 Nov '19

Hi, My setup is as follows: l4linux 4.19.0-l4 Fiasco.OC Raspberry Pi 3 Model B I am running a helloworld.go program which is causing the following stack trace. (gdb) bt full #0 0xffff0f60 in ?? () No symbol table info available. #1 0x0001143c in kernelCAS64 () at /usr/local/go/src/sync/atomic/asm_linux_arm.s:112 No locals. #2 0x0001103c in sync/atomic.loadUint64 (addr=0x10442080, val=48) at /usr/local/go/src/sync/atomic/64bit_arm.go:10 No locals. #3 0x00071f44 in internal/poll.(*fdMutex).rwlock (mu=0x10442080, read=false, ~r1=160) at /usr/local/go/src/internal/poll/fd_mutex.go:130 mutexBit.lo = 4 mutexMask.hi = 2147481600 mutexMask.lo = 0 mutexSema = 0x1044208c mutexWait.hi = 2048 mutexWait.lo = 0 #4 0x00072314 in internal/poll.(*FD).writeLock (fd=0x10442080, ~r0=...) at /usr/local/go/src/internal/poll/fd_mutex.go:237 No locals. #5 0x00072a94 in internal/poll.(*FD).Write (fd=0x10442080, p=..., ~r1=0, ~r2=...) at /usr/local/go/src/internal/poll/fd_unix.go:243 n = -1212420096 nn = 118 --Type <RET> for more, q to quit, c to continue without paging--c #6 0x00073958 in os.(*File).write (b=..., err=..., f=0x1040c0f0, n=2729821 44) at /usr/local/go/src/os/file_unix.go:243 No locals. #7 0x00072ffc in os.(*File).Write (b=..., err=..., f=0x1040c0f0, n=0) at /usr/local/go/src/os/file.go:144 err.data = 0x12c0e8 <fmt.ppFree> "" err.itab = 0x8b7a8 <fmt.Fprintln+20> "\004" ~r2.data = 0x10456014 "" ~r2.itab = 0x0 #8 0x0008b7fc in fmt.Fprintln (a=..., err=..., n=233960, w=...) at /usr/local/go/src/fmt/print.go:255 p = 0x10456080 #9 0x0008b888 in fmt.Println (a=..., err=..., n=0) at /usr/local/go/src/fmt/print.go:264 No locals. #10 0x000916c0 in main.main () at /home/pi/hello.go:6 No locals. On further analysis, I find that segmentation fault is at __kuser_cmpxchg64. I have enabled CONFIG_KUSER_HELPERS=y when building the linux kernel. Any hint please on how to proceed. Best Regards, Sateesh Kalidas

3 3

http://os.inf.tu-dresden.de/pipermail/l4-hackers/2019/008421.html
by Sateesh K 04 Nov '19

04 Nov '19

Hi, I am trying to bring up Raspberry Pi 3 B peripherals (specifically SD card) onto the linux which is onto of Fiasco.OC. I have read the following thread which is close to what I am doing. http://os.inf.tu-dresden.de/pipermail/l4-hackers/2019/008421.html. I have a question, how to get a .dtb file. Please help. Best Regards, Sateesh Kalidas LinkedIn : http://linkedin.com/in/sateeshkalidas <https://www.linkedin.com/in/sateeshkalidas> Twitter: https://twitter.com/sats34

2 1

CfP: FOSDEM 2020 - Microkernels and Component-based OS devroom
by Julian Stecklina 03 Oct '19

03 Oct '19

[Resent due to wrong Reply-To: header] FOSDEM 2020 - Microkernels and Component-based OS devroom CALL FOR PARTICIPATION The developers of several free and open-source microkernel-based operating systems will meet at FOSDEM 2020 [1] in Brussels, Belgium and will share a developer room. The devroom is currently looking for content in the form of talks and activities related to the area of microkernel-, unikernel-, and component-based operating systems. Possible topics include, but are not limited to: * introduction of a specific OS or framework * design of subsystems and the general architecture of an OS * used languages and tools * enabling support for hardware (architectures, device drivers) * enabling support for programming languages * maintenance and testing * security and robustness * trends and challenges * use cases, experiences, lessons learned and demos The deadline for your proposal is on December 1, 2019. Please use the Pentabarf [2] system to submit your proposals. You do not need to create a new account if you already have one. Make sure to include the following in your proposal: * title of your talk (will be printed in the FOSDEM booklet) * your full name (will be printed in the FOSDEM booklet) * a short abstract (one or two paragraphs) * duration of your talk (at least 25 and no longer than 55 min) The official devroom schedule (along with the accepted talks) will be announced on December 15 on the devroom's mailing list [3] and the speakers will be notified via e-mail. The schedule will also be published on the FOSDEM website. You are invited to send any other suggestions for what you want to see or do at the devroom to the list. We will also use this list to organize the traditional devroom dinner that will happen on the first day of FOSDEM. About the devroom Since the first Microkernel devroom at FOSDEM 2012, this devroom has been part of each following FOSDEM. The focus gradually widened to include component-based operating systems. By now it has become a somewhat institutionalized tradition for the community to meet there; to this date over a dozen projects have participated in one way or another. Each of the projects face similar challenges but come up with partially different solutions. Therefore, the goal of the devroom is to bring the various projects together, let them exchange ideas, cross-pollinate and socialize. Social events It also has become a habit that the microkernel projects dine together somewhere in Brussels. The year 2020 will not be any different, so there is going to be a dinner on Saturday night. We try place a reservation, so we will inform you about the exact location and time later. Consult the FOSDEM web and other projects for additional social events such as the famous FOSDEM-organized Friday Beer Event and the FOSDEM-arranged free sightseeing tours for spouses. About FOSDEM FOSDEM is a two-day event organised by volunteers to promote the widespread use of free and open source software. Taking place in the beautiful city of Brussels (Belgium), FOSDEM is widely recognised as the best such conference in Europe. FOSDEM covers a wide spectrum of free and open source software projects, and offers a platform for people to collaborate. To this end, FOSDEM has set up developer rooms (devrooms) with network/internet connectivity and projectors where teams can meet and showcase their projects. Devrooms are a place for teams to discuss, hack and publicly present latest directions, lightning talks, news and discussions. Besides developer rooms, FOSDEM also offers main tracks, lightning talks, certification exams and project stands. Every year, FOSDEM hosts more than 5000 developers at the ULB Solbosch campus. Participation and attendance is totally free, though the organisers gratefully accept donations and sponsorship. No registration necessary. Important dates recap * 2019-12-01: Deadline for submissions * 2019-12-15: Schedule published and speakers notified of acceptance * 2020-02-01: Devroom Dinner (details to be announced) * 2020-02-01/2020-02-02: The devroom takes places (check the schedule) Links [1] https://fosdem.org/ [2] https://penta.fosdem.org/submission/FOSDEM20 [3] https://lists.fosdem.org/listinfo/microkernel-devroom

1 0

CfP: FOSDEM 2020 - Microkernels and Component-based OS devroom
by Julian Stecklina 03 Oct '19

03 Oct '19

FOSDEM 2020 - Microkernels and Component-based OS devroom CALL FOR PARTICIPATION The developers of several free and open-source microkernel-based operating systems will meet at FOSDEM 2020 [1] in Brussels, Belgium and will share a developer room. The devroom is currently looking for content in the form of talks and activities related to the area of microkernel-, unikernel-, and component-based operating systems. Possible topics include, but are not limited to: * introduction of a specific OS or framework * design of subsystems and the general architecture of an OS * used languages and tools * enabling support for hardware (architectures, device drivers) * enabling support for programming languages * maintenance and testing * security and robustness * trends and challenges * use cases, experiences, lessons learned and demos The deadline for your proposal is on December 1, 2019. Please use the Pentabarf [2] system to submit your proposals. You do not need to create a new account if you already have one. Make sure to include the following in your proposal: * title of your talk (will be printed in the FOSDEM booklet) * your full name (will be printed in the FOSDEM booklet) * a short abstract (one or two paragraphs) * duration of your talk (at least 25 and no longer than 55 min) The official devroom schedule (along with the accepted talks) will be announced on December 15 on the devroom's mailing list [3] and the speakers will be notified via e-mail. The schedule will also be published on the FOSDEM website. You are invited to send any other suggestions for what you want to see or do at the devroom to the list. We will also use this list to organize the traditional devroom dinner that will happen on the first day of FOSDEM. About the devroom Since the first Microkernel devroom at FOSDEM 2012, this devroom has been part of each following FOSDEM. The focus gradually widened to include component-based operating systems. By now it has become a somewhat institutionalized tradition for the community to meet there; to this date over a dozen projects have participated in one way or another. Each of the projects face similar challenges but come up with partially different solutions. Therefore, the goal of the devroom is to bring the various projects together, let them exchange ideas, cross-pollinate and socialize. Social events It also has become a habit that the microkernel projects dine together somewhere in Brussels. The year 2020 will not be any different, so there is going to be a dinner on Saturday night. We try place a reservation, so we will inform you about the exact location and time later. Consult the FOSDEM web and other projects for additional social events such as the famous FOSDEM-organized Friday Beer Event and the FOSDEM-arranged free sightseeing tours for spouses. About FOSDEM FOSDEM is a two-day event organised by volunteers to promote the widespread use of free and open source software. Taking place in the beautiful city of Brussels (Belgium), FOSDEM is widely recognised as the best such conference in Europe. FOSDEM covers a wide spectrum of free and open source software projects, and offers a platform for people to collaborate. To this end, FOSDEM has set up developer rooms (devrooms) with network/internet connectivity and projectors where teams can meet and showcase their projects. Devrooms are a place for teams to discuss, hack and publicly present latest directions, lightning talks, news and discussions. Besides developer rooms, FOSDEM also offers main tracks, lightning talks, certification exams and project stands. Every year, FOSDEM hosts more than 5000 developers at the ULB Solbosch campus. Participation and attendance is totally free, though the organisers gratefully accept donations and sponsorship. No registration necessary. Important dates recap * 2019-12-01: Deadline for submissions * 2019-12-15: Schedule published and speakers notified of acceptance * 2020-02-01: Devroom Dinner (details to be announced) * 2020-02-01/2020-02-02: The devroom takes places (check the schedule) Links [1] https://fosdem.org/ [2] https://penta.fosdem.org/submission/FOSDEM20 [3] https://lists.fosdem.org/listinfo/microkernel-devroom

1 0

Fiasco.OC-UX on MIPS?
by Paul Boddie 30 Sep '19

30 Sep '19

Hello again, Having been using the MIPS-based CI20 single-board computer a bit more recently, I wondered if it might be possible to get the user mode Fiasco kernel working on MIPS. As I understand it, the UX variant of Fiasco only works on IA32 (x86, 32-bit), and although MIPS isn't particularly popular, it would at least be useful to me. However, having looked at the code, it seems like quite a bit of work to unpick the IA32-related aspects from the UX-specific code. There appears to be a fair amount of descriptor table usage, for instance, and despite reading the thesis about it [1], I don't feel I have a decent-enough overview of what would really need doing to port the UX variant to another architecture. So far, I've reviewed the MIPS code which employs a Proc abstraction and has a number of functions in the Mips namespace which could be replaced to maintain a "virtual" processor in the user mode kernel. It seems that the existing UX variant employs an Emulation abstraction for things like the processor status, and perhaps the Proc abstraction could take over these responsibilities. Quite how many privileged CPU registers would need maintaining probably depends on how much other code would still be using them. This brings me to an awkward difference between IA32 and MIPS: that of the page fault handling and memory management. MIPS has no inherent notion of page table structures that are held by the CPU and consulted by it when addressing errors occur. Instead, an exception handler is invoked to populate various privileged registers and to perform privileged instructions. However, it seems to me that Fiasco needs to maintain a page directory, anyway, and that the user mode kernel is not likely to be attempting the actual memory management operations, either. Consequently, it is tempting to think that the existing UX code might also be usable on MIPS. This would actually involve replacing a dedicated page directory implementation that appears to be present for MIPS with the one that other architectures appear to use. One detail that might need preserving is the code path dealing with TLB lookups (as a consequence of addressing errors), but it may be the case that page faults are dealt with more directly when signals arrive - perhaps what the UX code does now - and that the TLB exception vectors will remain unused. When considering some of these issues, I did wonder about the role of certain architecture-specific aspects of the MIPS implementation. The existing UX code seems to deal with "vectors" and seems rather connected to interrupt descriptor tables (or their emulation). On MIPS, exception vectors are typically resident in kernel memory and have specific addresses. It seems to me that the handling of exceptions (caused by signals) would direct execution via these vectors and thus preserve the existing exception.S implementation, albeit relocated to user memory. I guess the alternative would involve rewriting such things and using emulated processor features directly instead of trapping and handling privileged instructions. (I did also wonder to what extent the assembly language could also be migrated to C++ even for the native MIPS implementation.) That brings me to a discussion about the purpose of the user mode kernel variant. As I understand it, one of the aims is to preserve architecture- specific code so as to give that code some possible testing and general exercise: it might actually help in the development of the kernel more generally. Is this a reasonable understanding? Initially, I thought that UX would actually be architecture-neutral (in terms of mechanisms; obviously the code would be native), but it appears that a stated aim is to be able to run existing binaries unchanged. Would an architecture-neutral UX variant make any sense? I suppose I could go on and on, here, but as far as I can tell there are quite a few things that would need deciding in order to have a realistic chance of ever getting this working, amongst others that I will have forgotten or not considered... * Redefining the memory layout for kernel and task processes (so as to work within the host system constraints) * Separating generic and native functionality for various central abstractions * Replacing the memory management with a mechanism similar to that employed by the existing UX code * Defining a usable interrupt and exception paradigm (here, I am tempted to introduce a second-level controller for user mode interrupts) * Determining which modules are required and how the code should be built (largely using the existing UX configurations) * Testing and integrating the host system functionality required for the user mode kernel to function (process tracing, signal handling, system call invocations using the trampoline, thread management) On this last point, I investigated the trampoline mechanism on a MIPS Linux system, and it does seem viable. There are a few tricks to getting system calls working (that can be learned from glibc and the kernel itself), although the trick employed to cancel them may not work. Also, the paper [1] describes the need to acquire signal context information by deploying a signal handler and having processes capture context data on the kernel's behalf. This may not be necessary on MIPS since the interesting details - those that seem to be posted on the stack when IA32 exceptions occur - are available via the registers exposed by ptrace on MIPS. Indeed, I learned that ptrace is something of a fickle friend, not necessarily providing the same level of support on all systems. And thus, I wondered if there is still much sustained interest in the user mode approach or if people regard other technologies to be more practical and/or interesting. If you have read this far, maybe you might be able to share your thoughts on the topic. It has been frustrating trying to get a good overview of the code - I'm not sure the preprocessor really helps, to be honest - and while I would like to be able to get the user mode kernel working on another architecture, I would rather like to know if it is really worth my while. Getting a better understanding is therefore the first step in finding out. Paul [1] http://os.inf.tu-dresden.de/papers_ps/steinberg-beleg.pdf

4 11

Unable to create Goos FB in UEFI mode
by Adi Lima 09 Sep '19

09 Sep '19

Hi, I've been trying to make this one work, but in vain, so far... Either using qemu or in real hardware, all failed when booting from UEFI. >From qemu's stdio I know that the boot has been completed successfuly, but it seems like fb-drv was unable to support current default mode from EFI Framebuffer boot (or it is trying to switch to some other mode, and failed?). I was using standard ex_fb_spectrum_cc, with x86-fb.io, x86-fb.cfg and x86-legacy.devs, all loaded by grub2 as Multiboot2 and Module2 Everything was fine if booted using BIOS, but I haven't try on UEFI mode with CSM. Can you give me a hint of what to try next, please? Thanks in advance :-) -- FX. J. Adi Lima Trivision Studio

2 1