Hello again,
Having been using the MIPS-based CI20 single-board computer a bit more recently, I wondered if it might be possible to get the user mode Fiasco kernel working on MIPS. As I understand it, the UX variant of Fiasco only works on IA32 (x86, 32-bit), and although MIPS isn't particularly popular, it would at least be useful to me.
However, having looked at the code, it seems like quite a bit of work to unpick the IA32-related aspects from the UX-specific code. There appears to be a fair amount of descriptor table usage, for instance, and despite reading the thesis about it [1], I don't feel I have a decent-enough overview of what would really need doing to port the UX variant to another architecture.
So far, I've reviewed the MIPS code which employs a Proc abstraction and has a number of functions in the Mips namespace which could be replaced to maintain a "virtual" processor in the user mode kernel. It seems that the existing UX variant employs an Emulation abstraction for things like the processor status, and perhaps the Proc abstraction could take over these responsibilities. Quite how many privileged CPU registers would need maintaining probably depends on how much other code would still be using them.
This brings me to an awkward difference between IA32 and MIPS: that of the page fault handling and memory management. MIPS has no inherent notion of page table structures that are held by the CPU and consulted by it when addressing errors occur. Instead, an exception handler is invoked to populate various privileged registers and to perform privileged instructions. However, it seems to me that Fiasco needs to maintain a page directory, anyway, and that the user mode kernel is not likely to be attempting the actual memory management operations, either.
Consequently, it is tempting to think that the existing UX code might also be usable on MIPS. This would actually involve replacing a dedicated page directory implementation that appears to be present for MIPS with the one that other architectures appear to use. One detail that might need preserving is the code path dealing with TLB lookups (as a consequence of addressing errors), but it may be the case that page faults are dealt with more directly when signals arrive - perhaps what the UX code does now - and that the TLB exception vectors will remain unused.
When considering some of these issues, I did wonder about the role of certain architecture-specific aspects of the MIPS implementation. The existing UX code seems to deal with "vectors" and seems rather connected to interrupt descriptor tables (or their emulation). On MIPS, exception vectors are typically resident in kernel memory and have specific addresses.
It seems to me that the handling of exceptions (caused by signals) would direct execution via these vectors and thus preserve the existing exception.S implementation, albeit relocated to user memory. I guess the alternative would involve rewriting such things and using emulated processor features directly instead of trapping and handling privileged instructions. (I did also wonder to what extent the assembly language could also be migrated to C++ even for the native MIPS implementation.)
That brings me to a discussion about the purpose of the user mode kernel variant. As I understand it, one of the aims is to preserve architecture- specific code so as to give that code some possible testing and general exercise: it might actually help in the development of the kernel more generally. Is this a reasonable understanding? Initially, I thought that UX would actually be architecture-neutral (in terms of mechanisms; obviously the code would be native), but it appears that a stated aim is to be able to run existing binaries unchanged. Would an architecture-neutral UX variant make any sense?
I suppose I could go on and on, here, but as far as I can tell there are quite a few things that would need deciding in order to have a realistic chance of ever getting this working, amongst others that I will have forgotten or not considered...
* Redefining the memory layout for kernel and task processes (so as to work within the host system constraints)
* Separating generic and native functionality for various central abstractions
* Replacing the memory management with a mechanism similar to that employed by the existing UX code
* Defining a usable interrupt and exception paradigm (here, I am tempted to introduce a second-level controller for user mode interrupts)
* Determining which modules are required and how the code should be built (largely using the existing UX configurations)
* Testing and integrating the host system functionality required for the user mode kernel to function (process tracing, signal handling, system call invocations using the trampoline, thread management)
On this last point, I investigated the trampoline mechanism on a MIPS Linux system, and it does seem viable. There are a few tricks to getting system calls working (that can be learned from glibc and the kernel itself), although the trick employed to cancel them may not work. Also, the paper [1] describes the need to acquire signal context information by deploying a signal handler and having processes capture context data on the kernel's behalf. This may not be necessary on MIPS since the interesting details - those that seem to be posted on the stack when IA32 exceptions occur - are available via the registers exposed by ptrace on MIPS.
Indeed, I learned that ptrace is something of a fickle friend, not necessarily providing the same level of support on all systems. And thus, I wondered if there is still much sustained interest in the user mode approach or if people regard other technologies to be more practical and/or interesting. If you have read this far, maybe you might be able to share your thoughts on the topic.
It has been frustrating trying to get a good overview of the code - I'm not sure the preprocessor really helps, to be honest - and while I would like to be able to get the user mode kernel working on another architecture, I would rather like to know if it is really worth my while. Getting a better understanding is therefore the first step in finding out.
Paul
[1] http://os.inf.tu-dresden.de/papers_ps/steinberg-beleg.pdf