Fiasco.OC-UX on MIPS?

[Paul Boddie]

Hello again,

Having been using the MIPS-based CI20 single-board computer a bit more 
recently, I wondered if it might be possible to get the user mode Fiasco 
kernel working on MIPS. As I understand it, the UX variant of Fiasco only 
works on IA32 (x86, 32-bit), and although MIPS isn't particularly popular, it 
would at least be useful to me.

However, having looked at the code, it seems like quite a bit of work to 
unpick the IA32-related aspects from the UX-specific code. There appears to be 
a fair amount of descriptor table usage, for instance, and despite reading the 
thesis about it [1], I don't feel I have a decent-enough overview of what 
would really need doing to port the UX variant to another architecture.

So far, I've reviewed the MIPS code which employs a Proc abstraction and has a 
number of functions in the Mips namespace which could be replaced to maintain 
a "virtual" processor in the user mode kernel. It seems that the existing UX 
variant employs an Emulation abstraction for things like the processor status, 
and perhaps the Proc abstraction could take over these responsibilities. Quite 
how many privileged CPU registers would need maintaining probably depends on 
how much other code would still be using them.

This brings me to an awkward difference between IA32 and MIPS: that of the 
page fault handling and memory management. MIPS has no inherent notion of page 
table structures that are held by the CPU and consulted by it when addressing 
errors occur. Instead, an exception handler is invoked to populate various 
privileged registers and to perform privileged instructions. However, it seems 
to me that Fiasco needs to maintain a page directory, anyway, and that the 
user mode kernel is not likely to be attempting the actual memory management 
operations, either.

Consequently, it is tempting to think that the existing UX code might also be 
usable on MIPS. This would actually involve replacing a dedicated page 
directory implementation that appears to be present for MIPS with the one that 
other architectures appear to use. One detail that might need preserving is 
the code path dealing with TLB lookups (as a consequence of addressing 
errors), but it may be the case that page faults are dealt with more directly 
when signals arrive - perhaps what the UX code does now - and that the TLB 
exception vectors will remain unused.

When considering some of these issues, I did wonder about the role of certain 
architecture-specific aspects of the MIPS implementation. The existing UX code 
seems to deal with "vectors" and seems rather connected to interrupt 
descriptor tables (or their emulation). On MIPS, exception vectors are 
typically resident in kernel memory and have specific addresses.

It seems to me that the handling of exceptions (caused by signals) would 
direct execution via these vectors and thus preserve the existing exception.S 
implementation, albeit relocated to user memory. I guess the alternative would 
involve rewriting such things and using emulated processor features directly 
instead of trapping and handling privileged instructions. (I did also wonder 
to what extent the assembly language could also be migrated to C++ even for 
the native MIPS implementation.)

That brings me to a discussion about the purpose of the user mode kernel 
variant. As I understand it, one of the aims is to preserve architecture-
specific code so as to give that code some possible testing and general 
exercise: it might actually help in the development of the kernel more 
generally. Is this a reasonable understanding? Initially, I thought that UX 
would actually be architecture-neutral (in terms of mechanisms; obviously the 
code would be native), but it appears that a stated aim is to be able to run 
existing binaries unchanged. Would an architecture-neutral UX variant make any 
sense?

I suppose I could go on and on, here, but as far as I can tell there are quite 
a few things that would need deciding in order to have a realistic chance of 
ever getting this working, amongst others that I will have forgotten or not 
considered...

 * Redefining the memory layout for kernel and task processes (so as to work
   within the host system constraints)

 * Separating generic and native functionality for various central
   abstractions

 * Replacing the memory management with a mechanism similar to that employed
   by the existing UX code

 * Defining a usable interrupt and exception paradigm (here, I am tempted to
   introduce a second-level controller for user mode interrupts)

 * Determining which modules are required and how the code should be built
   (largely using the existing UX configurations)

 * Testing and integrating the host system functionality required for the user
   mode kernel to function (process tracing, signal handling, system call
   invocations using the trampoline, thread management)

On this last point, I investigated the trampoline mechanism on a MIPS Linux 
system, and it does seem viable. There are a few tricks to getting system 
calls working (that can be learned from glibc and the kernel itself), although 
the trick employed to cancel them may not work. Also, the paper [1] describes 
the need to acquire signal context information by deploying a signal handler 
and having processes capture context data on the kernel's behalf. This may not 
be necessary on MIPS since the interesting details - those that seem to be 
posted on the stack when IA32 exceptions occur - are available via the 
registers exposed by ptrace on MIPS.

Indeed, I learned that ptrace is something of a fickle friend, not necessarily 
providing the same level of support on all systems. And thus, I wondered if 
there is still much sustained interest in the user mode approach or if people 
regard other technologies to be more practical and/or interesting. If you have 
read this far, maybe you might be able to share your thoughts on the topic.

It has been frustrating trying to get a good overview of the code - I'm not 
sure the preprocessor really helps, to be honest - and while I would like to 
be able to get the user mode kernel working on another architecture, I would 
rather like to know if it is really worth my while. Getting a better 
understanding is therefore the first step in finding out.

Paul

[1] http://os.inf.tu-dresden.de/papers_ps/steinberg-beleg.pdf