Activating the sigma0 thread in the Fiasco kernel

[Paul Boddie]

Hello,

I've been trying to coerce L4Re and Fiasco.OC to work with the Ben NanoNote, 
which is related to the MIPS Creator CI20 that is (mostly - see earlier 
discussions) supported by this software. There are a few challenges involved, 
such as avoiding MIPS32r2 instructions that the NanoNote's SoC doesn't support 
(JZ4740 in the NanoNote versus JZ4780 in the CI20), and positioning things 
appropriately to avoid upsetting the installed bootloader.

Since I last posted anything about this, I've managed to get Fiasco to go 
about its bootstrapping business, proceeding from the bootstrap package, 
entering the kernel, running kernel_main, doing various initialisation tasks, 
enabling interrupts, and ending up in the init_workload method where a sigma0 
thread and a boot thread are created and activated.

It is at this point that I seem to have encountered a particularly stubborn 
problem: how to get the sigma0 thread to activate and return control to 
init_workload for the boot thread to be activated, then going on to finish 
what little remains of the bootstrapping process. Strangely, the boot thread 
has no difficulty in being activated and returning control if I put it first 
in the sequence, but sigma0 seems to cause something different to happen.

Now, for this activation, it seems that the real action begins in 
Context::switch_cpu, where an exchange of stack pointers occurs and a branch 
is made to the Context::switchin_context method, with the old context 
presumably being made to resume after the branchpoint. As far as I can tell, 
both threads manage to set this up correctly. I see that the new context then 
causes Thread::user_invoke to be called, with execution proceeding via the 
ret_from_user_invoke routine and ultimately into the user task. Looking at the 
address to which the CPU will "return" to in user mode, all seems reasonable 
and consistent with the configuration of sigma0.

However, one thing that baffles me somewhat is the way that interrupts are 
disabled in Thread::user_invoke. Given that the CPU ends up in the task in 
user mode, it would surely need some kind of exception or interrupt for the 
kernel to be re-entered, yet I don't see any operation that re-enables 
interrupts anywhere. Maybe this is the cause of my problem, but I don't 
understand how sigma0 would be different from anything else.

I have needed to change some CPU-level operations to adapt the code to the 
earlier microarchitecture revision, but I don't think I've made any obvious 
mistakes, and I have seemingly figured out how to describe the interrupt 
system because for a while I couldn't get past the Delay::init invocation in 
the bootstrap process which relies on interrupts working. (Some comments in 
the code would have helped me guess what numbers to use in certain 
invocations.)

Does anyone have any thoughts or guidance about what I should be looking at?

Thanks in advance and sorry for the wall of text!

Paul