Hello,
I have been trying to get L4Re working on another MIPS-based SoC and found
myself struggling to get the kernel to boot. Initially, I wondered about
various exotic concurrency issues, particularly since the SoC has two cores
(each with two threads, apparently), but I started to suspect that our old
nemesis, cache management, was to blame.
Anyway, to save you the long story, I had a closer look at the syncICache
routine that resides in the bootstrap package/module:
pkg/bootstrap/server/src/ARCH-mips/crt0.S
It turns out that syncICache is ineffective due to an erroneous loop
continuation test. Here's the offending code:
10:
synci 0($a0)
addu $a0, $a0, $v0
sltu $v1, $a0, $a1
bne $v1, $0, 10b
nop
Here, $a0 is the region start and $a1 is the region size, with $v0
being the cache line size. The above code is equivalent to the following:
address = start;
do
{
synci 0(address)
address += linesize;
}
while (address < size);
The address should, of course, be compared to start + size. So, cache entries
may not be flushed, particularly later in regions and at higher addresses,
omitting coverage even of the early kernel regions, making kernel execution
very unpredictable and unreliable. Perhaps the larger caches make this fault
more obvious on this particular SoC.
In the startup routine in the bootstrap package/module...
pkg/bootstrap/server/src/startup.cc
...there is a declaration for syncICache, along with code to flush the caches,
introduced purely for MIPS. To test my suspicions, I wrote a C++ version of
the routine with inline assembly language for the machine instructions, fixing
the loop continuation test. This seemed to allow the kernel to boot and the
system to start up as I originally expected.
Would you like me to contribute this fix? I think that unless inline assembly
is problematic for some toolchain choices, it might be clearer to maintain
this code in C++ rather than entirely in assembly language, thus avoiding such
faults in the first place.
This problem may have been around since the introduction of MIPS support into
L4Re, appearing first in r70 of the public Subversion repository from back in
May 2016.
Paul
