Hello,
Pushing on with my efforts to port L4Re to another MIPS-based SoC, I have been
attempting to activate more than one CPU core. Although there is some support
in Fiasco for the MIPS Coherence Manager (CM) and Cluster Power Controller
(CPC), these technologies are not supported in this particular SoC, which is
related to the JZ4780 used in the MIPS Creator CI20.
However, the Platform_control abstraction (in src/kern/mips/platform_control-
mips.cpp) does seem to provide some flexibility in the way secondary CPUs or
cores may be started. It seems that even in situations where the CM is not
available, the alloc_secondary_boot_code method will assemble a bootstrap
routine from snippets including one, _tramp_mp_cache, that initialises the
caches.
Unfortunately, it appears that this cache initialisation code does not work on
this particular SoC. The method used by_tramp_mp_cache is the familiar one
where the cache tags are invalidated for all cache index entries using the
Index_Store_Tag operation for each cache type. This is supposed to work on any
MIPS CPU, but through some tedious troubleshooting, I discovered that it does
not work for the data cache.
If the data cache variant of the Index_Store_Tag operation is performed for a
range of addresses starting from the base of KSEG0, I found that some kind of
side-effect would occur, possibly affecting the rather critical addresses in
that range. This would prevent Fiasco from continuing to start up, and things
like timers and the UART would not function, although the processor was not
halted and would still run code, illuminating LEDs as a last-resort debugging
tool.
There is some preliminary support for this SoC, as well as the JZ4780, in
Linux and it seems that instead of using Index_Store_Tag, the tag records are
explicitly invalidated and updated. However, it also seems to be the case that
caches can be invalidated using Hit_Invalidate. Changing the cache operations
in _tramp_mp_cache to use this invalidation operation in its two variants
appears to work, avoiding the problems described above.
Does anyone have any thoughts or recollections about this cache initialisation
routine?
On the topic of actually starting secondary cores, this SoC has its own set of
registers for resetting cores, interrupt control, mailboxes, and so on. I have
developed a fairly simple abstraction that performs similar core bootstrapping
work to that of the Cm abstraction (in src/kern/mips/cm.cpp). I then extend
the Platform_control abstraction so that it will call the
alloc_secondary_boot_code method, followed by the appropriate method to start
the secondary cores.
Is there any documentation for the SMP architecture in Fiasco and L4Re? I note
that interprocessor interrupts (IPI) are one element that I might need to
support, probably using the mailboxes.
Thanks in advance for any advice or feedback that may be given!
Paul
