Hi Joshua, hi @ all,
I start this year with first experiments in implementation of an kernel (based on L4.X2) on PowerPC, mostly to learn more about PowerPC/HW and design philosophies. So I read your posting with attention and have some questions ... and hope your answers would made it easier for my to understand.
You recommend "enabling virtual addressing immediatly". Did I understand it right, that you use virtual memory in kernel mode? Is this only for protection or means this, the kernel memory is vitual? There are other kernels using virtual memory in kernel mode?
My design doesn't have a kernel section in the tasks address space (on 32 bit machines a task is 4GB). I agonise to solve some of the problems with this design. In the case StringItems used to send Msg's from one space to another (means copy the information in the kernel) the message buffer on both sides can scattered and located across page bounderies. Handling this seems easier if the kernel has the same view to memory and pionters to memory regions (buffers) but this destroy my approach of lockfreeness, because the String Buffers in different tasks can by overlap or locate at the same virtual adress. (The sendphase of StringItems should by interruptable and at any time it should by possible to handle as much as possible StrinItem sends in the same time.) Handling this in kernel with the view of real memory avoid overlapping buffer regions from different threads. Do you see the issue like I and is there a solution I didn't see?
As I see on the homepage, the kernel runs on a G3 (aka PPC750). So you use the normal hardware table search?
Thanks for helping.
Regards
Kay-Uwe.
Am 07.02.2006 um 17:35 schrieb Joshua LeVasseur:
The PowerPC jumps to predefined code locations when delivering interrupts and traps, starting at 0x100 (with virtual addressing disabled). You only need to install code at those locations; PowerPC has no interrupt descriptors. I recommend enabling virtual addressing immediately, even while handling page faults; I've seen other kernels implement excessive logic in assembler, executing in physical mode; that isn't necessary. I enable virtual addressing within 19 or so instructions.
Maybe try volume 3 here: http://www-128.ibm.com/developerworks/eserver/articles/archguide.html I've never read those documents; I used the 603 architecture manual when porting L4Ka::Pistachio.
Consider reading about how I made my L4Ka::Pistachio PowerPC implementation frigging fast: http://l4ka.org/projects/pistachio/powerpc/ppc.pdf It covers:
- Fast system calls (without wasting a register for a system call
number, unlike all other kernels out there that I've seen)
- Fast address space ID management (the way ASIDs were meant to be
managed on the PowerPC, and again, the other kernels out there are doing it wrong)
- Fast page hash management. Most kernels perform O(n) searches
through the page hash when manually locating an entry. I instead perform O(1) lookups.
I recommend that you avoid looking at open source kernels for guidance. They don't push the limits of PowerPC.
-Josh
On Feb 7, 2006, at 16:18, Matthias Lange wrote:
Hi,
I am stuck with porting the Fiasco microkernel to PowerPC. I can't find any clear documentation about how to setup a interrupt description table. So, these are my questions:
- How is an interrupt descriptor formed on PowerPC? How does it
look? 2. Where (in memory) do I need to store these descriptors?
The IBM PowerPC Programming Manual gives me a good overview but no information on how to implement it. Can someone please point me to some meaningful documentation?
Regards,
Matthias.
Hi Kay-Uwe,
Sorry about the slow response; super busy here.
You recommend "enabling virtual addressing immediatly". Did I understand it right, that you use virtual memory in kernel mode? Is this only for protection or means this, the kernel memory is vitual? There are other kernels using virtual memory in kernel mode?
Probably all kernels use virtual addressing in kernel mode. If I remember correctly, the processor executes with degraded performance in physical mode, but maybe I'm thinking about other processor architectures.
We need virtual addressing for Pistachio, since the kernel algorithms rely on the virtual addresses. For example, the TCBs are managed via virtual addresses, to help reduce the number of TCB validity checks on the fast path, and for fast TCB lookup. Other features requiring virtual addressing: string copy (temporary mapping), UTCBs (contiguous area).
My design doesn't have a kernel section in the tasks address space (on 32 bit machines a task is 4GB). I agonise to solve some of the problems with this design. In the case StringItems used to send Msg's from one space to another (means copy the information in the kernel) the message buffer on both sides can scattered and located across page bounderies. Handling this seems easier if the kernel has the same view to memory and pionters to memory regions (buffers) but this destroy my approach of lockfreeness, because the String Buffers in different tasks can by overlap or locate at the same virtual adress. (The sendphase of StringItems should by interruptable and at any time it should by possible to handle as much as possible StrinItem sends in the same time.) Handling this in kernel with the view of real memory avoid overlapping buffer regions from different threads. Do you see the issue like I and is there a solution I didn't see?
The solution is to invalidate the string copy mapping when the thread is preempted. When the thread is resumed, you'll get a page fault on the string copy, and then you restore the mapping area. PowerPC's segments are perfect for solving this problem, because of the address space IDs: you dedicate one segment to string copy; you copy the segment of the destination thread's target area into the dedicated segment; and then you can copy. When preempting, and when finished, you invalidate the dedicated copy segment.
As I see on the homepage, the kernel runs on a G3 (aka PPC750). So you use the normal hardware table search?
Yes, I use the hardware's hashed page table. Since the hashed page table is limited in size, I also maintain a master copy of the information in a set of forward-mapped page tables. Perhaps the forward-mapped page tables are unnecessary in PowerPC; I need them in Pistachio to be compatible with the architecture-neutral code (mapping database).
Regards, Josh
Thanks for helping.
Regards
Kay-Uwe.
Am 07.02.2006 um 17:35 schrieb Joshua LeVasseur:
The PowerPC jumps to predefined code locations when delivering interrupts and traps, starting at 0x100 (with virtual addressing disabled). You only need to install code at those locations; PowerPC has no interrupt descriptors. I recommend enabling virtual addressing immediately, even while handling page faults; I've seen other kernels implement excessive logic in assembler, executing in physical mode; that isn't necessary. I enable virtual addressing within 19 or so instructions.
Maybe try volume 3 here: http://www-128.ibm.com/developerworks/eserver/articles/archguide.html I've never read those documents; I used the 603 architecture manual when porting L4Ka::Pistachio.
Consider reading about how I made my L4Ka::Pistachio PowerPC implementation frigging fast: http://l4ka.org/projects/pistachio/powerpc/ppc.pdf It covers:
- Fast system calls (without wasting a register for a system call
number, unlike all other kernels out there that I've seen)
- Fast address space ID management (the way ASIDs were meant to be
managed on the PowerPC, and again, the other kernels out there are doing it wrong)
- Fast page hash management. Most kernels perform O(n) searches
through the page hash when manually locating an entry. I instead perform O(1) lookups.
I recommend that you avoid looking at open source kernels for guidance. They don't push the limits of PowerPC.
-Josh
On Feb 7, 2006, at 16:18, Matthias Lange wrote:
Hi,
I am stuck with porting the Fiasco microkernel to PowerPC. I can't find any clear documentation about how to setup a interrupt description table. So, these are my questions:
- How is an interrupt descriptor formed on PowerPC? How does it
look? 2. Where (in memory) do I need to store these descriptors?
The IBM PowerPC Programming Manual gives me a good overview but no information on how to implement it. Can someone please point me to some meaningful documentation?
Regards,
Matthias.
Hi Joshua,
Am 01.03.2006 um 08:43 schrieb Joshua LeVasseur:
Hi Kay-Uwe,
Sorry about the slow response; super busy here.
Don't worry, I'm very glad to get such a detailed answer. Thanks, this was helping to understand things better.
You recommend "enabling virtual addressing immediatly". Did I understand it right, that you use virtual memory in kernel mode? Is this only for protection or means this, the kernel memory is vitual? There are other kernels using virtual memory in kernel mode?
Probably all kernels use virtual addressing in kernel mode. If I remember correctly, the processor executes with degraded performance in physical mode, but maybe I'm thinking about other processor architectures.
I take a deeper look in the darwin kernel sources and yes, it seems that they try to get the MMU running as fast as possible.
We need virtual addressing for Pistachio, since the kernel algorithms rely on the virtual addresses. For example, the TCBs are managed via virtual addresses, to help reduce the number of TCB validity checks on the fast path, and for fast TCB lookup. Other features requiring virtual addressing: string copy (temporary mapping), UTCBs (contiguous area).
My design doesn't have a kernel section in the tasks address space (on 32 bit machines a task is 4GB). I agonise to solve some of the problems with this design. In the case StringItems used to send Msg's from one space to another (means copy the information in the kernel) the message buffer on both sides can scattered and located across page bounderies. Handling this seems easier if the kernel has the same view to memory and pionters to memory regions (buffers) but this destroy my approach of lockfreeness, because the String Buffers in different tasks can by overlap or locate at the same virtual adress. (The sendphase of StringItems should by interruptable and at any time it should by possible to handle as much as possible StrinItem sends in the same time.) Handling this in kernel with the view of real memory avoid overlapping buffer regions from different threads. Do you see the issue like I and is there a solution I didn't see?
The solution is to invalidate the string copy mapping when the thread is preempted. When the thread is resumed, you'll get a page fault on the string copy, and then you restore the mapping area. PowerPC's segments are perfect for solving this problem, because of the address space IDs: you dedicate one segment to string copy; you copy the segment of the destination thread's target area into the dedicated segment; and then you can copy. When preempting, and when finished, you invalidate the dedicated copy segment.
My biggest mistake in thinking about the kernel in virtual mode was, to see the kernel as a unchangeable adress space. But the kernel is more an extension of a task and I can map both segments (source and destination) in the kernels adress space and if I switch the thread the mapping is changed (switched) also. This seems to work (theoretical, programming need more time). So it is no more than change the PPC Segment-Registers by thread switches.
As I see on the homepage, the kernel runs on a G3 (aka PPC750). So you use the normal hardware table search?
Yes, I use the hardware's hashed page table. Since the hashed page table is limited in size, I also maintain a master copy of the information in a set of forward-mapped page tables. Perhaps the forward-mapped page tables are unnecessary in PowerPC; I need them in Pistachio to be compatible with the architecture-neutral code (mapping database).
Regards, Josh
Thanks for the explanation
Regards
Kay-Uwe
l4-hackers@os.inf.tu-dresden.de