All,
I am currently running L4Linux2.6.14 on the Fiasco l4v2 microkernel using i386 arch with mostly default configurations. I need the ability to track the program counter and registers of all threads in the L4Linux task. Here's exactly what I would like to be able to do from a monitoring L4 task:
-Obtain the PC/regs for the first thread (thread 0) for L4Linux just before the thread executes it's first instruction. It would be nice to make the thread wait for a signal before beginning execution.
-Obtain the PC/regs for any additional thread created or any thread moved for L4Linux (i.e. from a l4_thread_ex_regs call) just before that thread executes. Again, it would be nice to make that thread wait for a signal before beginning its execution.
-Randomly preemptively obtain the PC/regs for all threads of L4Linux.
-Obtain the PC/regs for all threads of L4Linux on demand.
-Receive notification of thread exit for all threads of L4Linux.
There are a few items that I do not yet fully understand that *may* make matters more difficult:
-The setup code executing in libloader.s.so.
-Trampoline code
There are a few assumptions I am willing to have for initial testing to make the development easier:
-Assume Task number of L4Linux is known by the microkernel (i.e. hard code it in).
-Assume Task number of monitoring thread is known by the microkernel.
As a first crack, I have added some code to the microkernel to print out information on the L4Linux task, to see if that information would be good to transfer to the monitoring task. Below is a summary of things I have tried and results seen:
MODIFICATION #1 l4/kernel/fiasco/src/kern/context.cpp Context::schedule() Goal: Track current PC of L4Linux threads ======================= ... /* in the for loop */ Sys_task_new_frame *regs = sys_frame_cast<Sys_task_new_frame>(next_to_run->regs()); L4_uid id = regs->dst(); if (id.task() == 0xC) // hard coded task of L4Linux { unsigned int ip = next_to_run->regs()->ip(); unsigned int sp = next_to_run->regs()->sp(); printf("task: %d.%d @ ip: %08x sp: %08x\n", id.task(), id.lthread(), ip, sp); } ... ======================= Results: task: 12.3 @ ip: 004ccb0b sp: b00fff80 task: 12.3 @ ip: 004cd303 sp: b01fff7c task: 12.4 @ ip: 004cca9d sp: b00fff80 task: 12.5 @ ip: 00402ae6 sp: 00657f50 ======================= Notes: 004ccb0b - right after an 'int 0x30' in timer_irq_thread 004cd303 - right after an 'int 0x30' in irq_thread_hw 004cca9d - right after an 'int 0x30' in timer_irq_thread 00402ae6 - right after an 'int 0x30' in l4x_idle
MODIFICATION #2 l4/kernel/fiasco/src/kern/thread-syscall.cpp Thread::sys_task_new() Goal: Track starting PC of main L4Linux thread (i.e. thread 0) ======================= ... /* near tail of for loop, just before return */ if (taskno == 0xC) // hard coded L4Linux task ID { printf("new task: %d @ ip: %08x sp: %08x\n", taskno, regs->ip(), regs->sp()); } ======================= Results: new task: 12 @ ip: 00015ea0 sp: 00008f6c ======================= Notes: 00015ea0 - part of the libloader.s.so code
MODIFICATION #3 l4/kernel/fiasco/src/kern/thread-syscall.cpp Thread::sys_thread_ex_regs() Goal: Track PC entry of new L4Linux threads ======================= ... if (!dst->exists()) { unsigned int task_num = dst_id.task(); unsigned int lthread_num = regs->thread(); new_thread = true; check (new (dst_id) Thread (dst_task, dst_id, sched()->prio(),
mcp()) == dst); if (taskno == 0xC) { printf("new thread: %d.%d @ ip: %08x sp: %08x\n", task_num, lthread_num, regs->ip(), regs->sp()); } } else { unsigned int task_num = dst_id.task(); unsigned int lthread_num = regs->thread(); if (taskno == 0xC) { printf("modified thread: %d.%d @ ip: %08x sp: %08x\n", task_num, lthread_num, regs->ip(), regs->sp()); }
} ... ======================= Results: modified thread: 12.0 @ ip: ffffffff sp: ffffffff modified thread: 12.0 @ ip: ffffffff sp: ffffffff new thread: 12.1 @ ip: 0001ac10 sp: afe00000 new thread: 12.2 @ ip: 0001ac10 sp: aff00000 modified thread: 12.2 @ ip: ffffffff sp: ffffffff new thread: 12.3 @ ip: 0001ac10 sp: 00657ff8 modified thread: 12.3 @ ip: ffffffff sp: ffffffff new thread: 12.4 @ ip: 0001ac10 sp: b0100000 modified thread: 12.3 @ ip: ffffffff sp: ffffffff new thread: 12.5 @ ip: 0001ac10 sp: b0200000 modified thread: 12.3 @ ip: ffffffff sp: ffffffff new thread: 12.6 @ ip: 0001ac10 sp: b0300000 new thread: 12.7 @ ip: 0001ac10 sp: b0400000 ======================= Notes: 0001ac10 - part of the libloader.s.so code ffffffff - not really useful. wrong data perhaps.
QUESTIONS:
GENERAL QUESTIONS -Is there any easy to delay execution of newly created threads, both new task thread 0 and additional threads? -Is there an easy way to randomly interrupt L4Linux threads to check their PC value? -Do any L4Linux threads exit?
MODIFICATION #1 -Seems like threads resume execution right after an interrupt. I don't see threads getting preempted in the middle of servicing a system call, for example. Any ideas why not? Further, is there one thread dedicated to handling system calls? -Is there any better way to track where all the L4Linux threads are? Maybe somewhere else besides the scheduler? Are any other classes/functions helpful to get this information?
MODIFICATION #2 -I believe this is working correctly. See anything wrong here?
MODIFICATION #3 -I would like to catch all instances of creating new threads for L4Linux. Will this current approach work, or is it not correct? If my implementation is correct: -Why are L4Linux threads getting modified? -Any idea why my ip and sp values are 0xffffffff for modified threads? -Looks like all new threads start at the same place. How would they distinguish from one another in terms of functionality? Is that encoded in their stack (e.g. when they issue a 'ret', they'll return to different locations or maybe arguments to their function are on the stack?)
Any other comments you have would also be appreciated. Thanks!
Julian
Hi Julian,
that are a lot of questions which span quite same area. I will try to say something to some of them ...
I am currently running L4Linux2.6.14 on the Fiasco l4v2 microkernel using i386 arch with mostly default configurations. I need the ability to track the program counter and registers of all threads in the L4Linux task. Here's exactly what I would like to be able to do from a monitoring L4 task:
I don't know what kind of cooperation between tasks you assume, but you could try to instrument all tasks in the system to give you the required information via shared memory, without hacking the kernel.
-Obtain the PC/regs for the first thread (thread 0) for L4Linux just before the thread executes it's first instruction. It would be nice to make the thread wait for a signal before beginning execution.
The registers before the first instruction seem to be useless. IP and SP are set via exregs, the rest should be execution defined. You could directly get IP and SP from the exregser.
-Obtain the PC/regs for any additional thread created or any thread moved for L4Linux (i.e. from a l4_thread_ex_regs call) just before that thread executes. Again, it would be nice to make that thread wait for a signal before beginning its execution.
-Randomly preemptively obtain the PC/regs for all threads of L4Linux.
I don't know what you mean by "Randomly preemptively".
You could hack something in into the timeout interrupt handler of fiasco and use somewhat random timeouts.
-Obtain the PC/regs for all threads of L4Linux on demand.
Use exregs, for the IP, SP you can directly get the answer by issueing -1, IIRC. This should also explain your 0xffffffff below.
-Receive notification of thread exit for all threads of L4Linux.
Hack a notification into the thread lib? Maybe use the events server?
Cheers, Martin
Hi Martin,
Thanks for your response. A few clarifications to let you know where I am coming from:
Martin Pohlack wrote:
I don't know what kind of cooperation between tasks you assume, but you could try to instrument all tasks in the system to give you the required information via shared memory, without hacking the kernel.
Actually, I assume no cooperation between tasks. The purpose of the monitoring task is to test the trustworthyness of the L4Linux task, so I don't want to trust a possibly untrustworthy L4Linux task to cooperate.
-Obtain the PC/regs for the first thread (thread 0) for L4Linux just before the thread executes it's first instruction. It would be nice to make the thread wait for a signal before beginning execution.
The registers before the first instruction seem to be useless. IP and SP are set via exregs, the rest should be execution defined. You could directly get IP and SP from the exregser.
-Obtain the PC/regs for any additional thread created or any thread moved for L4Linux (i.e. from a l4_thread_ex_regs call) just before that thread executes. Again, it would be nice to make that thread wait for a signal before beginning its execution.
-Randomly preemptively obtain the PC/regs for all threads of L4Linux.
I don't know what you mean by "Randomly preemptively".
You could hack something in into the timeout interrupt handler of fiasco and use somewhat random timeouts.
The thinking behind random preemption is to somehow ensure that the monitoring agent gets a random sample of where the PC is. I was thinking along the lines of randomly change the timeout interrupt in fact. I'm not sure how good this would be for performance though; just a loose idea right now.
-Obtain the PC/regs for all threads of L4Linux on demand.
Use exregs, for the IP, SP you can directly get the answer by issueing -1, IIRC. This should also explain your 0xffffffff below.
Not completely sure what you mean here. Are you refering to the l4_thread_ex_regs system call? What do you mean by issueing -1, IIRC? If so, is it possible to use that from within the fiasco kernel? Or did you mean do that from with the L4Linux task? Also, I believe that does explain the 0xffffffff addresses. I must be seeing the 0xffffffff for IP and SP when a l4_thread_ex_regs is called with those values filled in for eip/esp.
Thanks!
Julian
On Mon Dec 12, 2005 at 15:50:17 -0500, Julian Grizzard wrote:
to track the program counter and registers of all threads in the L4Linux task. Here's exactly what I would like to be able to do from a monitoring L4 task:
I'll throw in some comments.
-Obtain the PC/regs for the first thread (thread 0) for L4Linux just before the thread executes it's first instruction. It would be nice to make the thread wait for a signal before beginning execution.
The regs are useless is this case as they haven't been filled in with anything. Only ip and sp are set. Making the thread wait is possible by changing the startup method. The monitor must be the exception handler then.
-Obtain the PC/regs for any additional thread created or any thread moved for L4Linux (i.e. from a l4_thread_ex_regs call) just before that thread executes. Again, it would be nice to make that thread wait for a signal before beginning its execution.
If the threads are started the same way as above this should be possible.
-Randomly preemptively obtain the PC/regs for all threads of L4Linux.
Getting the ip is possible, getting all the regs is more complicated due to our multiple extry paths into the kernel and thus different stack frames.
-Obtain the PC/regs for all threads of L4Linux on demand.
What's the difference from above? Works the same way, I'd guess.
-Receive notification of thread exit for all threads of L4Linux.
Threads do not exit, they just go to sleep (forever). So a thread exit can only be a user level thing.
There are a few items that I do not yet fully understand that *may* make matters more difficult:
-The setup code executing in libloader.s.so.
-Trampoline code
There are a few assumptions I am willing to have for initial testing to make the development easier:
-Assume Task number of L4Linux is known by the microkernel (i.e. hard code it in).
I'd say that's a really minor issue you have there ;)
-Assume Task number of monitoring thread is known by the microkernel.
As a first crack, I have added some code to the microkernel to print out information on the L4Linux task, to see if that information would be good to transfer to the monitoring task. Below is a summary of things I have tried and results seen:
MODIFICATION #1 l4/kernel/fiasco/src/kern/context.cpp Context::schedule() Goal: Track current PC of L4Linux threads ======================= ... /* in the for loop */ Sys_task_new_frame *regs = sys_frame_cast<Sys_task_new_frame>(next_to_run->regs()); L4_uid id = regs->dst(); if (id.task() == 0xC) // hard coded task of L4Linux { unsigned int ip = next_to_run->regs()->ip(); unsigned int sp = next_to_run->regs()->sp(); printf("task: %d.%d @ ip: %08x sp: %08x\n", id.task(), id.lthread(), ip, sp); } ... ======================= Results: task: 12.3 @ ip: 004ccb0b sp: b00fff80 task: 12.3 @ ip: 004cd303 sp: b01fff7c task: 12.4 @ ip: 004cca9d sp: b00fff80 task: 12.5 @ ip: 00402ae6 sp: 00657f50 ======================= Notes: 004ccb0b - right after an 'int 0x30' in timer_irq_thread 004cd303 - right after an 'int 0x30' in irq_thread_hw 004cca9d - right after an 'int 0x30' in timer_irq_thread 00402ae6 - right after an 'int 0x30' in l4x_idle
This makes perfectly sense.
MODIFICATION #2 l4/kernel/fiasco/src/kern/thread-syscall.cpp Thread::sys_task_new() Goal: Track starting PC of main L4Linux thread (i.e. thread 0) ======================= ... /* near tail of for loop, just before return */ if (taskno == 0xC) // hard coded L4Linux task ID { printf("new task: %d @ ip: %08x sp: %08x\n", taskno, regs->ip(), regs->sp()); } ======================= Results: new task: 12 @ ip: 00015ea0 sp: 00008f6c ======================= Notes: 00015ea0 - part of the libloader.s.so code
This can be expected.
MODIFICATION #3 l4/kernel/fiasco/src/kern/thread-syscall.cpp Thread::sys_thread_ex_regs() Goal: Track PC entry of new L4Linux threads ======================= ... if (!dst->exists()) { unsigned int task_num = dst_id.task(); unsigned int lthread_num = regs->thread(); new_thread = true; check (new (dst_id) Thread (dst_task, dst_id, sched()->prio(),
mcp()) == dst); if (taskno == 0xC) { printf("new thread: %d.%d @ ip: %08x sp: %08x\n", task_num, lthread_num, regs->ip(), regs->sp()); }} else { unsigned int task_num = dst_id.task(); unsigned int lthread_num = regs->thread(); if (taskno == 0xC) { printf("modified thread: %d.%d @ ip: %08x sp: %08x\n", task_num, lthread_num, regs->ip(), regs->sp()); }
} ... ======================= Results: modified thread: 12.0 @ ip: ffffffff sp: ffffffff modified thread: 12.0 @ ip: ffffffff sp: ffffffff new thread: 12.1 @ ip: 0001ac10 sp: afe00000 new thread: 12.2 @ ip: 0001ac10 sp: aff00000 modified thread: 12.2 @ ip: ffffffff sp: ffffffff new thread: 12.3 @ ip: 0001ac10 sp: 00657ff8 modified thread: 12.3 @ ip: ffffffff sp: ffffffff new thread: 12.4 @ ip: 0001ac10 sp: b0100000 modified thread: 12.3 @ ip: ffffffff sp: ffffffff new thread: 12.5 @ ip: 0001ac10 sp: b0200000 modified thread: 12.3 @ ip: ffffffff sp: ffffffff new thread: 12.6 @ ip: 0001ac10 sp: b0300000 new thread: 12.7 @ ip: 0001ac10 sp: b0400000 ======================= Notes: 0001ac10 - part of the libloader.s.so code
Yup.
ffffffff - not really useful. wrong data perhaps.
No, see the spec, when ip or sp are -1 then they're not changed, i.e. ex_regs is giving back values only.
QUESTIONS:
GENERAL QUESTIONS -Is there any easy to delay execution of newly created threads, both new task thread 0 and additional threads?
What one could do is put an exception handler task between the loader and l4linux and intercept interesting syscalls. This way you also get notifications for new threads. There are probably things which need to be enhanced/fixed/whatever in the kernel but theoretically this could work this way.
-Is there an easy way to randomly interrupt L4Linux threads to check their PC value?
If you monitor task runs L4Linux is not running (UP system) so you can just exregs them to query the ip.
-Do any L4Linux threads exit?
In the context of L4Linux this is possible, in the context of L4 it's not (see above).
MODIFICATION #1 -Seems like threads resume execution right after an interrupt.
int 0x30 is the ipc system call.
I don't see threads getting preempted in the middle of servicing a system call, for example. Any ideas why not?
Bad luck. I guess you'll need some threads with a higher prio doing some work to actually see this.
Further, is there one thread dedicated to handling system calls?
Yes, only one thread is handling Linux system calls.
-Is there any better way to track where all the L4Linux threads are? Maybe somewhere else besides the scheduler? Are any other classes/functions helpful to get this information?
What you are doing is reading the ip and sp from the return frame, you can basically do this from everywhere when you run on the behalf of some user thread. It's not limited to the schedule function.
MODIFICATION #2 -I believe this is working correctly. See anything wrong here?
Possible.
MODIFICATION #3 -I would like to catch all instances of creating new threads for L4Linux. Will this current approach work, or is it not correct?
Maybe you want to better hook into Thread::initialize so you get task_new and threads in one go.
If my implementation is correct: -Why are L4Linux threads getting modified? -Any idea why my ip and sp values are 0xffffffff for modified threads?
Again, see the spec. Querying pager/preempter is done with -1 in ip and sp, so this can be one cause of the things you're seeing.
-Looks like all new threads start at the same place. How would they distinguish from one another in terms of functionality? Is that encoded in their stack (e.g. when they issue a 'ret', they'll return to different locations or maybe arguments to their function are on the stack?)
The thread lib is doing this for us. All threads are executing some setup code before they're branched to their actual function (see l4/pkg/thread/lib/src/create.c).
Adam
All,
Okay, so let me back up a minute from trying to understand all of these items at the same time. What I would like to understand first is what are the main threads for the L4Linux server. Based on your feedback and some further analysis, here is what I believe are the main threads:
Thread 0 - l4env initialization, then sleeps Thread 1 - ?? Thread 2 - runs "main" to get the L4Linux server going, then sleeps Thread 3 - runs "l4env_linux_startup" function, then sleeps until told that L4Linux is exiting Thread 4 - Timer interrupt thread, handles the periodic timer for linux the entry functions for thread 4 are: timer_irq_thread --- the main handler l4lx_irq_timer_startup --- gets timer_irq_thread started l4lx_irq_timer_shutdown l4lx_irq_timer_enable l4lx_irq_timer_disable l4lx_irq_timer_ack l4lx_irq_timer_end Additional threads >= 5, one for each IRQ (virtual not implemented yet) the entry functions for these threads are irq_dev_thread --- the main handler l4lx_irq_dev_startup_hw --- gets irq_dev_thread started l4lx_irq_dev_shutdown_hw l4lx_irq_dev_enable_hw l4lx_irq_dev_disable_hw l4lx_irq_dev_ack_hw l4lx_irq_dev_end_hw l4lx_irq_dev_startup_virt l4lx_irq_dev_shutdown_virt l4lx_irq_dev_enable_virt l4lx_irq_dev_disable_virt l4lx_irq_dev_ack_virt l4lx_irq_dev_end_virt
Anything wrong with that description? What does thread 1 do? I can't seem to track it down. Also, which thread is responsible for handling system calls? Is that what thread 1 does maybe? Which L4Linux function does the system call handler function start?
Thanks!
Julian
Adam Lackorzynski wrote:
On Mon Dec 12, 2005 at 15:50:17 -0500, Julian Grizzard wrote:
to track the program counter and registers of all threads in the L4Linux task. Here's exactly what I would like to be able to do from a monitoring L4 task:
I'll throw in some comments.
-Obtain the PC/regs for the first thread (thread 0) for L4Linux just before the thread executes it's first instruction. It would be nice to make the thread wait for a signal before beginning execution.
The regs are useless is this case as they haven't been filled in with anything. Only ip and sp are set. Making the thread wait is possible by changing the startup method. The monitor must be the exception handler then.
-Obtain the PC/regs for any additional thread created or any thread moved for L4Linux (i.e. from a l4_thread_ex_regs call) just before that thread executes. Again, it would be nice to make that thread wait for a signal before beginning its execution.
If the threads are started the same way as above this should be possible.
-Randomly preemptively obtain the PC/regs for all threads of L4Linux.
Getting the ip is possible, getting all the regs is more complicated due to our multiple extry paths into the kernel and thus different stack frames.
-Obtain the PC/regs for all threads of L4Linux on demand.
What's the difference from above? Works the same way, I'd guess.
-Receive notification of thread exit for all threads of L4Linux.
Threads do not exit, they just go to sleep (forever). So a thread exit can only be a user level thing.
There are a few items that I do not yet fully understand that *may* make matters more difficult:
-The setup code executing in libloader.s.so.
-Trampoline code
There are a few assumptions I am willing to have for initial testing to make the development easier:
-Assume Task number of L4Linux is known by the microkernel (i.e. hard code it in).
I'd say that's a really minor issue you have there ;)
-Assume Task number of monitoring thread is known by the microkernel.
As a first crack, I have added some code to the microkernel to print out information on the L4Linux task, to see if that information would be good to transfer to the monitoring task. Below is a summary of things I have tried and results seen:
MODIFICATION #1 l4/kernel/fiasco/src/kern/context.cpp Context::schedule() Goal: Track current PC of L4Linux threads ======================= ... /* in the for loop */ Sys_task_new_frame *regs = sys_frame_cast<Sys_task_new_frame>(next_to_run->regs()); L4_uid id = regs->dst(); if (id.task() == 0xC) // hard coded task of L4Linux { unsigned int ip = next_to_run->regs()->ip(); unsigned int sp = next_to_run->regs()->sp(); printf("task: %d.%d @ ip: %08x sp: %08x\n", id.task(), id.lthread(), ip, sp); } ... ======================= Results: task: 12.3 @ ip: 004ccb0b sp: b00fff80 task: 12.3 @ ip: 004cd303 sp: b01fff7c task: 12.4 @ ip: 004cca9d sp: b00fff80 task: 12.5 @ ip: 00402ae6 sp: 00657f50 ======================= Notes: 004ccb0b - right after an 'int 0x30' in timer_irq_thread 004cd303 - right after an 'int 0x30' in irq_thread_hw 004cca9d - right after an 'int 0x30' in timer_irq_thread 00402ae6 - right after an 'int 0x30' in l4x_idle
This makes perfectly sense.
MODIFICATION #2 l4/kernel/fiasco/src/kern/thread-syscall.cpp Thread::sys_task_new() Goal: Track starting PC of main L4Linux thread (i.e. thread 0) ======================= ... /* near tail of for loop, just before return */ if (taskno == 0xC) // hard coded L4Linux task ID { printf("new task: %d @ ip: %08x sp: %08x\n", taskno, regs->ip(), regs->sp()); } ======================= Results: new task: 12 @ ip: 00015ea0 sp: 00008f6c ======================= Notes: 00015ea0 - part of the libloader.s.so code
This can be expected.
MODIFICATION #3 l4/kernel/fiasco/src/kern/thread-syscall.cpp Thread::sys_thread_ex_regs() Goal: Track PC entry of new L4Linux threads ======================= ... if (!dst->exists()) { unsigned int task_num = dst_id.task(); unsigned int lthread_num = regs->thread(); new_thread = true; check (new (dst_id) Thread (dst_task, dst_id, sched()->prio(),
mcp()) == dst);if (taskno == 0xC) { printf("new thread: %d.%d @ ip: %08x sp: %08x\n", task_num, lthread_num, regs->ip(), regs->sp()); } } else { unsigned int task_num = dst_id.task(); unsigned int lthread_num = regs->thread(); if (taskno == 0xC) { printf("modified thread: %d.%d @ ip: %08x sp: %08x\n", task_num, lthread_num, regs->ip(), regs->sp()); }
} ... ======================= Results: modified thread: 12.0 @ ip: ffffffff sp: ffffffff modified thread: 12.0 @ ip: ffffffff sp: ffffffff new thread: 12.1 @ ip: 0001ac10 sp: afe00000 new thread: 12.2 @ ip: 0001ac10 sp: aff00000 modified thread: 12.2 @ ip: ffffffff sp: ffffffff new thread: 12.3 @ ip: 0001ac10 sp: 00657ff8 modified thread: 12.3 @ ip: ffffffff sp: ffffffff new thread: 12.4 @ ip: 0001ac10 sp: b0100000 modified thread: 12.3 @ ip: ffffffff sp: ffffffff new thread: 12.5 @ ip: 0001ac10 sp: b0200000 modified thread: 12.3 @ ip: ffffffff sp: ffffffff new thread: 12.6 @ ip: 0001ac10 sp: b0300000 new thread: 12.7 @ ip: 0001ac10 sp: b0400000 ======================= Notes: 0001ac10 - part of the libloader.s.so code
Yup.
ffffffff - not really useful. wrong data perhaps.
No, see the spec, when ip or sp are -1 then they're not changed, i.e. ex_regs is giving back values only.
QUESTIONS:
GENERAL QUESTIONS -Is there any easy to delay execution of newly created threads, both new task thread 0 and additional threads?
What one could do is put an exception handler task between the loader and l4linux and intercept interesting syscalls. This way you also get notifications for new threads. There are probably things which need to be enhanced/fixed/whatever in the kernel but theoretically this could work this way.
-Is there an easy way to randomly interrupt L4Linux threads to check their PC value?
If you monitor task runs L4Linux is not running (UP system) so you can just exregs them to query the ip.
-Do any L4Linux threads exit?
In the context of L4Linux this is possible, in the context of L4 it's not (see above).
MODIFICATION #1 -Seems like threads resume execution right after an interrupt.
int 0x30 is the ipc system call.
I don't see threads getting preempted in the middle of servicing a system call, for example. Any ideas why not?
Bad luck. I guess you'll need some threads with a higher prio doing some work to actually see this.
Further, is there one thread dedicated to handling system calls?
Yes, only one thread is handling Linux system calls.
-Is there any better way to track where all the L4Linux threads are? Maybe somewhere else besides the scheduler? Are any other classes/functions helpful to get this information?
What you are doing is reading the ip and sp from the return frame, you can basically do this from everywhere when you run on the behalf of some user thread. It's not limited to the schedule function.
MODIFICATION #2 -I believe this is working correctly. See anything wrong here?
Possible.
MODIFICATION #3 -I would like to catch all instances of creating new threads for L4Linux. Will this current approach work, or is it not correct?
Maybe you want to better hook into Thread::initialize so you get task_new and threads in one go.
If my implementation is correct: -Why are L4Linux threads getting modified? -Any idea why my ip and sp values are 0xffffffff for modified threads?
Again, see the spec. Querying pager/preempter is done with -1 in ip and sp, so this can be one cause of the things you're seeing.
-Looks like all new threads start at the same place. How would they distinguish from one another in terms of functionality? Is that encoded in their stack (e.g. when they issue a 'ret', they'll return to different locations or maybe arguments to their function are on the stack?)
The thread lib is doing this for us. All threads are executing some setup code before they're branched to their actual function (see l4/pkg/thread/lib/src/create.c).
Adam
On Fri Dec 16, 2005 at 01:42:25 -0500, Julian Grizzard wrote:
Okay, so let me back up a minute from trying to understand all of these items at the same time. What I would like to understand first is what are the main threads for the L4Linux server. Based on your feedback and some further analysis, here is what I believe are the main threads:
Thread 0 - l4env initialization, then sleeps
Regionmanager thread.
Thread 1 - ??
Semaphore thread.
Thread 2 - runs "main" to get the L4Linux server going, then sleeps
Ok.
Thread 3 - runs "l4env_linux_startup" function, then sleeps
No, thread 3 does all the work. l4env_linux_startup() calls start_kernel() at the bottom!
until told that L4Linux is exiting Thread 4 - Timer interrupt thread, handles the periodic timer for linux the entry functions for thread 4 are: timer_irq_thread --- the main handler l4lx_irq_timer_startup --- gets timer_irq_thread started l4lx_irq_timer_shutdown l4lx_irq_timer_enable l4lx_irq_timer_disable l4lx_irq_timer_ack l4lx_irq_timer_end
Yup, thread 4 is the timer.
Additional threads >= 5, one for each IRQ (virtual not implemented yet)
virtual not implemented yet?
the entry functions for these threads are irq_dev_thread --- the main handler l4lx_irq_dev_startup_hw --- gets irq_dev_thread started l4lx_irq_dev_shutdown_hw l4lx_irq_dev_enable_hw l4lx_irq_dev_disable_hw l4lx_irq_dev_ack_hw l4lx_irq_dev_end_hw l4lx_irq_dev_startup_virt l4lx_irq_dev_shutdown_virt l4lx_irq_dev_enable_virt l4lx_irq_dev_disable_virt l4lx_irq_dev_ack_virt l4lx_irq_dev_end_virt
Not only irq threads but also all other threads that may be allocated by stubs etc.
Anything wrong with that description? What does thread 1 do? I can't seem to track it down.
All l4env apps have thread0 (region manager thread) and thread1 (semaphore), real work starts in thread2.
Also, which thread is responsible for handling system calls?
thread3, it actually runs the Linux kernel.
Is that what thread 1 does maybe? Which L4Linux function does the system call handler function start?
Maybe you mean l4x_user_dispatcher()?
Adam
Adam Lackorzynski wrote: </snip> cool
Additional threads >= 5, one for each IRQ (virtual not implemented yet)
virtual not implemented yet?
apparently not, but for my setup this hasn't really been a problem:
/* * _virt functions are empty for now. */ unsigned int l4lx_irq_dev_startup_virt(unsigned int irq) { printk("%s(%d) unimplemented\n", __func__, irq); return 0; } void l4lx_irq_dev_shutdown_virt(unsigned int irq) { printk("%s(%d) unimplemented\n", __func__, irq); } void l4lx_irq_dev_ack_virt(unsigned int irq) { printk("%s(%d) unimplemented\n", __func__, irq); } void l4lx_irq_dev_end_virt(unsigned int irq) { printk("%s(%d) unimplemented\n", __func__, irq); } void l4lx_irq_dev_enable_virt(unsigned int irq) { printk("%s(%d) unimplemented\n", __func__, irq); } void l4lx_irq_dev_disable_virt(unsigned int irq) { printk("%s(%d) unimplemented\n", __func__, irq); }
the entry functions for these threads are irq_dev_thread --- the main handler l4lx_irq_dev_startup_hw --- gets irq_dev_thread started l4lx_irq_dev_shutdown_hw l4lx_irq_dev_enable_hw l4lx_irq_dev_disable_hw l4lx_irq_dev_ack_hw l4lx_irq_dev_end_hw l4lx_irq_dev_startup_virt l4lx_irq_dev_shutdown_virt l4lx_irq_dev_enable_virt l4lx_irq_dev_disable_virt l4lx_irq_dev_ack_virt l4lx_irq_dev_end_virt
Not only irq threads but also all other threads that may be allocated by stubs etc.
okay, so I think I found the easiest solution to my question, which is to grep through the L4Linux source for l4lx_thread_create. I assume that all L4 threads in L4Linux are created through that API except threads 0 and 1 which you explained are created by the L4Env startup.
Anything wrong with that description? What does thread 1 do? I can't seem to track it down.
All l4env apps have thread0 (region manager thread) and thread1 (semaphore), real work starts in thread2.
gotcha
Also, which thread is responsible for handling system calls?
thread3, it actually runs the Linux kernel.
Is that what thread 1 does maybe? Which L4Linux function does the system call handler function start?
Maybe you mean l4x_user_dispatcher()?
ah, yes. that looks like the important function that I was thinking should exist somewhere.
excellent, Thanks!
Julian
On Fri Dec 16, 2005 at 21:44:01 -0500, Julian Grizzard wrote:
the entry functions for these threads are irq_dev_thread --- the main handler l4lx_irq_dev_startup_hw --- gets irq_dev_thread started
Not only irq threads but also all other threads that may be allocated by stubs etc.
okay, so I think I found the easiest solution to my question, which is to grep through the L4Linux source for l4lx_thread_create. I assume that all L4 threads in L4Linux are created through that API except
All threads should be created through this API, yes.
threads 0 and 1 which you explained are created by the L4Env startup.
And thread 2, which is the thread where main is running.
Adam
l4-hackers@os.inf.tu-dresden.de
-
Adam Lackorzynski -
Julian Grizzard -
Martin Pohlack