Track Program_Counter(PC)/regs of L4Linux Task

[Adam Lackorzynski]

On Mon Dec 12, 2005 at 15:50:17 -0500, Julian Grizzard wrote:
> to track the program counter and registers of all threads in the L4Linux
> task.  Here's exactly what I would like to be able to do from a
> monitoring L4 task:

I'll throw in some comments.
 
> -Obtain the PC/regs for the first thread (thread 0) for L4Linux just
> before the thread executes it's first instruction.  It would be nice to
> make the thread wait for a signal before beginning execution.

The regs are useless is this case as they haven't been filled in with
anything. Only ip and sp are set. Making the thread wait is possible by
changing the startup method. The monitor must be the exception handler
then.

> -Obtain the PC/regs for any additional thread created or any thread
> moved for L4Linux (i.e. from a l4_thread_ex_regs call) just before that
> thread executes.  Again, it would be nice to make that thread wait for a
> signal before beginning its execution.

If the threads are started the same way as above this should be
possible.

> -Randomly preemptively obtain the PC/regs for all threads of L4Linux.

Getting the ip is possible, getting all the regs is more complicated due
to our multiple extry paths into the kernel and thus different stack
frames.

> -Obtain the PC/regs for all threads of L4Linux on demand.

What's the difference from above? Works the same way, I'd guess.

> -Receive notification of thread exit for all threads of L4Linux.

Threads do not exit, they just go to sleep (forever). So a thread exit
can only be a user level thing.

> There are a few items that I do not yet fully understand that *may* make
> matters more difficult:
> 
> -The setup code executing in libloader.s.so.
> 
> -Trampoline code
> 
> There are a few assumptions I am willing to have for initial testing to
> make the development easier:
> 
> -Assume Task number of L4Linux is known by the microkernel (i.e. hard
> code it in).

I'd say that's a really minor issue you have there ;)

> -Assume Task number of monitoring thread is known by the microkernel.
> 
> As a first crack, I have added some code to the microkernel to print out
> information on the L4Linux task, to see if that information would be
> good to transfer to the monitoring task.  Below is a summary of things I
> have tried and results seen:
> 
> MODIFICATION #1
> l4/kernel/fiasco/src/kern/context.cpp
> Context::schedule()
> Goal: Track current PC of L4Linux threads
> =======================
> ...
> /* in the for loop */
> Sys_task_new_frame *regs =
> sys_frame_cast<Sys_task_new_frame>(next_to_run->regs());
> L4_uid id = regs->dst();
> if (id.task() == 0xC) // hard coded task of L4Linux
>   {
>     unsigned int ip = next_to_run->regs()->ip();
>     unsigned int sp = next_to_run->regs()->sp();
>     printf("task: %d.%d @ ip: %08x sp: %08x\n", id.task(), id.lthread(),
>             ip, sp);
>   }
> ...
> =======================
> Results:
> task: 12.3 @ ip: 004ccb0b sp: b00fff80
> task: 12.3 @ ip: 004cd303 sp: b01fff7c
> task: 12.4 @ ip: 004cca9d sp: b00fff80
> task: 12.5 @ ip: 00402ae6 sp: 00657f50
> =======================
> Notes:
> 004ccb0b - right after an 'int 0x30' in timer_irq_thread
> 004cd303 - right after an 'int 0x30' in irq_thread_hw
> 004cca9d - right after an 'int 0x30' in timer_irq_thread
> 00402ae6 - right after an 'int 0x30' in l4x_idle

This makes perfectly sense.
 
> MODIFICATION #2
> l4/kernel/fiasco/src/kern/thread-syscall.cpp
> Thread::sys_task_new()
> Goal: Track starting PC of main L4Linux thread (i.e. thread 0)
> =======================
> ...
> /* near tail of for loop, just before return */
> if (taskno == 0xC) // hard coded L4Linux task ID
>   {
>     printf("new task: %d @ ip: %08x sp: %08x\n",
>             taskno, regs->ip(), regs->sp());
>   }
> =======================
> Results:
> new task: 12 @ ip: 00015ea0 sp: 00008f6c
> =======================
> Notes:
> 00015ea0 - part of the libloader.s.so code

This can be expected.

> MODIFICATION #3
> l4/kernel/fiasco/src/kern/thread-syscall.cpp
> Thread::sys_thread_ex_regs()
> Goal: Track PC entry of new L4Linux threads
> =======================
> ...
> if (!dst->exists())
>   {
>     unsigned int task_num = dst_id.task();
>     unsigned int lthread_num = regs->thread();
>     new_thread = true;
>     check (new (dst_id) Thread (dst_task, dst_id, sched()->prio(),
> 
>            mcp()) == dst);
>     if (taskno == 0xC)
>       {
>         printf("new thread: %d.%d @ ip: %08x sp: %08x\n",
>                task_num, lthread_num, regs->ip(), regs->sp());
>       }
>   }
> else
>   {
>     unsigned int task_num = dst_id.task();
>     unsigned int lthread_num = regs->thread();
>     if (taskno == 0xC)
>       {
>         printf("modified thread: %d.%d @ ip: %08x sp: %08x\n",
>                task_num, lthread_num, regs->ip(), regs->sp());
>       }
> 
>   }
> ...
> =======================
> Results:
> modified thread: 12.0 @ ip: ffffffff sp: ffffffff
> modified thread: 12.0 @ ip: ffffffff sp: ffffffff
> new thread: 12.1 @ ip: 0001ac10 sp: afe00000
> new thread: 12.2 @ ip: 0001ac10 sp: aff00000
> modified thread: 12.2 @ ip: ffffffff sp: ffffffff
> new thread: 12.3 @ ip: 0001ac10 sp: 00657ff8
> modified thread: 12.3 @ ip: ffffffff sp: ffffffff
> new thread: 12.4 @ ip: 0001ac10 sp: b0100000
> modified thread: 12.3 @ ip: ffffffff sp: ffffffff
> new thread: 12.5 @ ip: 0001ac10 sp: b0200000
> modified thread: 12.3 @ ip: ffffffff sp: ffffffff
> new thread: 12.6 @ ip: 0001ac10 sp: b0300000
> new thread: 12.7 @ ip: 0001ac10 sp: b0400000
> =======================
> Notes:
> 0001ac10 - part of the libloader.s.so code

Yup.

> ffffffff - not really useful.  wrong data perhaps.

No, see the spec, when ip or sp are -1 then they're not changed, i.e.
ex_regs is giving back values only.

> QUESTIONS:
> 
> GENERAL QUESTIONS
> -Is there any easy to delay execution of newly created threads, both new
> task thread 0 and additional threads?

What one could do is put an exception handler task between the loader
and l4linux and intercept interesting syscalls. This way you also get
notifications for new threads. There are probably things which need to
be enhanced/fixed/whatever in the kernel but theoretically this could
work this way.

> -Is there an easy way to randomly interrupt L4Linux threads to check
> their PC value?

If you monitor task runs L4Linux is not running (UP system) so you can
just exregs them to query the ip.

> -Do any L4Linux threads exit?

In the context of L4Linux this is possible, in the context of L4 it's
not (see above).

> MODIFICATION #1
> -Seems like threads resume execution right after an interrupt.

int 0x30 is the ipc system call.

> I don't
> see threads getting preempted in the middle of servicing a system call,
> for example.  Any ideas why not?

Bad luck. I guess you'll need some threads with a higher prio doing
some work to actually see this.

> Further, is there one thread dedicated
> to handling system calls?

Yes, only one thread is handling Linux system calls.

> -Is there any better way to track where all the L4Linux threads are?
> Maybe somewhere else besides the scheduler?  Are any other
> classes/functions helpful to get this information?

What you are doing is reading the ip and sp from the return frame, you
can basically do this from everywhere when you run on the behalf of some
user thread. It's not limited to the schedule function.

> MODIFICATION #2
> -I believe this is working correctly.  See anything wrong here?

Possible.

> MODIFICATION #3
> -I would like to catch all instances of creating new threads for
> L4Linux.  Will this current approach work, or is it not correct?

Maybe you want to better hook into Thread::initialize so you get
task_new and threads in one go.

> If my implementation is correct:
> -Why are L4Linux threads getting modified?
> -Any idea why my ip and sp values are 0xffffffff for modified threads?

Again, see the spec. Querying pager/preempter is done with -1 in ip and
sp, so this can be one cause of the things you're seeing.

> -Looks like all new threads start at the same place.  How would they
> distinguish from one another in terms of functionality?  Is that encoded
> in their stack (e.g. when they issue a 'ret', they'll return to
> different locations or maybe arguments to their function are on the stack?)

The thread lib is doing this for us. All threads are executing some
setup code before they're branched to their actual function (see
l4/pkg/thread/lib/src/create.c).



Adam
-- 
Adam                 adam at os.inf.tu-dresden.de
  Lackorzynski         http://os.inf.tu-dresden.de/~adam/