Track Program_Counter(PC)/regs of L4Linux Task

[Julian Grizzard]

All,

I am currently running L4Linux2.6.14 on the Fiasco l4v2 microkernel
using i386 arch with mostly default configurations.  I need the ability
to track the program counter and registers of all threads in the L4Linux
task.  Here's exactly what I would like to be able to do from a
monitoring L4 task:

-Obtain the PC/regs for the first thread (thread 0) for L4Linux just
before the thread executes it's first instruction.  It would be nice to
make the thread wait for a signal before beginning execution.

-Obtain the PC/regs for any additional thread created or any thread
moved for L4Linux (i.e. from a l4_thread_ex_regs call) just before that
thread executes.  Again, it would be nice to make that thread wait for a
signal before beginning its execution.

-Randomly preemptively obtain the PC/regs for all threads of L4Linux.

-Obtain the PC/regs for all threads of L4Linux on demand.

-Receive notification of thread exit for all threads of L4Linux.

There are a few items that I do not yet fully understand that *may* make
matters more difficult:

-The setup code executing in libloader.s.so.

-Trampoline code

There are a few assumptions I am willing to have for initial testing to
make the development easier:

-Assume Task number of L4Linux is known by the microkernel (i.e. hard
code it in).

-Assume Task number of monitoring thread is known by the microkernel.

As a first crack, I have added some code to the microkernel to print out
information on the L4Linux task, to see if that information would be
good to transfer to the monitoring task.  Below is a summary of things I
have tried and results seen:

MODIFICATION #1
l4/kernel/fiasco/src/kern/context.cpp
Context::schedule()
Goal: Track current PC of L4Linux threads
=======================
...
/* in the for loop */
Sys_task_new_frame *regs =
sys_frame_cast<Sys_task_new_frame>(next_to_run->regs());
L4_uid id = regs->dst();
if (id.task() == 0xC) // hard coded task of L4Linux
  {
    unsigned int ip = next_to_run->regs()->ip();
    unsigned int sp = next_to_run->regs()->sp();
    printf("task: %d.%d @ ip: %08x sp: %08x\n", id.task(), id.lthread(),
            ip, sp);
  }
...
=======================
Results:
task: 12.3 @ ip: 004ccb0b sp: b00fff80
task: 12.3 @ ip: 004cd303 sp: b01fff7c
task: 12.4 @ ip: 004cca9d sp: b00fff80
task: 12.5 @ ip: 00402ae6 sp: 00657f50
=======================
Notes:
004ccb0b - right after an 'int 0x30' in timer_irq_thread
004cd303 - right after an 'int 0x30' in irq_thread_hw
004cca9d - right after an 'int 0x30' in timer_irq_thread
00402ae6 - right after an 'int 0x30' in l4x_idle



MODIFICATION #2
l4/kernel/fiasco/src/kern/thread-syscall.cpp
Thread::sys_task_new()
Goal: Track starting PC of main L4Linux thread (i.e. thread 0)
=======================
...
/* near tail of for loop, just before return */
if (taskno == 0xC) // hard coded L4Linux task ID
  {
    printf("new task: %d @ ip: %08x sp: %08x\n",
            taskno, regs->ip(), regs->sp());
  }
=======================
Results:
new task: 12 @ ip: 00015ea0 sp: 00008f6c
=======================
Notes:
00015ea0 - part of the libloader.s.so code



MODIFICATION #3
l4/kernel/fiasco/src/kern/thread-syscall.cpp
Thread::sys_thread_ex_regs()
Goal: Track PC entry of new L4Linux threads
=======================
...
if (!dst->exists())
  {
    unsigned int task_num = dst_id.task();
    unsigned int lthread_num = regs->thread();
    new_thread = true;
    check (new (dst_id) Thread (dst_task, dst_id, sched()->prio(),

           mcp()) == dst);
    if (taskno == 0xC)
      {
        printf("new thread: %d.%d @ ip: %08x sp: %08x\n",
               task_num, lthread_num, regs->ip(), regs->sp());
      }
  }
else
  {
    unsigned int task_num = dst_id.task();
    unsigned int lthread_num = regs->thread();
    if (taskno == 0xC)
      {
        printf("modified thread: %d.%d @ ip: %08x sp: %08x\n",
               task_num, lthread_num, regs->ip(), regs->sp());
      }

  }
...
=======================
Results:
modified thread: 12.0 @ ip: ffffffff sp: ffffffff
modified thread: 12.0 @ ip: ffffffff sp: ffffffff
new thread: 12.1 @ ip: 0001ac10 sp: afe00000
new thread: 12.2 @ ip: 0001ac10 sp: aff00000
modified thread: 12.2 @ ip: ffffffff sp: ffffffff
new thread: 12.3 @ ip: 0001ac10 sp: 00657ff8
modified thread: 12.3 @ ip: ffffffff sp: ffffffff
new thread: 12.4 @ ip: 0001ac10 sp: b0100000
modified thread: 12.3 @ ip: ffffffff sp: ffffffff
new thread: 12.5 @ ip: 0001ac10 sp: b0200000
modified thread: 12.3 @ ip: ffffffff sp: ffffffff
new thread: 12.6 @ ip: 0001ac10 sp: b0300000
new thread: 12.7 @ ip: 0001ac10 sp: b0400000
=======================
Notes:
0001ac10 - part of the libloader.s.so code
ffffffff - not really useful.  wrong data perhaps.




QUESTIONS:

GENERAL QUESTIONS
-Is there any easy to delay execution of newly created threads, both new
task thread 0 and additional threads?
-Is there an easy way to randomly interrupt L4Linux threads to check
their PC value?
-Do any L4Linux threads exit?

MODIFICATION #1
-Seems like threads resume execution right after an interrupt.  I don't
see threads getting preempted in the middle of servicing a system call,
for example.  Any ideas why not?  Further, is there one thread dedicated
to handling system calls?
-Is there any better way to track where all the L4Linux threads are?
Maybe somewhere else besides the scheduler?  Are any other
classes/functions helpful to get this information?

MODIFICATION #2
-I believe this is working correctly.  See anything wrong here?

MODIFICATION #3
-I would like to catch all instances of creating new threads for
L4Linux.  Will this current approach work, or is it not correct?
If my implementation is correct:
-Why are L4Linux threads getting modified?
-Any idea why my ip and sp values are 0xffffffff for modified threads?
-Looks like all new threads start at the same place.  How would they
distinguish from one another in terms of functionality?  Is that encoded
in their stack (e.g. when they issue a 'ret', they'll return to
different locations or maybe arguments to their function are on the stack?)

Any other comments you have would also be appreciated.  Thanks!

Julian