I use GDB stubs exclusively, in Bochs as well as in QEMU (I did not try L4/Fiasco on other virtual machines so far, and only a few times on physical hardware). There are approaches for debugging in the L4 runtime environment, such as Valgrind and probably also GDB (although I am not sure about that). They are usually quite old and not all of them still work. If you like, you can take a look at some of the work presented at http://www.inf.tu-dresden.de/index.php?node_id=1701 and http://www.inf.tu-dresden.de/index.php?node_id=1429 .
Anyway, at least with simple applications, my first approach is usually a close examination of the code: go through all your setup, and if you are unsure at some point whether you did it right or not, try to type your problem into Google. If you cannot find anything, send an e-mail to the list. People here are usually very nice and patient and they will be happy to help.
Regards!
Martin
P.S.: I answered you using the mailing list, and I would like to ask you to use the "Reply to list" or "Reply to all" button of your e-mail client, so that everyone can follow the discussion and benefit from our findings.
Am 02.04.2013 04:49, schrieb Jian Liu:
2013/4/2 Martin Unzner <munzner@os.inf.tu-dresden.de mailto:munzner@os.inf.tu-dresden.de>
you mention bochsdbg, so maybe you want to use the Bochs GDB stub? http://bochs.sourceforge.net/doc/docbook/user/debugging-with-gdb.html tells you how to do so. Using GDB, you can debug all kinds of programs on source code level.Hi Martin,
I am also wonder besides bochs, if there are other ways todebug fiasco system on source code level. What is your usual method to do it?
Best, Jian
Hi Keqin,
Thank you very much for your timely reply, but I still want to know ifI must recompile the source code to create another kind of image which can be used for debugging(like vmlinux)? Does this image hello.iso(generated from official guide: http://os.inf.tu-dresden.de/fiasco/build.html) provide all the necessary debug information in it?
the generated hello.iso file is an ISO CD-ROM image (see http://en.wikipedia.org/wiki/ISO_image), which contains all necessary binaries to boot a certain Fiasco.OC setup (e.g., the bootstrapper, the kernel, and the hello world application).
For debugging using BochsGDB or QEMU's GDB server you will need a single binary. For the Fiasco.OC kernel, you will find two binaries in the kernel's build directory. The 'fiasco' binary is the one that gets loaded, but it does not contain debug info. The 'fiasco.image' binary is the same binary but with debug information attached. So if you want to debug the kernel, tell GDB to load the latter binary.
For L4Re user-level applications (such as hello) you will find the respective binaries in the L4Re build directory (bin/<arch>/<L4API>). These binaries are not stripped by default so you can directly use them with GDB.
Cheers, Bjoern
Hi,Bjoern,
Thank you! I found that image according to your reply. It helped me a lot! But I still have one question. First, I typed this command in one terminal: qemu-system-i386 -S -cdrom hello.iso -serial stdio Second, in the qemu GUI: gdbserver tcp::1234 Third, in another terminal I typed: gdb hello(this is the binary with debug info) target remote localhost:1234 b main and then, when I typed run, gdb said: The "remote" target does not support "run". Try "help target" or "continue". so, I typed c. The hello program ran but it didnot stop at the breakpoint which I expected it will stop at that point(main()). Did I miss something?
Regards! Jimmy/Keqin ----------------------------------------------------------------------------------- Hi Keqin,
Thank you very much for your timely reply, but I still want to know ifI must recompile the source code to create another kind of image which can be used for debugging(like vmlinux)? Does this image hello.iso(generated from official guide: http://os.inf.tu-dresden.de/fiasco/build.html) provide all the necessary debug information in it?
the generated hello.iso file is an ISO CD-ROM image (see http://en.wikipedia.org/wiki/ISO_image), which contains all necessary binaries to boot a certain Fiasco.OC setup (e.g., the bootstrapper, the kernel, and the hello world application).
For debugging using BochsGDB or QEMU's GDB server you will need a single binary. For the Fiasco.OC kernel, you will find two binaries in the kernel's build directory. The 'fiasco' binary is the one that gets loaded, but it does not contain debug info. The 'fiasco.image' binary is the same binary but with debug information attached. So if you want to debug the kernel, tell GDB to load the latter binary.
For L4Re user-level applications (such as hello) you will find the respective binaries in the L4Re build directory (bin/<arch>/<L4API>). These binaries are not stripped by default so you can directly use them with GDB.
Cheers, Bjoern
_______________________________________________ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
-----------------------------------------------------------------------------------
Hi Keqin Shun,
Thank you! I found that image according to your reply. It helped me a lot! ButI still have one question. First, I typed this command in one terminal: qemu-system-i386 -S -cdrom hello.iso -serial stdio Second, in the qemu GUI: gdbserver tcp::1234
You can simply start qemu with the "-S -s" switches to get the same behavior.
Third, in another terminal I typed: gdb hello(this is the binary with debug info) target remote localhost:1234 b main and then, when I typed run, gdb said: The "remote" target does not support "run". Try "help target" or "continue". so, I typed c.
At the point you are connecting to the remote GDB server, it is already running, so 'continue' is the proper command as you found out.
The hello program ran but it didnot stop at the breakpoint which I expected it will stop at that point(main()).
You are hitting a general problem with debugging using GDB and a virtual machine here: When you type 'break main' in GDB, it sends a message to the remote GDB server to set a (software) breakpoint at the virtual address of the symbol main. Unfortunately, GDB does not support debugging across multiple address spaces (e.g., Fiasco, the bootstrapper, hello, any other loaded applications) and in order to set a breakpoint in your application, you will have to make sure that while setting the breakpoint, this exact application's address space is currently active, because the GDB server in Qemu/Bochs will lookup the physical address belonging to the virtual address using the currently loaded page table.
So, how do we do that? For debugging purpose (don't use in release code!) there is a header file l4/sys/kdebug.h. Including it gives you access to a macro called enter_kdebug(). What you can do to debug your app is to include the header file and place an enter_kdebug() as the first line inside your main() function. This will run your application until the point main() is started and enter the Fiasco JDB. At this point, the application will halt and you can set the breakpoint remotely from Qemu. Note, that this of course only works for functions that are executed _after_ your main() function is entered.
Furthermore, note that this is only needed for user applications. If you want to debug Fiasco, this will simply work, because the kernel's virtual addresses are the same in all applications.
Hth, Bjoern
Hi Bjoern, Thank you for your patience to explain, and I did what you told. Now the problem is that when the application halted at the enter_kdebug(), the GDB didnot show the prompt(it showed "continuing." all the time after I typed "c".) so that I can't type any other commands like making another breakpoint or letting the application continue.
Regards! keqin
04.04. 2013 9:34 pm, Bjoern wrote: ----------------------------------------------------------------------------------- Hi Keqin Shun,
Thank you! I found that image according to your reply. It helped me a lot! ButI still have one question. First, I typed this command in one terminal: qemu-system-i386 -S -cdrom hello.iso -serial stdio Second, in the qemu GUI: gdbserver tcp::1234
You can simply start qemu with the "-S -s" switches to get the same behavior.
Third, in another terminal I typed: gdb hello(this is the binary with debug info) target remote localhost:1234 b main and then, when I typed run, gdb said: The "remote" target does not support "run". Try "help target" or
"continue".
so, I typed c.
At the point you are connecting to the remote GDB server, it is already running, so 'continue' is the proper command as you found out.
The hello program ran but it didnot stop at the breakpoint which I expected it will stop at that point(main()).
You are hitting a general problem with debugging using GDB and a virtual machine here: When you type 'break main' in GDB, it sends a message to the remote GDB server to set a (software) breakpoint at the virtual address of the symbol main. Unfortunately, GDB does not support debugging across multiple address spaces (e.g., Fiasco, the bootstrapper, hello, any other loaded applications) and in order to set a breakpoint in your application, you will have to make sure that while setting the breakpoint, this exact application's address space is currently active, because the GDB server in Qemu/Bochs will lookup the physical address belonging to the virtual address using the currently loaded page table.
So, how do we do that? For debugging purpose (don't use in release code!) there is a header file l4/sys/kdebug.h. Including it gives you access to a macro called enter_kdebug(). What you can do to debug your app is to include the header file and place an enter_kdebug() as the first line inside your main() function. This will run your application until the point main() is started and enter the Fiasco JDB. At this point, the application will halt and you can set the breakpoint remotely from Qemu. Note, that this of course only works for functions that are executed _after_ your main() function is entered.
Furthermore, note that this is only needed for user applications. If you want to debug Fiasco, this will simply work, because the kernel's virtual addresses are the same in all applications.
Hth, Bjoern
_______________________________________________ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
-----------------------------------------------------------------------------------
Bjoern, Maybe I didnot say it clearly. Can you give me an example about "you can set the breakpoint remotely from Qemu"? The application will halt when the instrument "enter_kdebug()" hit, so that we can use jdb to debug. It is helpful but what I really want is a source code level debugging tool(e.g., I can set a breakpoint using a function name which in JDB I need input the memory address). It seems that gdb donot support the remotely source code level debug on fiasco, bootstrap or others applications accroding to your explanation. If so, what should I do if I can't use gdb. Is there any other tools to instead? And how can I debug the process from the very beginning of the system which it goes before JDB runs? Thanks a lot!
Regards! keqin
04.07.2013 11:22 am, Keqin Sun wrote: ----------------------------------------------------------------------------------- Hi Bjoern, Thank you for your patience to explain, and I did what you told. Now the problem is that when the application halted at the enter_kdebug(), the GDB didnot show the prompt(it showed "continuing." all the time after I typed "c".) so that I can't type any other commands like making another breakpoint or letting the application continue.
Regards! keqin
04.04. 2013 9:34 pm, Bjoern wrote: ----------------------------------------------------------------------------------- Hi Keqin Shun,
Thank you! I found that image according to your reply. It helped me a lot! ButI still have one question. First, I typed this command in one terminal: qemu-system-i386 -S -cdrom hello.iso -serial stdio Second, in the qemu GUI: gdbserver tcp::1234
You can simply start qemu with the "-S -s" switches to get the same behavior.
Third, in another terminal I typed: gdb hello(this is the binary with debug info) target remote localhost:1234 b main and then, when I typed run, gdb said: The "remote" target does not support "run". Try "help target" or
"continue".
so, I typed c.
At the point you are connecting to the remote GDB server, it is already running, so 'continue' is the proper command as you found out.
The hello program ran but it didnot stop at the breakpoint which I expected it will stop at that point(main()).
You are hitting a general problem with debugging using GDB and a virtual machine here: When you type 'break main' in GDB, it sends a message to the remote GDB server to set a (software) breakpoint at the virtual address of the symbol main. Unfortunately, GDB does not support debugging across multiple address spaces (e.g., Fiasco, the bootstrapper, hello, any other loaded applications) and in order to set a breakpoint in your application, you will have to make sure that while setting the breakpoint, this exact application's address space is currently active, because the GDB server in Qemu/Bochs will lookup the physical address belonging to the virtual address using the currently loaded page table.
So, how do we do that? For debugging purpose (don't use in release code!) there is a header file l4/sys/kdebug.h. Including it gives you access to a macro called enter_kdebug(). What you can do to debug your app is to include the header file and place an enter_kdebug() as the first line inside your main() function. This will run your application until the point main() is started and enter the Fiasco JDB. At this point, the application will halt and you can set the breakpoint remotely from Qemu. Note, that this of course only works for functions that are executed _after_ your main() function is entered.
Furthermore, note that this is only needed for user applications. If you want to debug Fiasco, this will simply work, because the kernel's virtual addresses are the same in all applications.
Hth, Bjoern
_______________________________________________ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
-----------------------------------------------------------------------------------
_______________________________________________ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
-----------------------------------------------------------------------------------
Hey again,
Can you give me an example about "you can set the breakpoint remotely from Qemu"?
the approach you used (target remote, break <location>) was the correct one.
The application will halt when the instrument "enter_kdebug()" hit, so that wecan use jdb to debug. It is helpful but what I really want is a source code level debugging tool(e.g., I can set a breakpoint using a function name which in JDB I need input the memory address). It seems that gdb donot support the remotely source code level debug on fiasco, bootstrap or others applications accroding to your explanation. If so, what should I do if I can't use gdb.
GDB works fine. Unless you want to debug multiple applications concurrently at which point GDB's lack of multiple address space support hits you.
Is there any other tools to instead? And how can I debug the process from the very beginning of the system which it goes before JDB runs?
That's harder at the moment. You could manually place the enter_kdebug() call in an earlier function (e.g., in uClibC's main function). But then you will get this behavior from all applications that were compiled with this libC and managing this can be a mess.
Bjoern
On 07.04.2013 05:22, Keqin Sun wrote:
Hi Bjoern, Thank you for your patience to explain, and I did what you told. Now the problem is that when the application halted at the enter_kdebug(), the GDB didnot show the prompt(it showed "continuing." all the time after I typed "c".) so that I can't type any other commands like making another breakpoint or letting the application continue.
Well, your L4Re application is still stuck in JDB and in addition to 'c'ontinuing from GDB, you will also need to tell JDB to resume execution using the 'g' command.
Björn
l4-hackers@os.inf.tu-dresden.de
-
Björn Döbel -
Keqin Sun -
Martin Unzner