I'm trying to build L4 on NixOS in a VM (long term goal: try to learn how to replace NixOS's kernel with L4Linux, then maybe Genode). I'm getting an error like below:
undefined reference to `__stack_chk_fail'
during linking of bootstrap.elf - in a GCC's error-barf just after:
[bootstrap] ==> Linking bootstrap.elf
I've put up the precise Nix expression/script I'm using to build it on github - at:
https://github.com/akavel/l4.nix/blob/0108dfcf8d84780e1ba884c41ece0d2dbf4ddd... but what it generally boils down to is a sequence of commands like this:
$ wget http://os.inf.tu-dresden.de/download/snapshots/l4re-core-2016082114.tar.xz $ tar xf l4re-core-2016082114.tar.xz $ cd l4re-core-2016082114 $ mkdir out $ cp src/l4/mk/defconfig/config.amd64 out/l4/.kconfig $ make -C src/l4 O=out/l4 olddefconfig $ make -C out/l4
I've tried various additional changes, based on some snippets found here and there on teh Internets, like: - adding V=0 to the last make (no idea what it does yet, just some cargo culting) - adding NIX_CFLAGS_COMPILE = "-fno-stack-protector"; to the Nix expression/script (also not really sure where it applies in the stack); but none of the above helped.
Any idea what I could try doing to push it further?
Thanks, /Mateusz.
Hm; tried adding the following line:
hardeningDisable = [ "stackprotector" "pic" ];
per https://github.com/NixOS/nixpkgs/issues/18895 and https://github.com/NixOS/nixpkgs/issues/18995. This now seems to get me through the bootstrap.elf linking phase! But it fails soon afterwards on bootstrap32.elf :-( with:
ld: skipping incompatible /nix/store/...-gcc-5.4.0/lib/gcc/x86_64-unknown-linux-gnu/5.4.0/libgcc.a when searching for -lgcc ld: cannot find -lgcc
Can I skip building bootstrap32.elf somehow? Or should I not? Do I even have to build l4, or do I only need fiasco for l4linux? (fiasco seems to build OK with the initial hardeningDisable = [ "all" ]; I tried.) Or none at all, just the sources?
Thanks, /Mateusz.
On Fri, Mar 24, 2017 at 10:06 PM, Mateusz Czaplinski czapkofan@gmail.com wrote:
I'm trying to build L4 on NixOS in a VM (long term goal: try to learn how to replace NixOS's kernel with L4Linux, then maybe Genode). I'm getting an error like below:
undefined reference to `__stack_chk_fail'
during linking of bootstrap.elf - in a GCC's error-barf just after:
[bootstrap] ==> Linking bootstrap.elf
I've put up the precise Nix expression/script I'm using to build it on github - at: https://github.com/akavel/l4.nix/blob/0108dfcf8d84780e1ba884c41ece0d 2dbf4ddd84/l4re.nix but what it generally boils down to is a sequence of commands like this:
$ wget http://os.inf.tu-dresden.de/download/snapshots/l4re-core- 2016082114.tar.xz $ tar xf l4re-core-2016082114 <(201)%20608-2114>.tar.xz $ cd l4re-core-2016082114 <(201)%20608-2114> $ mkdir out $ cp src/l4/mk/defconfig/config.amd64 out/l4/.kconfig $ make -C src/l4 O=out/l4 olddefconfig $ make -C out/l4
I've tried various additional changes, based on some snippets found here and there on teh Internets, like:
- adding V=0 to the last make (no idea what it does yet, just some cargo
culting)
- adding NIX_CFLAGS_COMPILE = "-fno-stack-protector"; to the Nix
expression/script (also not really sure where it applies in the stack); but none of the above helped.
Any idea what I could try doing to push it further?
Thanks, /Mateusz.
Seems I've got through by also adding gcc_multi to buildInputs. Yay for self-help! ;) and for the Internets. Let's boldly see what breaks next! ;)
Best Regards, /Mateusz.
On Sat, Mar 25, 2017 at 12:12 AM, Mateusz Czaplinski czapkofan@gmail.com wrote:
Hm; tried adding the following line:
hardeningDisable = [ "stackprotector" "pic" ];
per https://github.com/NixOS/nixpkgs/issues/18895 and https://github.com/NixOS/nixpkgs/issues/18995. This now seems to get me through the bootstrap.elf linking phase! But it fails soon afterwards on bootstrap32.elf :-( with:
ld: skipping incompatible /nix/store/...-gcc-5.4.0/lib/ gcc/x86_64-unknown-linux-gnu/5.4.0/libgcc.a when searching for -lgcc ld: cannot find -lgcc
Can I skip building bootstrap32.elf somehow? Or should I not? Do I even have to build l4, or do I only need fiasco for l4linux? (fiasco seems to build OK with the initial hardeningDisable = [ "all" ]; I tried.) Or none at all, just the sources?
Thanks, /Mateusz.
On Fri, Mar 24, 2017 at 10:06 PM, Mateusz Czaplinski czapkofan@gmail.com wrote:
I'm trying to build L4 on NixOS in a VM (long term goal: try to learn how to replace NixOS's kernel with L4Linux, then maybe Genode). I'm getting an error like below:
undefined reference to `__stack_chk_fail'
during linking of bootstrap.elf - in a GCC's error-barf just after:
[bootstrap] ==> Linking bootstrap.elf
I've put up the precise Nix expression/script I'm using to build it on github - at: https://github.com/akavel/l4.nix/blob/0108dfcf8d84780e1ba884 c41ece0d2dbf4ddd84/l4re.nix but what it generally boils down to is a sequence of commands like this:
$ wget http://os.inf.tu-dresden.de/download/snapshots/l4re-core-201 6082114.tar.xz $ tar xf l4re-core-2016082114 <(201)%20608-2114>.tar.xz $ cd l4re-core-2016082114 <(201)%20608-2114> $ mkdir out $ cp src/l4/mk/defconfig/config.amd64 out/l4/.kconfig $ make -C src/l4 O=out/l4 olddefconfig $ make -C out/l4
I've tried various additional changes, based on some snippets found here and there on teh Internets, like:
- adding V=0 to the last make (no idea what it does yet, just some
cargo culting)
- adding NIX_CFLAGS_COMPILE = "-fno-stack-protector"; to the Nix
expression/script (also not really sure where it applies in the stack); but none of the above helped.
Any idea what I could try doing to push it further?
Thanks, /Mateusz.
Hi,
good that you figured all out yourself. Just comments.
On Sat Mar 25, 2017 at 00:12:31 +0100, Mateusz Czaplinski wrote:
Hm; tried adding the following line:
hardeningDisable = [ "stackprotector" "pic" ];
per https://github.com/NixOS/nixpkgs/issues/18895 and https://github.com/NixOS/nixpkgs/issues/18995. This now seems to get me through the bootstrap.elf linking phase! But it fails soon afterwards on bootstrap32.elf :-( with:
ld: skipping incompatible /nix/store/...-gcc-5.4.0/lib/gcc/x86_64-unknown-linux-gnu/5.4.0/libgcc.a when searching for -lgcc ld: cannot find -lgcc
Can I skip building bootstrap32.elf somehow? Or should I not?
For booting, you'll need a bit of 32bit code, thus the multilib parts are indeed required.
Do I even have to build l4, or do I only need fiasco for l4linux? (fiasco seems to build OK with the initial hardeningDisable = [ "all" ]; I tried.) Or none at all, just the sources?
You need to build l4 for L4Linux as L4Linux also requires L4Re.
Adam
On Fri, Mar 24, 2017 at 10:06 PM, Mateusz Czaplinski czapkofan@gmail.com wrote:
I'm trying to build L4 on NixOS in a VM (long term goal: try to learn how to replace NixOS's kernel with L4Linux, then maybe Genode). I'm getting an error like below:
undefined reference to `__stack_chk_fail'
during linking of bootstrap.elf - in a GCC's error-barf just after:
[bootstrap] ==> Linking bootstrap.elf
I've put up the precise Nix expression/script I'm using to build it on github - at: https://github.com/akavel/l4.nix/blob/0108dfcf8d84780e1ba884c41ece0d 2dbf4ddd84/l4re.nix but what it generally boils down to is a sequence of commands like this:
$ wget http://os.inf.tu-dresden.de/download/snapshots/l4re-core- 2016082114.tar.xz $ tar xf l4re-core-2016082114 <(201)%20608-2114>.tar.xz $ cd l4re-core-2016082114 <(201)%20608-2114> $ mkdir out $ cp src/l4/mk/defconfig/config.amd64 out/l4/.kconfig $ make -C src/l4 O=out/l4 olddefconfig $ make -C out/l4
I've tried various additional changes, based on some snippets found here and there on teh Internets, like:
- adding V=0 to the last make (no idea what it does yet, just some cargo
culting)
- adding NIX_CFLAGS_COMPILE = "-fno-stack-protector"; to the Nix
expression/script (also not really sure where it applies in the stack); but none of the above helped.
Any idea what I could try doing to push it further?
Thanks, /Mateusz.
On Sun, Mar 26, 2017 at 11:51 PM, Adam Lackorzynski < adam@os.inf.tu-dresden.de> wrote:
For booting, you'll need a bit of 32bit code, thus the multilib parts are indeed required.
Thanks for info, nice to learn that!
You need to build l4 for L4Linux as L4Linux also requires L4Re.
Hm; so I'm still not exactly clear about how this is structured in the repo and how it builds together. Specifically: I seem to have built the project contained in the l4linux subdir of the l4re-snapshot-2016082114 archive; does it reach from l4linux/ to ../l4re/ and ../kernel/fiasco/ in process, through the makefiles from inside l4linux/ ? It didn't complain about any missing binaries, even though I didn't give it access to any precompiled l4re or fiasco libs before "make".
I seem to have replaced my old kernel with the result, and it went, ummm... suspiciously smooth... am I really running on Fiasco.OC+L4Linux? Or did I just accidentally recompile a regular kernel, and just renamed a few strings in it so that `uname --all` can now wink and say "huh huh, yeah sure, 4.7-l4, nod nod", but in reality I just tricked myself in an overly complicated way?
Is there some means by which I could test/verify that yes indeed, I am now, or am not, on L4 full and proper in my VM? Some L4 API, or some other "one weird trick" I can try, that can only work with L4 (vel Fiasco)? From inside or outside (I'm doing it in Hyper-V)? I've put down the some question in slightly different words on https://superuser.com/q/1193658/12184 if someone would like/prefer that format. If you're curious, the Nix expression I've used can be seen at: https://bitbucket.org/akavel/nixos-hyperv
I'd be very grateful for some more advice!
Thanks & Best Regards, /Mateusz.
On Wed Mar 29, 2017 at 23:14:11 +0200, Mateusz Czaplinski wrote:
On Sun, Mar 26, 2017 at 11:51 PM, Adam Lackorzynski < adam@os.inf.tu-dresden.de> wrote:
You need to build l4 for L4Linux as L4Linux also requires L4Re.
Hm; so I'm still not exactly clear about how this is structured in the repo and how it builds together. Specifically: I seem to have built the project contained in the l4linux subdir of the l4re-snapshot-2016082114 archive; does it reach from l4linux/ to ../l4re/ and ../kernel/fiasco/ in process, through the makefiles from inside l4linux/ ? It didn't complain about any missing binaries, even though I didn't give it access to any precompiled l4re or fiasco libs before "make".
No, you need to give that info. I think on your L4Linux build ARCH=x86 was somehow set so it just build a normal Linux kernel.
I seem to have replaced my old kernel with the result, and it went, ummm... suspiciously smooth... am I really running on Fiasco.OC+L4Linux? Or did I just accidentally recompile a regular kernel, and just renamed a few strings in it so that `uname --all` can now wink and say "huh huh, yeah sure, 4.7-l4, nod nod", but in reality I just tricked myself in an overly complicated way?
The top-level Makefile sets this "-l4" suffix, despite what version you're actually building.
Is there some means by which I could test/verify that yes indeed, I am now, or am not, on L4 full and proper in my VM? Some L4 API, or some other "one weird trick" I can try, that can only work with L4 (vel Fiasco)? From inside or outside (I'm doing it in Hyper-V)? I've put down the some
The build system won't just overwrite your host system, so it did not do that. (Did you bulid as root? Don't do that!) Further, some setup and configuration is needed.
Since L4Linux pretty adopted to L4Re you could for example check dmesg and see some L4-specific lines there. Or you could check /proc/interrupts.
Adam
On Fri, Mar 31, 2017 at 12:30 AM, Adam Lackorzynski < adam@os.inf.tu-dresden.de> wrote:
I think on your L4Linux build ARCH=x86 was somehow set so it just build a normal Linux kernel.
Indeed, seems that ARCH=x86_64 is called with defconfig. Is there some ARCH I can set to still get asked the L4 questions?
The top-level Makefile sets this "-l4" suffix, despite what version
you're actually building.
Ech, so I was right to be suspicious :) thanks.
The build system won't just overwrite your host system, so it did not do that. (Did you bulid as root? Don't do that!) Further, some setup and configuration is needed.
NixOS is very much automated, and the scripts I linked before result in NixOS automatically picking the result of the build (vmlinuz etc.) and adding a new default GRUB entry which would use it. So after a successful build and reboot, the new kernel is used.
As to building as root - why not? I'm doing the experiments in a fully controlled local VM, and there it's easier for me to just work as root, especially during intensive hacking. Does it break the L4Linux build somehow? (Also, Nix/NixOS builds stuff in custom non-root chroot jail IIUC anyway.)
As to "some setup and configuration" - what do you mean by that? Something L4/L4Linux-specific?
Since L4Linux pretty adopted to L4Re you could for example check dmesg and see some L4-specific lines there. Or you could check /proc/interrupts.
Oh, cool, thanks. Should it be enough to grep 'l4' or 'L4' in dmesg, or would it be something more cryptic? As to /proc/interrupts, what should I look for? `grep -i L4 /proc/interrupts` should to it?
Thanks again for your patience and help! :) /Mateusz.
On Mon Apr 03, 2017 at 19:19:18 +0200, Mateusz Czaplinski wrote:
On Fri, Mar 31, 2017 at 12:30 AM, Adam Lackorzynski < adam@os.inf.tu-dresden.de> wrote:
I think on your L4Linux build ARCH=x86 was somehow set so it just build a normal Linux kernel.
Indeed, seems that ARCH=x86_64 is called with defconfig. Is there some ARCH I can set to still get asked the L4 questions?
Yes, it's ARCH=l4, and it's the default.
The build system won't just overwrite your host system, so it did not do that. (Did you bulid as root? Don't do that!) Further, some setup and configuration is needed.
NixOS is very much automated, and the scripts I linked before result in NixOS automatically picking the result of the build (vmlinuz etc.) and adding a new default GRUB entry which would use it. So after a successful build and reboot, the new kernel is used.
I see.
As to building as root - why not? I'm doing the experiments in a fully controlled local VM, and there it's easier for me to just work as root, especially during intensive hacking. Does it break the L4Linux build somehow? (Also, Nix/NixOS builds stuff in custom non-root chroot jail IIUC anyway.)
As to "some setup and configuration" - what do you mean by that? Something L4/L4Linux-specific?
Yes. L4Re is a microkernel system and as such programs (including VMs like L4Linux) do not get access to everything per default. If a component wants to have access to a hardware device it needs to be explicitly granted to it. So most of the times it's not just working.
Since L4Linux pretty adopted to L4Re you could for example check dmesg and see some L4-specific lines there. Or you could check /proc/interrupts.
Oh, cool, thanks. Should it be enough to grep 'l4' or 'L4' in dmesg, or would it be something more cryptic? As to /proc/interrupts, what should I look for? `grep -i L4 /proc/interrupts` should to it?
Yes, there will be a couple of 'l4' in dmesg, and also in /proc/interrupts.
Please go to the l4 build directory and do "make qemu", then select the "L4Linux-basic" entry. There should be an L4Linux booting up. That's how it looks.
Adam
l4-hackers@os.inf.tu-dresden.de