Hi Qiang,

looks to me that bootstrap image to load is just too big as it interferes
with the reserved PFE areas of the platform. It seems unfortunate that
u-boot allows this to be loaded and that those areas are there at those
locations.

Anyway, I would recommend to change the loading address of bootstrap.
Please do this by creating a file Makeconf.local in the root directory
of your build-tree
(/home/qiangxu/prebuildl4/l4re-snapshot-24.08.0/obj/l4/builddir) with
the content:
DEFAULT_RELOC_arm64 = 0x0ec00000

Then build the image again. Please check the output as it needs to state
"Start address: 0x86000000"


Adam

On Mon Feb 10, 2025 at 17:25:22 +0800, qiang xu wrote:
> Hi Adam,
>
>    1. I followed your suggestion and changed *"kernel fiasco.debug
>    -serial_esc"* to *"kernel fiasco -serial_esc"*.
>    2. The error *"DT: FDT sanity check failed: FDT_ERR_BADMAGIC (-9) Total
>    RAM: 0MB"* disappeared after reloading the device tree (dtb).
>    3. Below is the boot command I used.
>
> `=> fatload mmc 0 0x80000000 s32g2xxa-evb.dtb
> 53414 bytes read in 17 ms (3 MiB/s)
> => fatload mmc 0 0x8000e000 l4re_uvmm-2vm.uimage
> 44056560 bytes read in 1871 ms (22.5 MiB/s)
> => bootm 0x8000e000 - 0x80000000
> ## Booting kernel from Legacy Image at 8000e000 ...
>    Image Name:   L4 Image #10
>    Image Type:   AArch64 Linux Kernel Image (uncompressed)
>    Data Size:    44056496 Bytes = 42 MiB
>    Load Address: 81200000
>    Entry Point:  81200000
>    Verifying Checksum ... OK
> ## Flattened Device Tree blob at 80000000
>    Booting using the fdt blob at 0x80000000
>    Loading Kernel Image
>    Using Device Tree in place at 0000000080000000, end 00000000800100a5
>    fixup: pfe0 set to 00:01:be:be:ef:11
>    fixup: pfe1 set to 00:01:be:be:ef:22
>    fixup: pfe2 set to 00:01:be:be:ef:33
>
> Starting kernel ...
>
>
> L4 Bootstrapper
>   Build: #10 Mon Feb 10 10:06:07 CST 2025, 11.4.0
>   RAM: 0000000080000000 - 00000000ffffffff: 2097152kB
>   RAM: 0000000880000000 - 00000008dfffffff: 1572864kB
>   Total RAM: 3584MB
>   New region for list regions:      [ 82ddd000,  83bbd807] {   de0808} Boot
>   Module
>   overlaps with:                    [ 83200000,  835dffff] {   3e0000} Arch
>   pfebufs@83200000
> Regions of list 'regions'
>     [ 34000000,  3407ffff] {    80000} Arch   pfebufs@34000000
>     [ 34080000,  3409ffff] {    20000} Arch   pfebufs@34080000
>     [ 80000000,  8000dfff] {     e000} Root   dtb
>     [ 81200000,  8121d777] {    1d778} Boot   bootstrap
>     [ 81200158,  81200187] {       30} Root   cpu_boot
>     [ 8121e1d0,  8121eac7] {      8f8} Boot   modinfo
>     [ 8121f000,  812e5fe7] {    c6fe8} Boot   Module
>     [ 812e6000,  812ebcd7] {     5cd8} Boot   Module
>     [ 812ec000,  8131e6cf] {    326d0} Boot   Module
>     [ 8131f000,  8141258f] {    f3590} Boot   Module
>     [ 81413000,  8142e537] {    1b538} Boot   Module
>     [ 8142f000,  814b7aa7] {    88aa8} Boot   Module
>     [ 814b8000,  814c50a5] {     d0a6} Boot   Module
>     [ 814c6000,  82dd9a52] {  1913a53} Boot   Module
>     [ 82dda000,  82ddb218] {     1219} Boot   Module
>     [ 82ddc000,  82ddc224] {      225} Boot   Module
>     [ 83200000,  835dffff] {   3e0000} Arch   pfebufs@83200000
>     [ 835e0000,  835fffff] {    20000} Arch   pfebufs@835e0000
>     [ 84000000,  843fffff] {   400000} Arch   shm@84000000
>     [ 85000000,  85000fff] {     1000} Arch   ddr@85000000
>     [ c0000000,  c03fffff] {   400000} Arch   shm@c0000000
>     [ c0400000,  c07fffff] {   400000} Arch   shm@c0400000
>     [ d0000000,  d000007f] {       80} Arch   shm@d0000000
>     [ d0000080,  d00000ff] {       80} Arch   shm@d0000080
>     [ ff800000,  ff9fffff] {   200000} Arch   atf@ff800000
> region overlap
>
> Key press reboots...
> `
> Please help me analyze where the issue might be. Thank you very much!
>
> Regards,
> Qiang
>
> On Mon, Feb 10, 2025 at 3:03 AM Adam Lackorzynski <adam@l4re.org> wrote:
>
> > Hi Qiang,
> >
> > thanks for the description and all the logs. Could you also please share
> > the command given in u-boot (bootm...)?
> >
> > From the logs, the following lines look suspicious:
> >
> > > DT: FDT sanity check failed: FDT_ERR_BADMAGIC (-9)
> > >   Total RAM: 0MB
> >
> > Indicating that something with the device tree is not ok.
> >
> > > kernel fiasco.debug -serial_esc
> >
> > Please use "fiasco" only. The fiasco.debug is considerably bigger to load
> > and could contribute to the region overlap.
> >
> >
> >
> > Adam
> >
> > On Thu Feb 06, 2025 at 19:53:49 +0800, qiang xu wrote:
> > > Hi Adam,
> > >   I encountered the error "region overlap" while running L4Re on S32G.
> > >
> > > First, I built the Linux BSP 38.0 using Yocto to generate the Image,
> > > s32g2xxa-evb.dtb, and fsl-image-dom0less-s32g274aevb.cpio.gz. Then, I
> > > compiled the S32G platform's L4Re image using the snapshots
> > > <https://l4re.org/download/snapshots/README.html> method. In the
> > > modules.list, I added the following:
> > >
> > > `
> > > entry uvmm-2vm
> > > kernel fiasco.debug -serial_esc
> > > roottask moe rom/uvmm-2vm.ned
> > > module uvmm
> > > module l4re
> > > module ned
> > > module s32g2xxa-evb.dtb
> > > module fsl-image-dom0less-s32g274aevb.cpio.gz
> > > module[shell] echo $SRC_BASE_ABS/pkg/uvmm/configs/vmm.lua
> > > module uvmm-2vm.ned
> > > module Image
> > > module cons
> > > `
> > >
> > > Finally, I ran the command make uimage E=uvmm-2vm PT=s32g in the
> > > /l4re-snapshot-24.08.0/obj/l4/builddir directory, which showed the
> > > following output:
> > >
> > > `
> > > make[1]: Entering directory
> > > '/home/qiangxu/prebuildl4/l4re-snapshot-24.08.0/src/l4'
> > > [bootstrap - nofpu] ... Building Dependencies
> > > Building entry "uvmm-2vm".
> > > Merging images:
> > > mod00:
> > >
> > /home/qiangxu/prebuildl4/l4re-snapshot-24.08.0/obj/fiasco/builddir/fiasco.debug
> > > [27393kB -> 796kB]
> > > mod01:
> > >
> > /home/qiangxu/prebuildl4/l4re-snapshot-24.08.0/obj/l4/builddir/bin/arm64_armv8a/l4f/sigma0
> > > [24kB]
> > > mod02:
> > >
> > /home/qiangxu/prebuildl4/l4re-snapshot-24.08.0/obj/l4/builddir/bin/arm64_armv8a/l4f/moe
> > > [202kB]
> > > mod03:
> > >
> > /home/qiangxu/prebuildl4/l4re-snapshot-24.08.0/obj/l4/builddir/bin/arm64_armv8a/l4f/uvmm
> > > [974kB]
> > > mod04:
> > >
> > /home/qiangxu/prebuildl4/l4re-snapshot-24.08.0/obj/l4/builddir/bin/arm64_armv8a/l4f/l4re
> > > [110kB]
> > > mod05:
> > >
> > /home/qiangxu/prebuildl4/l4re-snapshot-24.08.0/obj/l4/builddir/bin/arm64_armv8a/l4f/ned
> > > [547kB]
> > > mod06:
> > > /home/qiangxu/prebuildl4/l4re-snapshot-24.08.0/obj/conf/s32g2xxa-evb.dtb
> > > [53kB]
> > > mod07:
> > >
> > /home/qiangxu/prebuildl4/l4re-snapshot-24.08.0/obj/conf/fsl-image-dom0less-s32g274aevb.cpio.gz
> > > [25679kB]
> > > mod08:
> > >
> > /home/qiangxu/prebuildl4/l4re-snapshot-24.08.0/src/l4/pkg/uvmm/configs/vmm.lua
> > > [5kB]
> > > mod09:
> > /home/qiangxu/prebuildl4/l4re-snapshot-24.08.0/obj/conf/uvmm-2vm.ned
> > > [1kB]
> > > mod10: /home/qiangxu/prebuildl4/l4re-snapshot-24.08.0/obj/conf/Image
> > > [14211kB]
> > > mod11:
> > >
> > /home/qiangxu/prebuildl4/l4re-snapshot-24.08.0/obj/l4/builddir/bin/arm64_armv8a/l4f/cons
> > > [280kB]
> > > [bootstrap - nofpu] ... Generating bootstrap.ld
> > > [bootstrap - nofpu] ... Compiling startup.o
> > > [bootstrap - nofpu] ==> Linking l4re.elf
> > > [bootstrap] ==> Image post-processing l4re.elf
> > > [bootstrap - nofpu] ==> l4re.elf built
> > > ==> Installing l4re.elf in image directory
> > > ==> Installing l4re_uvmm-2vm in image directory
> > > ==> Installing l4re_uvmm-2vm.elf in image directory
> > > [bootstrap - nofpu] ... Generating l4re.raw
> > > ==> Installing l4re.raw in image directory
> > > ==> Installing l4re_uvmm-2vm.raw in image directory
> > > [bootstrap - nofpu] ... Generating l4re.uimage
> > > Image Name:   L4 Image #9
> > > Created:      Thu Feb 6 19:29:02 2025
> > > Image Type:   AArch64 Linux Kernel Image (uncompressed)
> > > Data Size:    44056496 Bytes = 43023.92 KiB = 42.02 MiB
> > > Load Address: 81200000
> > > Entry Point:  81200000
> > > ==> Installing l4re.uimage in image directory
> > > ==> Installing l4re_uvmm-2vm.uimage in image directory
> > > Image size(s) in bytes:
> > >             l4re_uvmm-2vm.elf:  44692328
> > >                      l4re.raw:  44056496
> > >                   l4re.uimage:  44056560
> > > Start address:
> > > --> Build-Nr: 9
> > > [bootstrap - nofpu] ==> Installing l4re.elf to local build-tree
> > > [bootstrap - nofpu] ==> Installing l4re_uvmm-2vm to local build-tree
> > > [bootstrap - nofpu] ==> Installing l4re_uvmm-2vm.elf to local build-tree
> > > `
> > > Finally, I tried loading the l4re_uvmm-2vm.uimage and s32g2xxa-evb.dtb
> > > files in U-Boot for booting, but it resulted in an error. Can you help me
> > > identify where the issue might be?
> > >
> > > `
> > > L4 Bootstrapper
> > >   Build: #8 Thu Feb 6 14:58:40 CST 2025, 11.4.0
> > > DT: FDT sanity check failed: FDT_ERR_BADMAGIC (-9)
> > >
> > > L4 Bootstrapper
> > >   Build: #8 Thu Feb 6 14:58:40 CST 2025, 11.4.0
> > >   Total RAM: 0MB
> > >   New region for list regions:      [ 82ddd000,  855979ff] {  27baa00}
> > Boot
> > >   Module
> > >   overlaps with:                    [ 83200000,  835dffff] {   3e0000}
> > Arch
> > >   pfebufs@83200000
> > > Regions of list 'regions'
> > >     [ 34000000,  3407ffff] {    80000} Arch   pfebufs@34000000
> > >     [ 34080000,  3409ffff] {    20000} Arch   pfebufs@34080000
> > >     [ 80000000,  8000d0a5] {     d0a6} Root   dtb
> > >     [ 81200000,  8121d777] {    1d778} Boot   bootstrap
> > >     [ 81200158,  81200187] {       30} Root   cpu_boot
> > >     [ 8121e1d0,  8121eadf] {      910} Boot   modinfo
> > >     [ 8121f000,  812e5f7f] {    c6f80} Boot   Module
> > >     [ 812e6000,  812ebcd7] {     5cd8} Boot   Module
> > >     [ 812ec000,  8131e6cf] {    326d0} Boot   Module
> > >     [ 8131f000,  8141258f] {    f3590} Boot   Module
> > >     [ 81413000,  8142e537] {    1b538} Boot   Module
> > >     [ 8142f000,  814b7aa7] {    88aa8} Boot   Module
> > >     [ 814b8000,  814c50a5] {     d0a6} Boot   Module
> > >     [ 814c6000,  82dd9a52] {  1913a53} Boot   Module
> > >     [ 82dda000,  82ddb218] {     1219} Boot   Module
> > >     [ 82ddc000,  82ddc22a] {      22b} Boot   Module
> > >     [ 83200000,  835dffff] {   3e0000} Arch   pfebufs@83200000
> > >     [ 835e0000,  835fffff] {    20000} Arch   pfebufs@835e0000
> > >     [ 84000000,  843fffff] {   400000} Arch   shm@84000000
> > >     [ 85000000,  85000fff] {     1000} Arch   ddr@85000000
> > >     [ c0000000,  c03fffff] {   400000} Arch   shm@c0000000
> > >     [ c0400000,  c07fffff] {   400000} Arch   shm@c0400000
> > >     [ d0000000,  d000007f] {       80} Arch   shm@d0000000
> > >     [ d0000080,  d00000ff] {       80} Arch   shm@d0000080
> > > region overlap
> > >
> > > Key press reboots...
> > >
> > > `
> > > Regards,
> > > Qiang
> > >
> > >
> > > On Wed, Jan 8, 2025 at 10:13 AM qiang xu <qiangxu431@gmail.com> wrote:
> > >
> > > > Hi Adam,
> > > >
> > > > Awesome, thanks for the info!
> > > >
> > > > Regards,
> > > > Qiang
> > > >
> > > > On Tue, Jan 7, 2025 at 10:03 PM Adam Lackorzynski <adam@l4re.org>
> > wrote:
> > > >
> > > >> Hi Qiang,
> > > >>
> > > >> thanks for the great feedback.
> > > >>
> > > >> It's correct, the L4Re microerkernel Fiasco runs in EL2 only. It runs
> > > >> both VMs and L4Re applications side by side, so yes, it is a
> > hypervisor
> > > >> too.
> > > >>
> > > >>
> > > >> Best regards,
> > > >> Adam
> > > >>
> > > >> On Tue Jan 07, 2025 at 19:34:20 +0800, qiang xu wrote:
> > > >> >   Hi Adam,
> > > >> >
> > > >> >  Thank you very much for your support. The explanation was very
> > clear.
> > > >> From
> > > >> > the printed information, it seems to be running fine. I have one
> > more
> > > >> > question: Can Fiasco be used as a hypervisor? >From the example of
> > > >> > l4re_VM-multi, it seems to have the ability to run multiple
> > operating
> > > >> > systems on a single platform, and in L4Re, only Fiasco runs in EL2
> > > >> > privilege level. I'm not sure if I understand this correctly, so I
> > > >> wanted
> > > >> > to confirm with you. If it cannot be used as a hypervisor, what
> > > >> > functionality does it lack that a hypervisor would typically have?
> > > >> >
> > > >> > Regards,
> > > >> > Qiang
> > > >> >
> > > >> >
> > > >> > On Tue, Jan 7, 2025 at 7:53 AM Adam Lackorzynski <adam@l4re.org>
> > wrote:
> > > >> >
> > > >> > > Hi Qiang,
> > > >> > >
> > > >> > > If I'm not mistaken, the provided BSP is using u-boot. With
> > u-boot,
> > > >> the
> > > >> > > uimage file is a good fit. When booting up, you can stop at the
> > u-boot
> > > >> > > prompt and do something like this:
> > > >> > > # fatload mmc 0 0x80f00000 fsl-s32g274a-evb.dtb
> > > >> > > # fatload mmc 0 0x80ffffc0 l4re_vm-multi-p2p_s32g.uimage
> > > >> > > # bootm 0x80ffffc0  - 0x80f00000
> > > >> > >
> > > >> > > You need to copy the l4re_vm-multi-p2p_s32g.uimage file to the mmc
> > > >> first
> > > >> > > of course, for example, through Linux.
> > > >> > > Alternative you could also load the elf-image or raw-image into
> > memory
> > > >> > > via your hardware debugger, whatever method the debugger
> > supports. Do
> > > >> it
> > > >> > > by intercepting at the u-boot prompt.
> > > >> > >
> > > >> > > A third option is to use the tftp network capabilities of u-boot
> > to
> > > >> load
> > > >> > > the uimage via network.
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > Adam
> > > >> > >
> > > >> > > On Mon Jan 06, 2025 at 11:04:35 +0800, qiang xu wrote:
> > > >> > > > Hi Adam,
> > > >> > > >
> > > >> > > > Thank you for your response.
> > > >> > > >
> > > >> > > > I found the following images in the L4Re pre-built images
> > folder:
> > > >> > > > “l4re_vm-multi-p2p_s32g.efi,” “l4re_vm-multi-p2p_s32g.elf,” and
> > > >> > > > “l4re_vm-multi-p2p_s32g.uimage.” I would like to understand how
> > > >> these
> > > >> > > > images can be combined with the BSP built using Yocto and
> > placed on
> > > >> the
> > > >> > > SD
> > > >> > > > card for booting.
> > > >> > > >
> > > >> > > >  I have been following the examples from the L4Re wiki
> > > >> > > > <https://github.com/kernkonzept/manifest/wiki/MultipleVMs> to
> > get
> > > >> > > started
> > > >> > > > with L4Re. These examples run very well on QEMU, but there is an
> > > >> issue—I
> > > >> > > am
> > > >> > > > unable to use GDB to debug the Fiasco kernel.I noticed that L4Re
> > > >> can also
> > > >> > > > run on the S32G platform, so I would like to run L4Re on S32G
> > and
> > > >> use a
> > > >> > > > hardware debugging tool to debug the Fiasco kernel during its
> > > >> execution.
> > > >> > > > However, the wiki does not provide instructions on how to run
> > L4Re
> > > >> on
> > > >> > > real
> > > >> > > > hardware.Therefore, I would like your guidance on how to run the
> > > >> examples
> > > >> > > > from the wiki <
> > > >> https://github.com/kernkonzept/manifest/wiki/MultipleVMs>
> > > >> > > on
> > > >> > > > the S32G platform and boot from an SD card.
> > > >> > > >
> > > >> > > > Regards,
> > > >> > > > Qiang
> > > >> > > >
> > > >> > > > On Mon, Jan 6, 2025 at 9:56 AM Adam Lackorzynski <adam@l4re.org
> > >
> > > >> wrote:
> > > >> > > >
> > > >> > > > > Hi Qiang,
> > > >> > > > >
> > > >> > > > > On Sat Jan 04, 2025 at 17:03:21 +0800, qiang xu wrote:
> > > >> > > > > > I would like to know how to run L4Re on the S32G. I couldn’t
> > > >> find any
> > > >> > > > > > related documentation. I’m a beginner in this technical
> > field,
> > > >> so
> > > >> > > could
> > > >> > > > > you
> > > >> > > > > > provide friendly step-by-step instructions?
> > > >> > > > >
> > > >> > > > > One option is to use the snapshot from l4re.org as it has a
> > > >> > > menu-driven
> > > >> > > > > target selection which includes the S32G. This hides quite a
> > few
> > > >> of the
> > > >> > > > > details but gives examples for the S32G right away.
> > > >> > > > > How do you boot your S32G?
> >

Adam
--
Adam                 adam@os.inf.tu-dresden.de
  Lackorzynski         http://os.inf.tu-dresden.de/~adam/