On Thu, Oct 31, 2024 at 8:47 PM Philipp Eppelt <philipp.eppelt@kernkonzept.com> wrote:

Hi Mohamed,

Am 31.10.24 um 14:30 schrieb Mohamed Dawod:
> Thanks Philipp,
>
> Multiple CPUs worked for virt-arm64 machine
Yippie!

> I tried to launch 2 linux VMs on top of L4 using uvmm and assign 2 CPUs to one
> of the two VMs and another 2 CPUs to the other one.
> I noticed that the linux booting process becomes slower and as more CPUs are
> added to qemu with -smp option and passed to the VMs as more as the VM booting
> becomes more slower!
> Also VMs become working randomly (sometimes they work and sometimes they hang or
> one of them hangs) >
> Why does this strange behaviour happen when using uvmm and 2 Linux VMs ?
To make this easier please show me your ned script starting the VMs.

If you use start_vm() please note that the `cpus` parameter takes a bitmap. So
make sure to start VM1 with `cpus=0x3` and VM2 with `cpus=0xc` to place them on
separate cores of a four core platform (e.g. QEMU with -smp 4).

Cheers,
Philipp

>
> Thanks,
> Regards
>
> On Wed, Oct 30, 2024 at 8:46 PM Philipp Eppelt <philipp.eppelt@kernkonzept.com
> <mailto:philipp.eppelt@kernkonzept.com>> wrote:
>
> Hi Mohamed,
>
> Am 29.10.24 um 10:43 schrieb Mohamed Dawod:
> > Hello,
> > I'm trying to provide multiple CPUs for a linux VM on top of L4.
> > I'm using the qemu virt machine and building for aarch64. so I used *-smp*
> > option to provide more CPUs.
> >
> > $ qemu-system-aarch64 -M virt,virtualization=true -cpu cortex-a57
> -smp 4 -m
> > 1024 -kernel ....etc....
> I'm not sure which gic version qemu uses. Please try setting it explicitly to
> with gic-version=3 argument: `-M virt,virtualization=true,gic-version=3`
>
> >
> > Unfortunately, This didn't work. I tried to add more CPU device nodes to
> the dts
> > file *virt-arm_virt-64.dts *but it also didn't work. >
> > I think that it's because of the provided interrupt-controller with
> > *virt-arm_virt-64.dts* in /l4/pkg/uvmm/conf/dts/ which mentioned that it
> > supports only one CPU.
> >
> > icsoc {
> > compatible = "simple-bus";
> > #address-cells = <2>;
> > #size-cells = <2>;
> > ranges;
> >
> > /* Uvmm will adapt the compatible string depending on the
> present gic
> > * version. It expects reg entries that provide enough space
> for the
> > * Cpu/Dist interface for gicv2 (at least 0x1000, 0x1000) or the
> > * Dist/Redist interface for gicv3 (0x10000, 0x20000 *
> number of cpus).
>
> I'm not an expert for ARM64, but judging from the line above I'd say you
> have to
> increase the size of the second reg entry. For example for four cores:
> reg = <0 0x40000 0 0x10000>,
> <0 0x50000 0 0x80000>;
>
> You should be able to just use the github version of this file, it has a gic
> node that is configured for 32 cores and comes with four CPU nodes.
> https://github.com/kernkonzept/uvmm/blob/master/configs/dts/virt-arm_virt-64.dts <https://github.com/kernkonzept/uvmm/blob/master/configs/dts/virt-arm_virt-64.dts>
>
>
> > * *The entries provided here support any gicv2 setup or a
> gicv3 setup
> > * with one Cpu.*
> > */
> > gic: interrupt-controller {
> > compatible = "arm,gic-400", "arm,cortex-a15-gic",
> > "arm,cortex-a9-gic";
> > #interrupt-cells = <3>;
> > #address-cells = <0>;
> > interrupt-controller;
> > reg = <0 0x40000 0 0x10000>,
> > <0 0x50000 0 0x20000>;
> > };
> > };
> >
> >
> > My question now, is there any workaround to support multiple CPUs for virt
> > machine on arm64 ?
>
> Multiple CPUs should work. For SMP there are a couple of things to consider:
> - QEMU: -smp parameter
> - Kernel configuration for SMP and the number of maximum cores
> - The DTS defines the maximum number of cores for the uvmm will set up. So
> adding CPU device nodes is the correct path.
> - The ned script defines the number of cores available at runtime to uvmm. No
> cpu parameter in the start_vm({}) call means the VM gets access to all cpus.
> - Linux must of course also support SMP, but that's very likely not the problem
> here ;-)
>
> I hope this sheds some light.
>
> Cheers,
> Philipp
>
> --
> philipp.eppelt@kernkonzept.com <mailto:philipp.eppelt@kernkonzept.com> -
> Tel. 0351-41 883 221
> http://www.kernkonzept.com <http://www.kernkonzept.com>
>
> Kernkonzept GmbH. Sitz: Dresden. Amtsgericht Dresden, HRB 31129.
> Geschäftsführer: Dr.-Ing. Michael Hohmuth
> _______________________________________________
> l4-hackers mailing list -- l4-hackers@os.inf.tu-dresden.de
> <mailto:l4-hackers@os.inf.tu-dresden.de>
> To unsubscribe send an email to l4-hackers-leave@os.inf.tu-dresden.de
> <mailto:l4-hackers-leave@os.inf.tu-dresden.de>
>
>
>
> *Driving Innovation! Visit our website www.avelabs.com
> <http://www.avelabs.com/>*, to read Avelabs Confidentiality Notice, follow this
> link: http://www.avelabs.com/email/disclaimer.html
> <http://www.avelabs.com/email/disclaimer.html>

--
philipp.eppelt@kernkonzept.com - Tel. 0351-41 883 221
http://www.kernkonzept.com

Kernkonzept GmbH. Sitz: Dresden. Amtsgericht Dresden, HRB 31129.
Geschäftsführer: Dr.-Ing. Michael Hohmuth