About IO server settings for DMA in L4Linux

[Jesse-SC Chou (周書正)]

Hi Adam,

Many thanks for your reply.
With your instruction, I have tried to revise my I/O server configurations to the simplest form, which contains only 2 devices and no DMAMEM.
Here is my I/O server configurations:
----------------------------------------
l4linux-io.devs:

local Res = Io.Res
local Hw = Io.Hw
Io.hw_add_devices(function()
  DEVICE1 = Hw.Device(function()
    Property.hid = "device1";
    compatible = {"arm,device1"};
    Resource.regs = Res.mmio(0x14000000, 0x15ffffff);
  end);
  DEVICE2 = Hw.Device(function()
    Property.hid = "device2";
    compatible = {"arm,device2"};
    Resource.regs = Res.mmio(0x1F000000, 0x1F3fffff);
    Resource.irq = Res.irq(66);
  end);
end)
----------------------------------------
l4linux-io.io:

local hw = Io.system_bus()

Io.add_vbus("l4linux", Io.Vi.System_bus
{  
  DEVICE1 = wrap(hw:match("arm,device1"));
  DEVICE2 = wrap(hw:match("arm,device2"));
})
----------------------------------------
However, I've got a very strange virtual memory to physical mapping without the flag "Io.Hw_device_DF_dma_supported".
On my platform, the memory physical memory is at "0x20000000 ~ 0x27FFFFFF".
When without the flag "Io.Hw_device_DF_dma_supported", the dma_alloc_coherent() give me a physical address "0x05a2d000". This is not a reasonable physical memory address.
When adding the flag "Io.Hw_device_DF_dma_supported" to DEVICE1, the dma_alloc_coherent() give me a physical address "0x2502d000". This is a reasonable physical memory address.

I traced into dma_alloc_coherent(), finally it will call to l4x_virt_to_phys(). I printed out the content of v2p_for_each(i) data structure:
----------------------------------------
Without "Io.Hw_device_DF_dma_supported" in DEVICE1: (the physical address is not correct)

i->virt=0x02000000 i->phys=0x02000000 i->size=48f000
i->virt=0x02600000 i->phys=0x02600000 i->size=4000000
i->virt=0x06604000 i->phys=0x060c0000 i->size=40000
i->virt=0x06732000 i->phys=0x05a2d000 i->size=1000
----------------------------------------
With "Io.Hw_device_DF_dma_supported" in DEVICE1: (the physical address is reasonable)

i->virt=0x02000000 i->phys=0x2017c000 i->size=48f000
i->virt=0x02600000 i->phys=0x21c00000 i->size=4000000
i->virt=0x06604000 i->phys=0x256c0000 i->size=40000
i->virt=0x06732000 i->phys=0x2502d000 i->size=1000
----------------------------------------
The result seems the v2p mapping is wrong when without "Io.Hw_device_DF_dma_supported".

I have the following questions:
1. The above result seems not reasonable. Do you have any idea?
2. I also noticed that in arm_dma_alloc(), l4x_dmapool_mem_alloc() is wrapped by "CONFIG_L4_DMAPOOL". Is "CONFIG_L4_DMAPOOL" mandatory?
3. If "CONFIG_L4_DMA_POOL" is optional, arm_dma_alloc() will call "__dma_alloc(dev, size, handle, gfp, prot, false, attrs, __builtin_return_address(0))".
the 6th argument is "false". Does it mean the dma_alloc_coherent() in L4Linux always return a non-coherent DMA region? That is, we have to handle the cache issue by myself.

Any suggestion is very appreciated. Thanks.

BR,
Jesse

-----Original Message-----
From: l4-hackers [mailto:l4-hackers-bounces at os.inf.tu-dresden.de] On Behalf Of Adam Lackorzynski
Sent: Saturday, December 10, 2016 6:48 AM
To: l4-hackers at os.inf.tu-dresden.de
Subject: Re: About IO server settings for DMA in L4Linux

Hi,

On Fri Dec 09, 2016 at 09:21:37 +0000, Jesse-SC Chou (©P®Ñ¥¿) wrote:
> I'm a newbie of L4 micro-kernel. The Fiasco/L4/L4Linux is really a good design for system architecture in the future.
> I'm trying to porting the L4 system to my target then evaluate.
> On my target, there is an DMA controller, which moves data to flash.
> Originally, Native-Linux uses dma_alloc_coherent() to allocate a DMA-dedicated memory.
> CPU copies data to that DMA-dedicated memory, then DMA controller moves data from DMA-dedicated memory to flash.
> 
> In L4Linux, I guess there should be some settings of DMA in I/O server configuration files.
> I'm searching for an example of configuring DMA-related settings.
> 
> I use the snapshot ¡§l4re-snapshot-2016082114¡¨. In my current 
> configuration, "DEVICE1" is the DMA controller, "DEVICE2" is the flash, and "DMAMEM" is the DMA-dedicated memory. However, it generates some DMA-related error in boot messages.
> Such as:
> 
> IO      | no 'iommu' capability found use CPU-phys for DMA
> IO      | warning: inconsistent fixed resource @ device: /System Bus/
> error: failed to get physical address for 2000000.
> 
> And so on.
> 
> The first thing I want to do is to make the boot message correct and I have the following questions:
> 1. Is the "DMAMEM" device needed to be configured? If needed, how to specify this to be a DMA memory?
> 2. Will dma_alloc_coherent() use the "DMAMEM" or other memory? Can I use dma_alloc_coherent() as Native-Linux?
> 3. I've also found a flag "Property.flags = Io.Hw_device_DF_dma_supported;". I'm not sure its usage. I just add this flag to DEVICE1(DMA controller) and DEVICE2(flash), is it correct?
> 4. Is my l4linux-io.devs and l4linux-io.io correct or need other modification for DMA?
> 5. I¡¦ve found ¡§IO      |   DMADOM  [00000000000000-00000000000000 1] non-pref (32bit) (align=0 flags=6)¡¨ in the boot message. What the ¡§DMADOM¡¨ means?
> 
> Any suggestion is very appreciated. Thanks.

I think you're thinking too complicated. In case all of the devices (including the DMA engine) are driven from within L4Linux it should just work. Your description sounds as this is the case.
L4Linux makes sure the proper physical addresses are used. Just specify the MMIO regions and IRQs of the devices in the IO config (and in the device tree) so that L4Linux is able to access them. Do not use any flags in the IO config.



Adam
-- 
Adam                 adam at os.inf.tu-dresden.de
  Lackorzynski         http://os.inf.tu-dresden.de/~adam/

_______________________________________________
l4-hackers mailing list
l4-hackers at os.inf.tu-dresden.de
http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers