L4 on ARM: cache coherency and virtual address space
Hello, I am implementing an USB stack for ARM machine maintained by the L4 microkernel. It works quite well at the moment - USB hubs and HID devices are working properly. However, there are few open issues there: 1. Since ARM machine I use does not provide cache snooping, i.e. I have to flush/invalidate the caches manually, I've attempted to use the l4_cache_dma_coherent function. It did worked to some degree, but it flushed/invalidated only *some* lines of cache within requested region, instead of *all* lines of cache. Since I've failed to trace the bug, I've extended sigma0 with new functionality - ability to map uncached memory. I still don't like my hack though and I would like to find an official way to either change the attributes (from cached to uncached) of given page, or have fully working l4_cache_dma_coherent. 2. The USB stack is a multithreaded app where I would like to have different servers (e.g. server for HID input events, server for mass storage devices). Unfortunately a server seemed to work only when started from the main thread. Why is it so? Was that my fault? 3. When the mass storage server receives either read or write call, it needs to map caller's memory in uncached manner. Moreover, USB stack needs *physical* address of the caller's buffer. How can I find that out? with best regards, Michal. -- Michal Schulz
Hi, On Sun May 08, 2011 at 22:18:28 +0200, Michal Schulz wrote:
I am implementing an USB stack for ARM machine maintained by the L4 microkernel. It works quite well at the moment - USB hubs and HID devices are working properly. However, there are few open issues there:
1. Since ARM machine I use does not provide cache snooping, i.e. I have to flush/invalidate the caches manually, I've attempted to use the l4_cache_dma_coherent function. It did worked to some degree, but it flushed/invalidated only *some* lines of cache within requested region, instead of *all* lines of cache. Since I've failed to trace the bug, I've extended sigma0 with new functionality - ability to map uncached memory. I still don't like my hack though and I would like to find an official way to either change the attributes (from cached to uncached) of given page, or have fully working l4_cache_dma_coherent.
'io' has the feature to get uncached memory, defining a resource Mmio_ram(size_in_bytes, 1_for_superpages_0_otherwise) in the configuration. That's the official way of getting uncached memory.
2. The USB stack is a multithreaded app where I would like to have different servers (e.g. server for HID input events, server for mass storage devices). Unfortunately a server seemed to work only when started from the main thread. Why is it so? Was that my fault?
The registry defaults to using the main thread but can be created with other parameters. The extended version is L4Re::Util::Registry_server<> server(L4Re::Env::env()->main_thread(), L4Re::Env::env()->factory());
3. When the mass storage server receives either read or write call, it needs to map caller's memory in uncached manner.
You do a map/unmap for each request? I'd establish the shared memory once.
Moreover, USB stack needs *physical* address of the caller's buffer. How can I find that out?
When using io as a memory supplier for uncached memory the resource of this virtual device also comes with its physical address. Adam -- Adam adam@os.inf.tu-dresden.de Lackorzynski http://os.inf.tu-dresden.de/~adam/
Hello Adam, On Tue, May 10, 2011 at 6:26 PM, Adam Lackorzynski <adam@os.inf.tu-dresden.de> wrote:
1. Since ARM machine I use does not provide cache snooping, i.e. I have to flush/invalidate the caches manually, I've attempted to use the l4_cache_dma_coherent function. It did worked to some degree, but it flushed/invalidated only *some* lines of cache within requested region, instead of *all* lines of cache. Since I've failed to trace the bug, I've extended sigma0 with new functionality - ability to map uncached memory. I still don't like my hack though and I would like to find an official way to either change the attributes (from cached to uncached) of given page, or have fully working l4_cache_dma_coherent.
'io' has the feature to get uncached memory, defining a resource Mmio_ram(size_in_bytes, 1_for_superpages_0_otherwise) in the configuration. That's the official way of getting uncached memory.
The solution you gave is not working, since moe allocated all available RAM already before io even started. Log says: MOE: Hello world MOE: found 502196 KByte free memory MOE: found RAM from 90000000 to af000000 [...] IO | WARNING: phys mmio resource allocation failed IO | IOMEM [000000903b3000-000000903b3fff 1000] non-pref (32bit) (align=fff flags=c002)
The registry defaults to using the main thread but can be created with other parameters. The extended version is L4Re::Util::Registry_server<> server(L4Re::Env::env()->main_thread(), L4Re::Env::env()->factory());
Thanks for the info, I will check that ASAP! with best regards, Michal. -- Michal Schulz
Hi, On Tue May 10, 2011 at 21:11:21 +0200, Michal Schulz wrote:
On Tue, May 10, 2011 at 6:26 PM, Adam Lackorzynski <adam@os.inf.tu-dresden.de> wrote:
1. Since ARM machine I use does not provide cache snooping, i.e. I have to flush/invalidate the caches manually, I've attempted to use the l4_cache_dma_coherent function. It did worked to some degree, but it flushed/invalidated only *some* lines of cache within requested region, instead of *all* lines of cache. Since I've failed to trace the bug, I've extended sigma0 with new functionality - ability to map uncached memory. I still don't like my hack though and I would like to find an official way to either change the attributes (from cached to uncached) of given page, or have fully working l4_cache_dma_coherent.
'io' has the feature to get uncached memory, defining a resource Mmio_ram(size_in_bytes, 1_for_superpages_0_otherwise) in the configuration. That's the official way of getting uncached memory.
The solution you gave is not working, since moe allocated all available RAM already before io even started. Log says:
MOE: Hello world MOE: found 502196 KByte free memory MOE: found RAM from 90000000 to af000000 [...] IO | WARNING: phys mmio resource allocation failed IO | IOMEM [000000903b3000-000000903b3fff 1000] non-pref (32bit) (align=fff flags=c002)
The error message is misleading (sorry) and hasn't anything to do with the memory allocation itself. For the cache, does it work when you use l4_cache_dma_coherent_full()? Which core are you using? Adam -- Adam adam@os.inf.tu-dresden.de Lackorzynski http://os.inf.tu-dresden.de/~adam/
participants (2)
-
Adam Lackorzynski -
Michal Schulz