Hey Martin, thanks for that tip. How do I use this device for L4linux though (and only for it)? I am not sure where memory allocation occurs when L4linux faults or if it is pre-allocated completely to L4linux. Best, ramya ________________________________________ From: l4-hackers [l4-hackers-bounces@os.inf.tu-dresden.de] on behalf of Martin Schröder [martin.schroeder@openlimit.com] Sent: 09 September 2014 11:56 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Physical memory allocation to L4linux Hi Ramya, Am 09.09.2014 um 11:39 schrieb Masti Ramya Jayaram:

Hi all,

I start l4linux by specifying 64M on memory in its configuration file. I would like this memory to be a fixed range of physical memory and I was wondering if there is a way to do it.

MYRAM => new Device() { .hid= "myram"; new-res Mmio(0x10000000 .. 0x13FFFFFF, 0xC000); } [http://os.inf.tu-dresden.de/pipermail/l4-hackers/2014/006617.html] Best regards, Martin. _______________________________________________ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers

On Tue Sep 09, 2014 at 16:03:11 +0000, Masti Ramya Jayaram wrote:

I would like to explain my situation better. Here is what I intend:

a. What is the lowest module (bootstrap, fiasco, sigma, moe, ned,l4linux) that can be confined not access a portion of the address space?

Without changing anything and with this list of program, it's just L4Linux. Reason is that ned typically has a cap to sigma0 because it needs to give it to io. When you would like to make a little change, in Fiasco there's a function handle_sigma0_page_fault that covers page-fault by sigma0. So if you add a check on pfa there and return false you should be able to exclude a memory region from any user program.

I know that bootstrap and fiasco run in privileged mode, so there is no way to stop them. What is the next module?

b. Assuming that it is sigma, I would like to do the following: Have three regions in the physical address space (not necessarily memory):

i) one for bootstrap, fiasco ii) moe, ned, sigma, l4linux iii). Special region accessible only from (i) - bootstrap and fiasco.

If it is not possible to confine sigma, i.e., "hide a portion of the address space from it", then add sigma to lists (i,iii) and remove it from (ii) - and so on for the other modules.

If it is moe, could my goal be reached by adding an IO device as a blocker but then not give moe the corresponding capability?

In the stack moe is below any notion of an IO device, so that would not work. Adam -- Adam adam@os.inf.tu-dresden.de Lackorzynski http://os.inf.tu-dresden.de/~adam/

Update: Sorry, I misunderstood a few things in the previous email. The change was in fiasco and not sigma and therefore, in theory, no user space process should have access to the protected region. This also implies that sigma0 will not have access if I understand correctly which is rather nice. So, I tried changing the handle_sigma0_page_fault function. This is located in kernel/fiasco/src/kern/ia32/thread-ia32.cpp to prevent mapping of some range of physical memory (NOT RAM) as follows: IMPLEMENT inline bool Thread::handle_sigma0_page_fault(Address pfa) { size_t size; // Check if mapping a superpage doesn't exceed the size of physical memory if (Cpu::have_superpages() // Some distributions do not allow to mmap below a certain threshold // (like 64k on Ubuntu 8.04) so we cannot map a superpage at 0 if // we're Fiasco-UX && (!Config::Is_ux || !(pfa < Config::SUPERPAGE_SIZE))) { pfa &= Config::SUPERPAGE_MASK; size = Config::SUPERPAGE_SIZE; } else { pfa &= Config::PAGE_MASK; size = Config::PAGE_SIZE; } ---------------------------------------------------------------------------------------------------------------------------------- - My change start if((pfa>=0xe0000000) && (pfa<=0xefffffff)){ printf("\nNot allowed...denying\n"); return Mem_space::Insert_err_nomem; } ---------------------------------------------------------------------------------------------------------------------------------- - My change end return mem_space()->v_insert(Mem_space::Phys_addr(pfa), Mem_space::Addr(pfa), Mem_space::Size(size), Mem_space::Page_writable | Mem_space::Page_user_accessible) != Mem_space::Insert_err_nomem; } However, I realized that when I try to map this address space using l4io_request_iomem, I see that it goes through and the call never reaches until fiasco's sigma0_page_fault handler. A little more debugging revealed that it goes as far as the following function in l4/pkg/sigma0/server/src/memmap.cc: static void map_mem(l4_fpage_t fp, Memory_type fn, l4_umword_t t, Answer *an) { an->clear(); Mem_man *m; switch (fn) { case Ram: m = Mem_man::ram(); break; case Io_mem: case Io_mem_cached: m = &iomem; break; default: return; } unsigned long addr = m->alloc(Region::bs(fp.raw & ~((1UL << 12) - 1), 1UL << l4_fpage_size(fp), t)); if (addr == ~0UL) return; /* the Fiasco kernel makes the page non-cachable if the frame * address is greater than mem_high */ an->snd_base(addr); an->snd_fpage(addr, l4_fpage_size(fp), false, fn != Io_mem); an->tag = l4_msgtag(0, 0, 1, 0); return; }

From here on, the generics make it a little hard to parse. So I am not sure where the control flow ends but it does not get to fiasco's sigma0 page fault handler. Given this,

a. does the handle_sigma0_page_fault only work on memory. The region I want to protect is not RAM. b. Given that the setup of sigma0's memory happens in kernel/fiasco/src/kern/kernel_thread-std.cpp, is there more to be done here? Thanks. Ramya ________________________________________ From: Masti Ramya Jayaram Sent: 10 September 2014 09:46 To: Adam Lackorzynski; l4-hackers@os.inf.tu-dresden.de Subject: RE: Physical memory allocation to L4linux Hey Adam, thanks a lot for the replies. I still have a few questions/clarifications though. :-) a. Is there a way to implement the "MODE=sigma" for linux? I could not find anything that points to it. b. Now if I were to tweak the page fault handler in sigma, then does that imply that even a corrupt moe/ned/l4linux cannot get access to the protected part of the address space? c. Who are the clients of this page fault handler? Is it just what runs on top of sigma like moe, ned, l4linux or even faisco? d. If it is just moe, ned, l4linux, then can I use the same page fault trick to ensure that (moe,ned,sigma) they always get memory from a certain dedicated region? Thanks a ton for your patience (to everyone on the list and particularly you and Martin). I totally appreciate it. Best, Ramya ________________________________________ From: l4-hackers [l4-hackers-bounces@os.inf.tu-dresden.de] on behalf of Adam Lackorzynski [adam@os.inf.tu-dresden.de] Sent: 09 September 2014 23:16 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Physical memory allocation to L4linux On Tue Sep 09, 2014 at 16:03:11 +0000, Masti Ramya Jayaram wrote:

I would like to explain my situation better. Here is what I intend:

a. What is the lowest module (bootstrap, fiasco, sigma, moe, ned,l4linux) that can be confined not access a portion of the address space?

Without changing anything and with this list of program, it's just L4Linux. Reason is that ned typically has a cap to sigma0 because it needs to give it to io. When you would like to make a little change, in Fiasco there's a function handle_sigma0_page_fault that covers page-fault by sigma0. So if you add a check on pfa there and return false you should be able to exclude a memory region from any user program.

I know that bootstrap and fiasco run in privileged mode, so there is no way to stop them. What is the next module?

b. Assuming that it is sigma, I would like to do the following: Have three regions in the physical address space (not necessarily memory):

i) one for bootstrap, fiasco ii) moe, ned, sigma, l4linux iii). Special region accessible only from (i) - bootstrap and fiasco.

If it is not possible to confine sigma, i.e., "hide a portion of the address space from it", then add sigma to lists (i,iii) and remove it from (ii) - and so on for the other modules.

If it is moe, could my goal be reached by adding an IO device as a blocker but then not give moe the corresponding capability?

In the stack moe is below any notion of an IO device, so that would not work. Adam -- Adam adam@os.inf.tu-dresden.de Lackorzynski http://os.inf.tu-dresden.de/~adam/ _______________________________________________ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers

Hi, I realized that the best place to modify sigma0's capabilities are when it is created. I see that this happens in kernel/fiasco/src/kern/kernel_thread-std.cpp in the init_workload() function. I do not exactly understand how sigma0 gathers all the capabilities - the code is a little hard to parse for me. Could someone explain this to me? Is there a way to pass only limited capabilities to sigma0 -aka, allow the default and then prune the resource tree? For now, my sigma0 output report that it has the following: SIGMA0: Dump of all resource maps RAM:------------------------ [0:10b000;13ffff] [4:140000;18cfff] [0:18d000;2ecfff] [4:2ed000;2edfff] [0:2ee000;3fffff] [0:45b000;119cfff] [4:119d000;1360fff] [0:1361000;25b98fff] IOMEM:---------------------- [0:0;fffff] [0:29000000;fedfffff] [0:fee01000;ffffffff] IO PORTS-------------------------- [0:0;fffffff] is there a way for me for example to limit RAM and IOMEM access to other spaces? Thanks, ramya ________________________________________ From: Masti Ramya Jayaram Sent: 11 September 2014 15:51 To: Adam Lackorzynski; l4-hackers@os.inf.tu-dresden.de Subject: RE: Physical memory allocation to L4linux Hey guys, I see that the IO request reaches the io_map function in kernel/fiasco/src/kern/ia32/map_util-io.cpp: L4_error io_map(Space *from, L4_fpage const &fp_from, Space *to, L4_fpage const &fp_to, Mword control) { typedef Map_traits<Io_space> Mt; Mt::Addr rcv_pos = Mt::get_addr(fp_to); Mt::Addr snd_pos = Mt::get_addr(fp_from); Mt::Size rcv_size = Mt::Size::from_shift(fp_to.order()); Mt::Size snd_size = Mt::Size::from_shift(fp_from.order()); snd_pos = snd_pos.trunc(snd_size); rcv_pos = rcv_pos.trunc(rcv_size); Mt::constraint(snd_pos, snd_size, rcv_pos, rcv_size, Mt::Addr(0)); ....... } However, when I do a "return L4_error:Map_failed" from the beginning of this function to deny all IO requests, it does not propagate back to sigma0. Could someone explain how exactly this communication/IPC between the kernel and sigma works? Thanks, Ramya ________________________________________ From: Masti Ramya Jayaram Sent: 10 September 2014 16:07 To: Adam Lackorzynski; l4-hackers@os.inf.tu-dresden.de Subject: RE: Physical memory allocation to L4linux Update: Sorry, I misunderstood a few things in the previous email. The change was in fiasco and not sigma and therefore, in theory, no user space process should have access to the protected region. This also implies that sigma0 will not have access if I understand correctly which is rather nice. So, I tried changing the handle_sigma0_page_fault function. This is located in kernel/fiasco/src/kern/ia32/thread-ia32.cpp to prevent mapping of some range of physical memory (NOT RAM) as follows: IMPLEMENT inline bool Thread::handle_sigma0_page_fault(Address pfa) { size_t size; // Check if mapping a superpage doesn't exceed the size of physical memory if (Cpu::have_superpages() // Some distributions do not allow to mmap below a certain threshold // (like 64k on Ubuntu 8.04) so we cannot map a superpage at 0 if // we're Fiasco-UX && (!Config::Is_ux || !(pfa < Config::SUPERPAGE_SIZE))) { pfa &= Config::SUPERPAGE_MASK; size = Config::SUPERPAGE_SIZE; } else { pfa &= Config::PAGE_MASK; size = Config::PAGE_SIZE; } ---------------------------------------------------------------------------------------------------------------------------------- - My change start if((pfa>=0xe0000000) && (pfa<=0xefffffff)){ printf("\nNot allowed...denying\n"); return Mem_space::Insert_err_nomem; } ---------------------------------------------------------------------------------------------------------------------------------- - My change end return mem_space()->v_insert(Mem_space::Phys_addr(pfa), Mem_space::Addr(pfa), Mem_space::Size(size), Mem_space::Page_writable | Mem_space::Page_user_accessible) != Mem_space::Insert_err_nomem; } However, I realized that when I try to map this address space using l4io_request_iomem, I see that it goes through and the call never reaches until fiasco's sigma0_page_fault handler. A little more debugging revealed that it goes as far as the following function in l4/pkg/sigma0/server/src/memmap.cc: static void map_mem(l4_fpage_t fp, Memory_type fn, l4_umword_t t, Answer *an) { an->clear(); Mem_man *m; switch (fn) { case Ram: m = Mem_man::ram(); break; case Io_mem: case Io_mem_cached: m = &iomem; break; default: return; } unsigned long addr = m->alloc(Region::bs(fp.raw & ~((1UL << 12) - 1), 1UL << l4_fpage_size(fp), t)); if (addr == ~0UL) return; /* the Fiasco kernel makes the page non-cachable if the frame * address is greater than mem_high */ an->snd_base(addr); an->snd_fpage(addr, l4_fpage_size(fp), false, fn != Io_mem); an->tag = l4_msgtag(0, 0, 1, 0); return; }

From here on, the generics make it a little hard to parse. So I am not sure where the control flow ends but it does not get to fiasco's sigma0 page fault handler. Given this,

a. does the handle_sigma0_page_fault only work on memory. The region I want to protect is not RAM. b. Given that the setup of sigma0's memory happens in kernel/fiasco/src/kern/kernel_thread-std.cpp, is there more to be done here? Thanks. Ramya ________________________________________ From: Masti Ramya Jayaram Sent: 10 September 2014 09:46 To: Adam Lackorzynski; l4-hackers@os.inf.tu-dresden.de Subject: RE: Physical memory allocation to L4linux Hey Adam, thanks a lot for the replies. I still have a few questions/clarifications though. :-) a. Is there a way to implement the "MODE=sigma" for linux? I could not find anything that points to it. b. Now if I were to tweak the page fault handler in sigma, then does that imply that even a corrupt moe/ned/l4linux cannot get access to the protected part of the address space? c. Who are the clients of this page fault handler? Is it just what runs on top of sigma like moe, ned, l4linux or even faisco? d. If it is just moe, ned, l4linux, then can I use the same page fault trick to ensure that (moe,ned,sigma) they always get memory from a certain dedicated region? Thanks a ton for your patience (to everyone on the list and particularly you and Martin). I totally appreciate it. Best, Ramya ________________________________________ From: l4-hackers [l4-hackers-bounces@os.inf.tu-dresden.de] on behalf of Adam Lackorzynski [adam@os.inf.tu-dresden.de] Sent: 09 September 2014 23:16 To: l4-hackers@os.inf.tu-dresden.de Subject: Re: Physical memory allocation to L4linux On Tue Sep 09, 2014 at 16:03:11 +0000, Masti Ramya Jayaram wrote:

I would like to explain my situation better. Here is what I intend:

a. What is the lowest module (bootstrap, fiasco, sigma, moe, ned,l4linux) that can be confined not access a portion of the address space?

Without changing anything and with this list of program, it's just L4Linux. Reason is that ned typically has a cap to sigma0 because it needs to give it to io. When you would like to make a little change, in Fiasco there's a function handle_sigma0_page_fault that covers page-fault by sigma0. So if you add a check on pfa there and return false you should be able to exclude a memory region from any user program.

I know that bootstrap and fiasco run in privileged mode, so there is no way to stop them. What is the next module?

b. Assuming that it is sigma, I would like to do the following: Have three regions in the physical address space (not necessarily memory):

i) one for bootstrap, fiasco ii) moe, ned, sigma, l4linux iii). Special region accessible only from (i) - bootstrap and fiasco.

If it is not possible to confine sigma, i.e., "hide a portion of the address space from it", then add sigma to lists (i,iii) and remove it from (ii) - and so on for the other modules.

If it is moe, could my goal be reached by adding an IO device as a blocker but then not give moe the corresponding capability?

In the stack moe is below any notion of an IO device, so that would not work. Adam -- Adam adam@os.inf.tu-dresden.de Lackorzynski http://os.inf.tu-dresden.de/~adam/ _______________________________________________ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers

Hey Adam, Thanks for the input. I will try changing one of the suggested functions. It would be great out of curiosity if you could elucidate why the following occurs. I see that there is a new iomem region being created and this happens only when I try to read from the mapped region - not until then. But none of this reaches the kernel or maybe I am not sure where to look.. In fact, I see that the init_mem file parses the kernel memory descriptors to find the regions. So under what circumstances does sigma0 go back to the kernel for mem/io? Thanks a lot! Ramya

On 15 Sep 2014, at 00:38, "Adam Lackorzynski" <adam@os.inf.tu-dresden.de> wrote:

...

On Fri Sep 12, 2014 at 16:34:17 +0000, Masti Ramya Jayaram wrote: I realized that the best place to modify sigma0's capabilities are when it is created. I see that this happens in kernel/fiasco/src/kern/kernel_thread-std.cpp in the init_workload() function. I do not exactly understand how sigma0 gathers all the capabilities - the code is a little hard to parse for me. Could someone explain this to me? Is there a way to pass only limited capabilities to sigma0 -aka, allow the default and then prune the resource tree?

So, I've changed my mind, let's not modify the kernel but sigma0. This seems easier to me. As sigma0 is the root of memory this should not be a problem and can be argued. Sigma0 handles memory at first-come-first-serve approach and only gives it out once, so allocating it before servicing client requests should hide it from clients (see memmap.cc). However, there's also an unmap call which would need to be handled to not free it again. Another approach could be to exclude the memory region when initializing the internal data structures in init_mem.cc. Whatever works/fits better.

Adam -- Adam adam@os.inf.tu-dresden.de Lackorzynski http://os.inf.tu-dresden.de/~adam/

_______________________________________________ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers

Hey Adam, Yes I can confirm that I was a able to trace the code from fpage_map() to map() but I see zero addresses there - i.e. When I check the receiver and sender addresses and size everything is zero. This was despite it being correct until the handle_sigma0_request() function in mem_map.cc. So I just assumed I was looking in the wrong place. I will try to dig a little more and check. Sent from my phone

On 15 Sep 2014, at 23:55, "Adam Lackorzynski" <adam@os.inf.tu-dresden.de> wrote:

Hi,

...

On Mon Sep 15, 2014 at 18:25:37 +0000, Masti Ramya Jayaram wrote: Thanks for the input. I will try changing one of the suggested functions.

It would be great out of curiosity if you could elucidate why the following occurs.

I see that there is a new iomem region being created and this happens only when I try to read from the mapped region - not until then. But none of this reaches the kernel or maybe I am not sure where to look..

Well, it should reach the kernel. For sigma0 mappings appear out of nothing (it's the root), so you should at least see it in the mapping loop in map_util.cpp's map() function (also see the comments on sigma0 there).

...

In fact, I see that the init_mem file parses the kernel memory descriptors to find the regions. So under what circumstances does sigma0 go back to the kernel for mem/io?

In this case there a fpage is created that is used in the reply to the client which will then create the mapping.

Adam -- Adam adam@os.inf.tu-dresden.de Lackorzynski http://os.inf.tu-dresden.de/~adam/

_______________________________________________ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers