Hello,
writing a L4Linux Driver I'm expiring the C-Interface and low level(?) flexpages.
So far, I can push a Dataspace Capability via flexpage, and the receiving server is able to read its content. But, whatever the server writes in that mem-region doesn't appear on the clients side.
/* client.c */
l4_fpage_t fpage = l4_obj_fpage( dataspace, 0, L4_CAP_FPAGE_RW ); l4_utcb_mr()->mr[0] = size; l4_utcb_br()->br[0] = fpage.raw;
/* server.c */
l4_utcb_br()->br[0] = dataspace | L4_RCV_ITEM_SINGLE_CAP;
l4_ipc_wait(l4_utcb(), 0, L4_IPC_NEVER);
// allocate, and attach Dataspace ... // read & write Dataspace ... l4_utcb_mr()->mr[0] = returnValue; l4_ipc_reply_and_wait(l4_utcb(), l4_msgtag(0, 1, 0, 0), 0, L4_IPC_NEVER);
As I am creating the fpage with L4_CAP_FPAGE_RW my guess was I can write to that mem-region and the client can read my changes.
Anyway, maybe I need some more understandings about capabilities & flexpages. (Are there any precise slides or papers about this topic?) So, the client creates a dataspace capability and allocates some memory. But, what happens when the flexpage is pushed to the server? The capability is copied and both pointing to the same allocated memory, or is the Cap and(!) the memory it is pointing to copied to server?
Thanks, ba_f
Hi,
On Tue May 10, 2016 at 16:44:35 +0200, ba_f wrote:
writing a L4Linux Driver I'm expiring the C-Interface and low level(?) flexpages.
So far, I can push a Dataspace Capability via flexpage, and the receiving server is able to read its content. But, whatever the server writes in that mem-region doesn't appear on the clients side.
That sounds pretty confusing to me. When you read the proper contents that would mean you're writing to the same physical memory location, so given you do not read from a copy and the write is successful, the update should be visible in the memory.
/* client.c */
l4_fpage_t fpage = l4_obj_fpage( dataspace, 0, L4_CAP_FPAGE_RW ); l4_utcb_mr()->mr[0] = size; l4_utcb_br()->br[0] = fpage.raw;
That is more like a setup for a receive. Please have a look at some calls in l4sys, such as l4_scheduler_run_thread or l4_task_map which send caps to the callee. Note that the l4_map_obj_control+l4_obj_fpage pair(s) must be at the end of the MR-array. And do not forget proper values for the sending l4_msgtag.
/* server.c */
l4_utcb_br()->br[0] = dataspace | L4_RCV_ITEM_SINGLE_CAP;
l4_ipc_wait(l4_utcb(), 0, L4_IPC_NEVER);
// allocate, and attach Dataspace ... // read & write Dataspace ... l4_utcb_mr()->mr[0] = returnValue; l4_ipc_reply_and_wait(l4_utcb(), l4_msgtag(0, 1, 0, 0), 0, L4_IPC_NEVER);
As I am creating the fpage with L4_CAP_FPAGE_RW my guess was I can write to that mem-region and the client can read my changes.
When you could not write, you'd get a page-fault and your program would stop.
Anyway, maybe I need some more understandings about capabilities & flexpages. (Are there any precise slides or papers about this topic?) So, the client creates a dataspace capability and allocates some memory. But, what happens when the flexpage is pushed to the server? The capability is copied and both pointing to the same allocated memory, or is the Cap and(!) the memory it is pointing to copied to server?
When you transfer a capability to a dataspace, you just transfer the access rights to that particular dataspace to the other party. Not the memory. After the transfer, both can talk to the very same dataspace, map it and use it. Typically when sharing dataspaces that is called shared memory.
Adam
Hi,
Am 2016-05-12 00:20, schrieb Adam Lackorzynski:
Hi,
/* client.c */
l4_fpage_t fpage = l4_obj_fpage( dataspace, 0, L4_CAP_FPAGE_RW ); l4_utcb_mr()->mr[0] = size; l4_utcb_br()->br[0] = fpage.raw;
That is more like a setup for a receive. Please have a look at some calls in l4sys, such as l4_scheduler_run_thread or l4_task_map which send caps to the callee. Note that the l4_map_obj_control+l4_obj_fpage pair(s) must be at the end of the MR-array. And do not forget proper values for the sending l4_msgtag.
This is what my trial(!) looks now:
/* client.c */
l4_utcb_mr()->mr[0] = size; l4_utcb_mr()->mr[1] = l4_map_obj_control(dataspace, L4_ITEM_CONT); l4_utcb_mr()->mr[2] = l4_obj_fpage( dataspace, 0, L4_CAP_FPAGE_RW ).raw;
/* server.c */
l4_utcb_br()->bdr = 0; // l4_utcb_br()->br[0] = l4_map_obj_control(dataspace, L4_RCV_ITEM_SINGLE_CAP); l4_utcb_br()->br[0] = L4_ITEM_MAP; l4_utcb_br()->br[1] = l4_obj_fpage(dataspace, 0, L4_CAP_FPAGE_RW).raw;
tag = l4_ipc_wait(l4_utcb(), &label, L4_IPC_NEVER); while(1){ r = l4_ipc_error(tag, l4_utcb()); if ( r) { fprintf(stderr, "IPC error: %x\n", r); tag = l4_ipc_wait(l4_utcb(), &label, L4_IPC_NEVER); continue; } ... tag = l4_ipc_reply_and_wait(l4_utcb(), l4_msgtag(0, 1, 0, 0), &label, L4_IPC_NEVER);
This gives me "IPC error: 9" when doing l4_ipc_reply_and_wait(). I don't receive a dataspace from client, neither.
Anyway, maybe I need some more understandings about capabilities & flexpages. (Are there any precise slides or papers about this topic?) So, the client creates a dataspace capability and allocates some memory. But, what happens when the flexpage is pushed to the server? The capability is copied and both pointing to the same allocated memory, or is the Cap and(!) the memory it is pointing to copied to server?
When you transfer a capability to a dataspace, you just transfer the access rights to that particular dataspace to the other party. Not the memory. After the transfer, both can talk to the very same dataspace, map it and use it. Typically when sharing dataspaces that is called shared memory.
Adam
Actually, I would like to understand what's going on. I know every thread has its UTCBs and an IPC_CALL just copies the MRs from caller to callee. But, flexpages seem to be a special case. I guess, the BRs are for receiving flexpages. And there must be some kind of parsing of the MRs since l4_map_obj_control+l4_obj_fpage pair(s) are recognized. And somehow the kernel must find the matching BRs. Where does all this stuff happen?
Thanks,
ba_f
Hi,
On Thu May 12, 2016 at 16:09:47 +0200, ba_f wrote:
Am 2016-05-12 00:20, schrieb Adam Lackorzynski:
Hi,
/* client.c */
l4_fpage_t fpage = l4_obj_fpage( dataspace, 0, L4_CAP_FPAGE_RW ); l4_utcb_mr()->mr[0] = size; l4_utcb_br()->br[0] = fpage.raw;
That is more like a setup for a receive. Please have a look at some calls in l4sys, such as l4_scheduler_run_thread or l4_task_map which send caps to the callee. Note that the l4_map_obj_control+l4_obj_fpage pair(s) must be at the end of the MR-array. And do not forget proper values for the sending l4_msgtag.
This is what my trial(!) looks now:
/* client.c */
l4_utcb_mr()->mr[0] = size; l4_utcb_mr()->mr[1] = l4_map_obj_control(dataspace, L4_ITEM_CONT); l4_utcb_mr()->mr[2] = l4_obj_fpage( dataspace, 0, L4_CAP_FPAGE_RW ).raw;
Referring to the functions I pointed you to, this should look like this:
l4_utcb_mr()->mr[0] = size; l4_utcb_mr()->mr[1] = l4_map_obj_control(0, 0); l4_utcb_mr()->mr[2] = l4_obj_fpage(dataspace, 0, L4_CAP_FPAGE_RWX).raw;
ret = l4_ipc_call(server, l4_utcb(), l4_msgtag(your_proto, 1, 1, 0), L4_IPC_NEVER)
/* server.c */
l4_utcb_br()->bdr = 0; // l4_utcb_br()->br[0] = l4_map_obj_control(dataspace, L4_RCV_ITEM_SINGLE_CAP); l4_utcb_br()->br[0] = L4_ITEM_MAP; l4_utcb_br()->br[1] = l4_obj_fpage(dataspace, 0, L4_CAP_FPAGE_RW).raw;
I think this should look like this: l4_utcb_br()->br[0] = dataspace.cap() | L4_RCV_ITEM_SINGLE_CAP; l4_utcb_br()->bdr = 0;
tag = l4_ipc_wait(l4_utcb(), &label, L4_IPC_NEVER); while(1){ r = l4_ipc_error(tag, l4_utcb()); if ( r) { fprintf(stderr, "IPC error: %x\n", r); tag = l4_ipc_wait(l4_utcb(), &label, L4_IPC_NEVER); continue; } ... tag = l4_ipc_reply_and_wait(l4_utcb(), l4_msgtag(0, 1, 0, 0), &label, L4_IPC_NEVER);
This gives me "IPC error: 9" when doing l4_ipc_reply_and_wait(). I don't receive a dataspace from client, neither.
9 is L4_IPC_REMSGCUT: Cut receive message, due to message buffer is too small.
Which tells us that the receive side did not provide enough receive capacity (aka where to put the receiving cap).
Anyway, maybe I need some more understandings about capabilities & flexpages. (Are there any precise slides or papers about this topic?) So, the client creates a dataspace capability and allocates some memory. But, what happens when the flexpage is pushed to the server? The capability is copied and both pointing to the same allocated memory, or is the Cap and(!) the memory it is pointing to copied to server?
When you transfer a capability to a dataspace, you just transfer the access rights to that particular dataspace to the other party. Not the memory. After the transfer, both can talk to the very same dataspace, map it and use it. Typically when sharing dataspaces that is called shared memory.
Adam
Actually, I would like to understand what's going on. I know every thread has its UTCBs and an IPC_CALL just copies the MRs from caller to callee.
Yes...
But, flexpages seem to be a special case.
... they are. The difference is described in the l4_msgtag, the second param describes the number of MRs to be transmitted as pure data, the third one describes how many items are there, such as flexpages.
I guess, the BRs are for receiving flexpages. And there must be some kind of parsing of the MRs since l4_map_obj_control+l4_obj_fpage pair(s) are recognized.
Both right.
And somehow the kernel must find the matching BRs.
That's right. The bdr = 0 points the kernel to the br field index.
Where does all this stuff happen?
In the kernel :) kern/thread-ipc.cpp is probably a good place to start exploring this, e.g. in transfer_msg_items()
Adam
Nope,
still the same ERR=9.
I think this should look like this: l4_utcb_br()->br[0] = dataspace.cap() | L4_RCV_ITEM_SINGLE_CAP; l4_utcb_br()->bdr = 0;
I think the cap() function is C++ only so left this out. Guess this ok since 'typedef l4_cap_idx_t l4re_ds_t;'
/* server.c */
l4re_ds_t dataspace = l4re_util_cap_alloc(); if (l4_is_invalid_cap( dataspace)) return 1;
l4_utcb_br()->br[0] = dataspace | L4_RCV_ITEM_SINGLE_CAP; l4_utcb_br()->bdr = 0;
Greets,
ba_f
Am 2016-05-12 23:56, schrieb Adam Lackorzynski:
Hi,
On Thu May 12, 2016 at 16:09:47 +0200, ba_f wrote:
Am 2016-05-12 00:20, schrieb Adam Lackorzynski:
Hi,
/* client.c */
l4_fpage_t fpage = l4_obj_fpage( dataspace, 0, L4_CAP_FPAGE_RW ); l4_utcb_mr()->mr[0] = size; l4_utcb_br()->br[0] = fpage.raw;
That is more like a setup for a receive. Please have a look at some calls in l4sys, such as l4_scheduler_run_thread or l4_task_map which send caps to the callee. Note that the l4_map_obj_control+l4_obj_fpage pair(s) must be at the end of the MR-array. And do not forget proper values for the sending l4_msgtag.
This is what my trial(!) looks now:
/* client.c */
l4_utcb_mr()->mr[0] = size; l4_utcb_mr()->mr[1] = l4_map_obj_control(dataspace, L4_ITEM_CONT); l4_utcb_mr()->mr[2] = l4_obj_fpage( dataspace, 0, L4_CAP_FPAGE_RW ).raw;
Referring to the functions I pointed you to, this should look like this:
l4_utcb_mr()->mr[0] = size; l4_utcb_mr()->mr[1] = l4_map_obj_control(0, 0); l4_utcb_mr()->mr[2] = l4_obj_fpage(dataspace, 0, L4_CAP_FPAGE_RWX).raw;
ret = l4_ipc_call(server, l4_utcb(), l4_msgtag(your_proto, 1, 1, 0), L4_IPC_NEVER)
/* server.c */
l4_utcb_br()->bdr = 0; // l4_utcb_br()->br[0] = l4_map_obj_control(dataspace, L4_RCV_ITEM_SINGLE_CAP); l4_utcb_br()->br[0] = L4_ITEM_MAP; l4_utcb_br()->br[1] = l4_obj_fpage(dataspace, 0, L4_CAP_FPAGE_RW).raw;
I think this should look like this: l4_utcb_br()->br[0] = dataspace.cap() | L4_RCV_ITEM_SINGLE_CAP; l4_utcb_br()->bdr = 0;
tag = l4_ipc_wait(l4_utcb(), &label, L4_IPC_NEVER); while(1){ r = l4_ipc_error(tag, l4_utcb()); if ( r) { fprintf(stderr, "IPC error: %x\n", r); tag = l4_ipc_wait(l4_utcb(), &label, L4_IPC_NEVER); continue; } ... tag = l4_ipc_reply_and_wait(l4_utcb(), l4_msgtag(0, 1, 0, 0), &label, L4_IPC_NEVER);
This gives me "IPC error: 9" when doing l4_ipc_reply_and_wait(). I don't receive a dataspace from client, neither.
9 is L4_IPC_REMSGCUT: Cut receive message, due to message buffer is too small.
Which tells us that the receive side did not provide enough receive capacity (aka where to put the receiving cap).
Actually, I would like to understand what's going on. I know every thread has its UTCBs and an IPC_CALL just copies the MRs from caller to callee.
Yes...
But, flexpages seem to be a special case.
... they are. The difference is described in the l4_msgtag, the second param describes the number of MRs to be transmitted as pure data, the third one describes how many items are there, such as flexpages.
I guess, the BRs are for receiving flexpages. And there must be some kind of parsing of the MRs since l4_map_obj_control+l4_obj_fpage pair(s) are recognized.
Both right.
And somehow the kernel must find the matching BRs.
That's right. The bdr = 0 points the kernel to the br field index.
Where does all this stuff happen?
In the kernel :) kern/thread-ipc.cpp is probably a good place to start exploring this, e.g. in transfer_msg_items()
Adam
On Fri May 13, 2016 at 19:57:20 +0200, ba_f wrote:
Nope, still the same ERR=9.
I think this should look like this: l4_utcb_br()->br[0] = dataspace.cap() | L4_RCV_ITEM_SINGLE_CAP; l4_utcb_br()->bdr = 0;
I think the cap() function is C++ only so left this out. Guess this ok since 'typedef l4_cap_idx_t l4re_ds_t;'
/* server.c */
l4re_ds_t dataspace = l4re_util_cap_alloc(); if (l4_is_invalid_cap( dataspace)) return 1;
l4_utcb_br()->br[0] = dataspace | L4_RCV_ITEM_SINGLE_CAP; l4_utcb_br()->bdr = 0;
Any progress? Hard to tell from just the snippets and without building a test case. Did you find any difference to existing code?
Adam
l4-hackers@os.inf.tu-dresden.de