Hi Paul,
On 2/9/21 1:20 AM, Paul Boddie wrote:
On Monday, 8 February 2021 09:54:08 CET Christian Ludwig wrote:
On Sunday, 07.02.2021, 01:54 +0100 Paul Boddie wrote:
What considerations need to be given to concurrency with regard to the use of the l4re_rm_attach function and similar functionality? Is access to the region mapper/manager from different threads in the same task advisable, are there thread safety issues, and if so, can these be sensibly worked around?
Calls to the region manager are IPC calls and are synchronized by a wait-queue in the kernel. The region manager thread only serves one client at a time. In fact if you use multiple threads in a task it serves all page-faults for all threads already. It's thread-safe by design.
OK. I wasn't really sure about the concurrency aspects of the region manager itself. Some frustrating experience with my own code led me to question this rather than just take it for granted.
However, the capability slot allocator (l4re_util_cap_alloc) is not thread-safe. You need to introduce some synchronization if you use it from multiple threads.
This is very useful to know. I imagine that thread safety is regarded as unnecessary for many use-cases, but it does surprise me that it has to be introduced to make the allocator usable in a multithreaded program. Then again, I might be doing things that are not particularly normal for L4Re programs.
I agree that an out-of-the-box mechanism is helpful here and will prevent headaches in many situations, plus you wouldn't need to know about this. However, the (internal) discussion did not yet reach consensus.
Once you know about this shortcoming, a thread-safety wrapper around cap_alloc is fortunately straight forward.
What kind of resource limits apply to L4Re tasks? In my work, I am allocating capabilities for objects acting as dataspaces. Things seem to work fairly well up to a certain point, and then capability allocation seems to fail, and program failure is then not particularly graceful.
(I currently observe that creating 160 threads, each with its own IPC gate capability seems to be unproblematic, but more than this will produce errors that manifest themselves either in pthread library code or in my own code where a critical section is supposed to be enforced.)
It looks like there is a capability limit per task at 4096 for L4Re::Util::cap_alloc. 160 thread (2 caps) + 160 IPC gates = 3*160 -> 480 caps
If you are handing out more than 3600 Dataspaces you might be running into this limit. Now I'm fuzzy on the details, so I'll just provide a general direction. I don't know if you can replenish the Util::cap_alloc by providing additional memory to it or if you need to setup your own allocator (see l4re-core/l4re/util/libs/cap_alloc.cc). For the latter, you can set it up with a bigger piece of memory, to increase the number of manage capability slots.
On 2/7/21 1:54 AM, Paul Boddie wrote:
I might also ask about support in L4Re for C++ threading constructs. When developing my simulation, I decided to use the standard C++ support for threading, mutexes, condition variables, and so on, as opposed to using pthread functionality. I imagine that C++ library support for these things just wraps the pthread functionality, but I wonder if there are not other considerations that I might need to be aware of.
You are right, std::thread wraps libpthread. I guess you know about l4re-core/libstdc++-headers/include/thread-l4?
To my knowledge the behavior for the C++ constructs don't differ from the pthread ones. Be careful with std::thread.join(), the thread needs to cooperate and terminate/exit by itself. If it doesn't, your caller waits for a long time.
If you directly interact with the UTCB or use low-level functions which setup the UTCB, make sure to not do any other function calls, as they might change the UTCB. (That's also a high scorer in the headache list.)
Cheers, Philipp