Christian Helmuth wrote:
Hello David,
On Tue, Aug 01, 2006 at 04:55:53AM -0700, David Silcott wrote:
I've ported my multithreaded tcp server to L4 using flips but I am having some problems getting it to run properly. It is a quite a bit to explain so I've written a simple tcp echoserver to demonstrate my problem.
Ouch, already at this point I remember some limitations of the FLIPS experiment and L4VFS. FLIPS supports concurrent blocking RPCs, e.g. send() and recv(), only for calls using different L4VFS connections, see l4vfs_init_connection(). If one uses the same connection for two blocking calls the latter one is blocked in the underlying IPC in the microkernel.
L4VFS currently implements no extensive multi-threading support and therefore initializes only one connection to FLIPS per task, see l4/pkg/l4vfs/lib/libc_backends/socket_io/socket.c line 44 ff.
L4VFS *does* support multithreading applications, for example with the term server. Therefore many functions in the IDL interfaces have this [allow_reply_only] flag set which basically allows the server to not answer calls directly but delayed with the help of a worker thread. In between the session thread in the server is ready to receive new calls.
Grep in l4vfs/term_server for "DICE_NO_REPLY" for examples. The problem here seems to be that the current implementation of flips does not utilize this (there are no worker threads, right?).
Even transparently supporting one session per client thread by L4VFS would not always solve the problem always, as clients may use a user-level thread package on top of L4-threads (e.g. Java) or might interrupt threads with ex-regs, which could result in even one thread calling several blocking operations at once. The problem must be solved in the servers.
The following happens when the server starts and the client connects and send data.
- Echoserver starts and listens on port 2222
- client connects, server creates a child_server
thread to service client and starts listening again 3. client sends string to child_server thread which is received. child_server however is unable to complete send(echo), it seems to be blocked at this point.
I guess your server blocks in accept() and the child_server blocks during its "send()" IPC in the kernel.
- The send is only completed when another client
attempts a connect(..and the server moves on from the accept(..) statement).
Now the thread in FLIPS replies the accept() request and enters an ipc_wait(). The child_server "send()"-IPC is unblocked and the packet is echoed.
To fix this issue you should start in the before-mentioned socket.c file and init L4VFS connections to FLIPS per thread.
You could fix it this way but it would be an intransparent hack (which might be perfectly ok for your situation). You could also enhance flips with worker threads.
If you fix the problem either way please send patches back!
Cheers, Martin Pohlack