l4vfs and stdin/stdout handles
Hi all. I am trying to make a test using l4vfs libc backends so the three predefined file descriptors 0, 1, 2 work for stdin, stdout and stderr correspondingly. It appears that these file descriptors are already predefined for some purpose: when I am trying to open files like this: // open initial file descriptors // stdin fd = open("/vc/vc0", O_RDONLY); LOG("stdin: fd=%d, errno=%d", fd, errno); // strout fd = open("/vc/vc0", O_WRONLY); LOG("stdout: fd=%d, errno=%d", fd, errno); // stderr fd = open("/vc/vc0", O_WRONLY); LOG("stderr: fd=%d, errno=%d", fd, errno); -- like in term_con_test example for term_con server, the 3, 4, 5 descriptors are returned from open(). If I not use term_con at all, and not open vc0 three times as above, read(0, ....); and write(1,....); works but strangely. For example, read() returns immediately with no error and returns a single symbol (EOF, I think). But as I tested in Linux and OS/2 operating systems, when stdin is a console, read(0,...); must block until '\n' is returned (user pressed an Enter key). So, it seems that the file descriptors 0, 1, 2 are reserved, but they are not working as expected. Maybe, my setup is incorrect or something is not yet implemented? I have a menu.lst entry like this: title The "Hello, world!" program kernel $(A)/bootstrap -serial modaddr 0x02000000 module $(B)/fiasco $(FIASCOARGS) module $(B)/sigma0 module $(B)/roottask \ task modname 'simple_file_server' attached 6 modules module $(B)/log module $(B)/names module $(B)/dm_phys module $(B)/l4io module $(B)/rtc module $(B)/l4con module $(B)/simple_ts -t 300 module $(B)/name_server module $(B)/term_con module $(B)/simple_file_server -v 14 module $(B)/os2/config.sys module $(B)/os2/mini33.exe module $(B)/os2/minicmd.exe module $(B)/os2/msg.dll module $(B)/os2/doscalls.dll module $(B)/os2/sub32.dll module $(B)/fstab \ -c /file \ -c /file/system -v 14 -b / -m /file/system \ -c /vc -v 132 -b / -m /vc module $(B)/os2/os2server -d c: -m /file/system vbeset $(VBE_MODE) I needed to relink a term_con server to an address 0x01450000 instead of the default one of 0x01400000 because the latter was the same as with simple_ts server (0x01400000 too). I was not able to use loader server to load at other address because I needed to launch two l4vfs servers, term_con and simple_file_server where simple_file_server serves all files through fprov_proxy file provider (I don't want to use an extra bmodfs server to serve files for loader -- it would be an ugly decision, I think). The 'os2server' server uses the files that were served by simple_file_server and it tries to use l4vfs servers for reading from stdin. My makefile looks like this: PKGDIR ?= ../.. L4DIR ?= $(PKGDIR)/../.. TARGET = os2server # the default relocation address. This may be superseded by a STATIC file. DEFAULT_RELOC = 0x01800000 # list your .c files here # os2server.c SRC_C = main.c utility.c io.c MountReg.c globals.c native_dynlink.c l4_alloc_mem.c \ Shared/token.c \ Shared/modmgr.c \ Shared/ixfmgr.c \ Shared/ixfmgr_lx.c \ Shared/ixfmgr_lx_load.c \ Shared/ixfmgr_ne.c \ Shared/ixfmgr_lx_fixuplx.c \ Shared/ixfmgr_lx_loadobjlx.c \ Shared/ixfmgr_lx_modlx.c \ Shared/ixfmgr_lx_execlx.c \ Shared/ixfmgr_lx_debuglx.c \ Shared/memmgr.c \ Shared/api/apistub.c \ Shared/cfgparser.c \ Shared/processmgr.c #Shared/ow_dlfcn.c \ # if your server implements the server side of an idl defined in an idl-file # of your package, list the idl file name(s) here (no path needed) SERVERIDL = os2server.idl # list additional library paths and libraries here LIBS = -los2server -lparsecmdline -lthread -lcon -lcontxt -lconstream-server CFLAGS = -I$(PKGDIR)/include MODE=l4env_base include $(L4DIR)/mk/prog.mk Maybe some libs are interfere with l4vfs ones or something like this? I suspect the l4con server libs interfere with l4vfs libs. For example, l4con has its own getchar() implementation but in uCLibc this function is implemented through l4vfs backends. When I am trying to use getchar() or scanf() functions, I see that read() function is called behind the scenes, and it don't get blocked until Enter is pressed. So, any help is appreciated. WBR, valery
Hi Valery, huh, many questions :-). On 20.06.2010 12:08, Valery V. Sedletski wrote:
I am trying to make a test using l4vfs libc backends so the three predefined file descriptors 0, 1, 2 work for stdin, stdout and stderr correspondingly. It appears that these file descriptors are already predefined for some purpose:
Depending on the IO backend used the 3 standard file descriptors are already opened for you.
when I am trying to open files like this:
// open initial file descriptors // stdin fd = open("/vc/vc0", O_RDONLY); LOG("stdin: fd=%d, errno=%d", fd, errno); // strout fd = open("/vc/vc0", O_WRONLY); LOG("stdout: fd=%d, errno=%d", fd, errno); // stderr fd = open("/vc/vc0", O_WRONLY); LOG("stderr: fd=%d, errno=%d", fd, errno);
-- like in term_con_test example for term_con server, the 3, 4, 5 descriptors are returned from open().
This is what you expect in an posix environment, where your shell passes these on to you on fork. As there is no real fork support in l4vfs, I implemented special parameters which are evaluated and eaten before main() is executed in pkg/l4vfs/lib/libc_backends/io/operations.c:init_io(). These parameters trigger the open to the VCs for you (--stdin, --stdout, --stderr).
If I not use term_con at all, and not open vc0 three times as above, read(0, ....); and write(1,....); works but strangely. For example, read() returns immediately with no error and returns a single symbol (EOF, I think). But as I tested in Linux and OS/2 operating systems, when stdin is a console, read(0,...); must block until '\n' is returned (user pressed an Enter key).
Not sure what happens here, IIRC, dietlibc behaved a bit strange when printf etc. was used on invalid FDs. These days you will probably be using uclibc, right? Maybe you are using the minimal_io backend, this may just ignore reads.
So, it seems that the file descriptors 0, 1, 2 are reserved, but they are not working as expected. Maybe, my setup is incorrect or something is not yet implemented?
It is quite likely that something is not implemented here. For example, if you use the minimal_io backend, only write to 1 and 2 is implemented. There is some documentation, for example, in pkg/dietlibc/lib/backends/README. I haven't touched the source in the last two years, maybe others have a more current view on things? HTH, Martin
On Sun, 20 Jun 2010 14:19:29 +0200, Martin Pohlack wrote:
-- like in term_con_test example for term_con server, the 3, 4, 5 descriptors are returned from open().
As there is no real fork support in l4vfs, I implemented special parameters which are evaluated and eaten before main() is executed in pkg/l4vfs/lib/libc_backends/io/operations.c:init_io(). These parameters trigger the open to the VCs for you (--stdin, --stdout, --stderr).
I checked init_io(), it seems to do the same as a fragment in term_con_test example -- it opens a terminal device and checks the file descriptors to be 0, 1, 2 (with unessential details). I forgot to mention that my program is an API server which implements an API and starts a client program using that API. I load the client program, all needed libraries for it. Then I start a function called trampoline() in a separate task, prepare an environment for the program in that function (including file descriptors). Then I just setup registers, push params for C startup code on stack, and call an entry point in a client program. I tried to add "--stdin /dev/vc0" to API server's command line. The init_io() function opens all file descriptors successfully. Then these file descriptors seem to be inherited by a client program (if I try to open /dev/vc0 in trampoline(), it returns a fd equal to 3, 4, 5, so 0, 1, 2 are already used). But when I try calling read(0, ...); (I call it in the API call from the client program, so the task id at the moment is from the client program) it says that os2serve| api_DosRead(): threadid: 11.0 os2serve| read(): fd = '0', buf = '0x0001afd0', count = '1'' term_con| l4vfs_common_io_read_component(): invalid read request - not owning o term_con: bject and so on many times. I think it is because of the task number different from the API server one. (file descriptors are owned by server, not a client program). Also, I tried to close all file descriptors (0, 1, 2) before opening them at the start of trampoline(). It fails: os2serve| close(): local fd '0' os2serve| close(): server ret os2serve| close(): Error in close, unknown case! os2serve| close(): local fd '1' os2serve| close(): server ret os2serve| close(): Error in close, unknown case! os2serve| close(): local fd '2' os2serve| l4_exec_lx(): l4ts_create_task() returned: 0, taskid=11.0 os2serve| close(): server ret os2serve| close(): Error in close, unknown case! No idea, why the LOG tag is 'os2server' -- it executes in the separate task. I also tried to add init_io() call at the start of trampoline() but have no idea how to pass it the command line with --stdin, --stdout, --stderr arguments -- when a program is started via roottask, it gets its command line from GRUB command line passed through bootinfo structure. But what to do if I start task via task server?
If I not use term_con at all, and not open vc0 three times as above, read(0, ....); and write(1,....); works but strangely. For example, read() returns immediately with no error and returns a single symbol (EOF, I think). But as I tested in Linux and OS/2 operating systems, when stdin is a console, read(0,...); must block until '\n' is returned (user pressed an Enter key).
Not sure what happens here, IIRC, dietlibc behaved a bit strange when printf etc. was used on invalid FDs. These days you will probably be using uclibc, right? Maybe you are using the minimal_io backend, this may just ignore reads.
Yes, I use uCLibc. But minimal_io is not used here. I checked the linker line, it states that it links with c_be_io.o.a.
I haven't touched the source in the last two years, maybe others have a more current view on things?
Yes, I think l4vfs is a promising area. I plan trying to write a filesystem support layer for OS/2 IFS API, probably as an l4vfs server with libraries-plugins implementing different filesystems. At the first stage, it will be a ramdisk with ramfs format. Also, probably, a configuration namespace (registry) like in IBM's WorkplaceOS, could be implemented using l4vfs. WBR, valery
On 21.06.2010 00:58, Valery V. Sedletski wrote:
On Sun, 20 Jun 2010 14:19:29 +0200, Martin Pohlack wrote:
-- like in term_con_test example for term_con server, the 3, 4, 5 descriptors are returned from open().
As there is no real fork support in l4vfs, I implemented special parameters which are evaluated and eaten before main() is executed in pkg/l4vfs/lib/libc_backends/io/operations.c:init_io(). These parameters trigger the open to the VCs for you (--stdin, --stdout, --stderr).
I checked init_io(), it seems to do the same as a fragment in term_con_test example -- it opens a terminal device and checks the file descriptors to be 0, 1, 2 (with unessential details). I forgot to mention that my program is an API server which implements an API and starts a client program using that API. I load the client program, all needed libraries for it. Then I start a function called trampoline() in a separate task, prepare an environment for the program in that function (including file descriptors). Then I just setup registers, push params for C startup code on stack, and call an entry point in a client program.
I think it is not so easy to transfer these FDs into another task, as server state is associated with many FDs (depending on the server).
I tried to add "--stdin /dev/vc0" to API server's command line. The init_io() function opens all file descriptors successfully. Then these file descriptors seem to be inherited by a client program (if I try to open /dev/vc0 in trampoline(), it returns a fd equal to 3, 4, 5, so 0, 1, 2 are already used). But when I try calling read(0, ...); (I call it in the API call from the client program, so the task id at the moment is from the client program) it says that
I looked again in operations.c:init_io() If you do not provide "--stdin x" etc. argument, dummy FDs are put into place in the local FD table, e.g., here: ft_fill_entry(1, file_desc); // fill in dummy entry I vaguely recall doing this to prevent accidents with printf() etc. in dietlibc which had no error checking regarding invalid FDs. I have not revisited this code since we switched to uclibc years ago ...
os2serve| api_DosRead(): threadid: 11.0 os2serve| read(): fd = '0', buf = '0x0001afd0', count = '1'' term_con| l4vfs_common_io_read_component(): invalid read request - not owning o term_con: bject
and so on many times. I think it is because of the task number different from the API server one. (file descriptors are owned by server, not a client program).
Exactly. This is the server state I mentioned above. If you want to have nice support for fork etc. (which requires FD inheritance amongst other things) you will need to extend l4vfs to inform all relevant servers of resource transfers. This may be initiated from the client program, but some generic functions in servers to support transferring resource seems in order. This may prove to be a challenging design if you want to do it securely though.
Also, I tried to close all file descriptors (0, 1, 2) before opening them at the start of trampoline(). It fails:
os2serve| close(): local fd '0' os2serve| close(): server ret os2serve| close(): Error in close, unknown case! os2serve| close(): local fd '1' os2serve| close(): server ret os2serve| close(): Error in close, unknown case! os2serve| close(): local fd '2' os2serve| l4_exec_lx(): l4ts_create_task() returned: 0, taskid=11.0 os2serve| close(): server ret os2serve| close(): Error in close, unknown case!
No idea, why the LOG tag is 'os2server' -- it executes in the separate task.
I also tried to add init_io() call at the start of trampoline() but have no idea how to pass it the command line with --stdin, --stdout, --stderr arguments -- when a program is started via roottask, it gets its command line from GRUB command line passed through bootinfo structure. But what to do if I start task via task server?
Hmm, I think the init_io() function should be called as a constructor, you should not call it manually. Have you checked out the loader? I think you may not want to use the task server directly. http://os.inf.tu-dresden.de/l4env/doc/html/loader/interfacel4loader_1_1app.h... HTH, Martin
On Mon, 21 Jun 2010 17:29:00 +0200, Martin Pohlack wrote:
I checked init_io(), it seems to do the same as a fragment in term_con_test example -- it opens a terminal device and checks the file descriptors to be 0, 1, 2 (with unessential details). I forgot to mention that my program is an API server which implements an API and starts a client program using that API. I load the client program, all needed libraries for it. Then I start a function called trampoline() in a separate task, prepare an environment for the program in that function (including file descriptors). Then I just setup registers, push params for C startup code on stack, and call an entry point in a client program.
I think it is not so easy to transfer these FDs into another task, as server state is associated with many FDs (depending on the server).
Yes, transfer to another task is difficult but I meant to execute the same startup code as in init_io() but for client task, not a server. Now I specified '--stdin /dev/vc0 --stdout /dev/vc0 --stderr /dev/vc0' in server command line and mounted term_con at /dev. -- After that I got one l4con tab opened and all terminal i/o got in there. So, it works but the server owns the terminal. But it is desirable to do the same for client program. I start it using task server, so I cannot specify command line, so I must init terminal i/o other way.
I tried to add "--stdin /dev/vc0" to API server's command line. The init_io() function opens all file descriptors successfully. Then these file descriptors seem to be inherited by a client program (if I try to open /dev/vc0 in trampoline(), it returns a fd equal to 3, 4, 5, so 0, 1, 2 are already used). But when I try calling read(0, ...); (I call it in the API call from the client program, so the task id at the moment is from the client program) it says that
I changed now the app behaviour, so all terminal reads and writes are performed in server stub but not in the context of a client. The server owns a terminal and works with it. But it is not the intended way. Of course, I need each client program have its own std* file descriptors.
I looked again in operations.c:init_io()
If you do not provide "--stdin x" etc. argument, dummy FDs are put into place in the local FD table, e.g., here:
ft_fill_entry(1, file_desc); // fill in dummy entry
I vaguely recall doing this to prevent accidents with printf() etc. in dietlibc which had no error checking regarding invalid FDs. I have not revisited this code since we switched to uclibc years ago ...
So, they're just dummy. I didn't know that ft_fill_entry() fills the local descriptor table. So, it is also needed besides just open()'ing it. But how can I get the file descriptor got from open() to be in a range of 0..2? It seems they are already predefined at the moment, though are dummy..
If you want to have nice support for fork etc. (which requires FD inheritance amongst other things) you will need to extend l4vfs to inform all relevant servers of resource transfers. This may be initiated from the client program, but some generic functions in servers to support transferring resource seems in order.
This may prove to be a challenging design if you want to do it securely though.
Yes, that may be challenging and interesting task :) but for now I need just to assign the descriptors to the client from the very beginning
Have you checked out the loader? I think you may not want to use the task server directly.
http://os.inf.tu-dresden.de/l4env/doc/html/loader/interfacel4loader_1_1app.h...
Yes, I checked. But at the moment the loader uses ldso and can parse ELF only. There was an extra exec layer besides loader itself, so there was a potential possibility to add support of other formats to the loader. But now it seems to be not possible (ELF only). I need support of OS/2 LX format executables, so we implemented support for it ourself. Now the server parses LX format executable, loads all needed libraries, applies fixups and gets a ready for execute binary image. That image is executed by calling its entry point from trampoline() which starts in a separate task. For that, a task server is used. We cannot use loader server because the format is not ELF. We think the previous design with separate exec and loader server was better with possibility to add more executable parsers. WBR, valery
Hi Valery, sorry for the delay, I was a little busy in the last days. On 22.06.2010 22:35, Valery V. Sedletski wrote:
On Mon, 21 Jun 2010 17:29:00 +0200, Martin Pohlack wrote:
I checked init_io(), it seems to do the same as a fragment in term_con_test example -- it opens a terminal device and checks the file descriptors to be 0, 1, 2 (with unessential details). I forgot to mention that my program is an API server which implements an API and starts a client program using that API. I load the client program, all needed libraries for it. Then I start a function called trampoline() in a separate task, prepare an environment for the program in that function (including file descriptors). Then I just setup registers, push params for C startup code on stack, and call an entry point in a client program.
I think it is not so easy to transfer these FDs into another task, as server state is associated with many FDs (depending on the server).
Yes, transfer to another task is difficult but I meant to execute the same startup code as in init_io() but for client task, not a server. Now I specified '--stdin /dev/vc0 --stdout /dev/vc0 --stderr /dev/vc0' in server command line and mounted term_con at /dev. -- After that I got one l4con tab opened and all terminal i/o got in there. So, it works but the server owns the terminal. But it is desirable to do the same for client program. I start it using task server, so I cannot specify command line, so I must init terminal i/o other way.
I did a quick check how to specify command lines and didn't find a way. I have probably overlooked it. Can others jump in here?
I tried to add "--stdin /dev/vc0" to API server's command line. The init_io() function opens all file descriptors successfully. Then these file descriptors seem to be inherited by a client program (if I try to open /dev/vc0 in trampoline(), it returns a fd equal to 3, 4, 5, so 0, 1, 2 are already used). But when I try calling read(0, ...); (I call it in the API call from the client program, so the task id at the moment is from the client program) it says that
I changed now the app behaviour, so all terminal reads and writes are performed in server stub but not in the context of a client. The server owns a terminal and works with it. But it is not the intended way. Of course, I need each client program have its own std* file descriptors.
I looked again in operations.c:init_io()
If you do not provide "--stdin x" etc. argument, dummy FDs are put into place in the local FD table, e.g., here:
ft_fill_entry(1, file_desc); // fill in dummy entry
I vaguely recall doing this to prevent accidents with printf() etc. in dietlibc which had no error checking regarding invalid FDs. I have not revisited this code since we switched to uclibc years ago ...
So, they're just dummy. I didn't know that ft_fill_entry() fills the local descriptor table. So, it is also needed besides just open()'ing it. But how can I get the file descriptor got from open() to be in a range of 0..2? It seems they are already predefined at the moment, though are dummy..
No, open uses ft_fill_entry(). But open will not reuse FDs marked as in-use (as done by ft_fill_entry()). * Have you tried closing them? * Have you tried dup2()ing into them? * You can always modify the library code to not reserve these special FDs if you want to. I probably cannot give you any more detailed information regarding this problem as I stopped developing l4vfs about 2 years ago. Maybe a current maintainer will jump into the discussion here ... Alternatively, the Dresden L4 group released a new userland version which may be supported more actively. Maybe this is an alternative for you? http://os.inf.tu-dresden.de/L4Re/overview.html HTH, Martin
participants (2)
-
Martin Pohlack -
Valery V. Sedletski