8.8 Role-File runux.mk

8.8.1 Purpose

Use the ptest (runux) role to run 'automated' tests on your package.

This role runs Fiasco-UX with the specified server and test-application and compares the generated output with expected output. If the output does not match a message is displayed and the build process is interrupted.

You may also trigger this test run in the l4/pkg directory by issuing make ptest.

8.8.2 Make-Targets

all::: Phony target. Runs Fiasco UX with given arguments and compares output to expected output.
ptest::: Phony target. Same as all.
genexp::: Phony target. Generate the expected output.
plainrun::: Phony target. Simply runs Fiasco UX with given arguments. No watchdog is installed, no output filtering applied, and no output comparison.
clean::: Delete intermediate compilation files.
cleanall::: Delete all generated files (also the expected output file if generated by make).

8.8.3 Required Parameters

TEST_SERVER The name of the binary to be run as the server to be tested. If this binary name contains no directory elements ('/'), then the binary is assumed to be in the architecture's and L4 API's binary directory. $(L4DIR)/bin/$(ARCH)/$(L4API)
If there are directory elements, is used as is.
If empty, a warning is issued and make terminates successfully.

8.8.4 Optional Parameters

TEST_CLIENT The name of the binary to be run as the application testing the server. Same naming rules apply as with TEST_SERVER.

FIASCOUX_<api> The Fiasco-UX binary to use for the specified $(L4API) Default: $(L4DIR)/kernel/fiasco/build-ux-$(L4API) (with l4 removed from $(L4API)).

BASE_SERVERS The binary names of the base servers required to run the test server. Default: log names dm_phys
If this variable does not contain directory parts ('/'), then it is prefixed with the binary directory of user-land executables. Otherwise it is taken as is.
The binaries are then prefixed with -l to run as modules of Fiasco-UX.

EXPECTED_OUT The name of the file to contain the expected output of the test-run. This file is compared to the output generated in the test-run using diff. Default: expected.txt.
If not specified, the file can be generated by running make with the target genexp. This will also create another file to indicate that the expected output file has been generated. It will then be deleted when running make cleanall.
Note: Because the test is run in an OBJ directory, this should be relative to the OBJ directory (e.g., ../expected.txt).

TIMEOUT The timeout the watchdog waits for Fiasco-UX before killing it. This is the safe-guard for test-applications which do not shut down Fiasco-UX themselves or if Fiasco-UX jumps into the kernel-debugger.
The time is specified in seconds. Default: 10.

DEBUG_PERL If the Perl-script used to run Fiasco-UX with a watchdog is doing something wrong, you may set this variable to 1 to get some debugging output. Default: 0.

SYSTEMS Specify the systems to run this test for. This determines the $(ARCH) and $(API) variable used to find the user-land binaries in the l4/bin directory.

EXPECT_FAIL If set will accept failure of output comparison. If output matches expected output, an error will be issued. This option has no effect in generate mode.

FILTER_KEEP_ALL If set, the output of Fiasco UX will not be filtered.

COMPARE_CMD If set, this command is run instead of the default command. This command should only generate output if there is an error. If no output is generated by the command, success is assumed. In case of an error the last 1000 lines of the output are printed to stderr.
Default: diff -u $(EXPECTED_OUT) $(TMP_OUT)

NO_FBUF_DEV If set, Fiasco UX will not open a window to display graphical output. This is only relevant if your setup does graphical output and you would like to test it in batched mode (no DISPLAY).

USE_SYMBOLS If set to 'y', Fiasco UX will load its Symbols. This helps when debugging. Default: 'y'.

USE_LINES If set to 'y', Fiasco UX will load its Lines. This helps when debugging. Default: 'y'.

8.8.5 Example

PKGDIR = ..
L4DIR ?= $(PKGDIR)/../..

EXPECTED = ../expected.txt

TEST_SERVER = dice_hello_server
TEST_CLIENT = dice_hello_client

include $(L4DIR)/mk/runux.mk

The $(COMPARE_CMD) can be used to apply own filter rules as well. An example that sorts the output depending on the LOG tag is the following:

COMPARE_CMD = "cat $(TMP_OUT)|sort -t '|' -k1,1 -s >$(TMP_OUT) ; diff -u
$(EXPECTED_OUT) $(TMP_OUT)"

L4 Checker 2012-04-11

`TEST_CLIENT`	The name of the binary to be run as the application testing the server. Same naming rules apply as with TEST_SERVER.
`FIASCOUX_<api>`	The Fiasco-UX binary to use for the specified $(L4API) Default: $(L4DIR)/kernel/fiasco/build-ux-$(L4API) (with `l4` removed from $(L4API)).
`BASE_SERVERS`	The binary names of the base servers required to run the test server. Default: `log names dm_phys` If this variable does not contain directory parts ('/'), then it is prefixed with the binary directory of user-land executables. Otherwise it is taken as is. The binaries are then prefixed with `-l` to run as modules of Fiasco-UX.
`EXPECTED_OUT`	The name of the file to contain the expected output of the test-run. This file is compared to the output generated in the test-run using `diff`. Default: `expected.txt`. If not specified, the file can be generated by running make with the target `genexp`. This will also create another file to indicate that the expected output file has been generated. It will then be deleted when running `make cleanall`. Note: Because the test is run in an OBJ directory, this should be relative to the OBJ directory (e.g., `../expected.txt`).
`TIMEOUT`	The timeout the watchdog waits for Fiasco-UX before killing it. This is the safe-guard for test-applications which do not shut down Fiasco-UX themselves or if Fiasco-UX jumps into the kernel-debugger. The time is specified in seconds. Default: 10.
`DEBUG_PERL`	If the Perl-script used to run Fiasco-UX with a watchdog is doing something wrong, you may set this variable to 1 to get some debugging output. Default: 0.
`SYSTEMS`	Specify the systems to run this test for. This determines the $(ARCH) and $(API) variable used to find the user-land binaries in the `l4/bin` directory.
`EXPECT_FAIL`	If set will accept failure of output comparison. If output matches expected output, an error will be issued. This option has no effect in generate mode.
`FILTER_KEEP_ALL`	If set, the output of Fiasco UX will not be filtered.
`COMPARE_CMD`	If set, this command is run instead of the default command. This command should only generate output if there is an error. If no output is generated by the command, success is assumed. In case of an error the last 1000 lines of the output are printed to stderr. Default: diff -u $(EXPECTED_OUT) $(TMP_OUT)
`NO_FBUF_DEV`	If set, Fiasco UX will not open a window to display graphical output. This is only relevant if your setup does graphical output and you would like to test it in batched mode (no DISPLAY).
`USE_SYMBOLS`	If set to 'y', Fiasco UX will load its Symbols. This helps when debugging. Default: 'y'.
`USE_LINES`	If set to 'y', Fiasco UX will load its Lines. This helps when debugging. Default: 'y'.