Preliminary NUL documentation

Author: Michal Sojka (sojka@os.inf.tu-dresden.de)

Date: 2011-08-26 09:02:06 CEST

Preliminary NUL documentation

Table of Contents

1 Introduction

Nova Userland (NUL) is a user level software used with Nova microhypervisor.

2 Compilation

To see full command lines run scons VERBOSE=1. To compile a debugging version (with less optimization applied), use scons debug=1.

Most of the compiler parameters is defined in base/SConscript.

3 Sigma0

3.1 Basic operation

Sigma0 is the root task (i.e. it runs in a root protection domain and it is the first task run by the hypervisor). Its initialization is controlled by command line parameters. Every parameter basically describe the operations to be performed on startup. In Sigma0 sources, every parameter is defined with PARAM_HANDLER() macro. Its first argument is the name of the parameter, the second argument is the code to be performed when the parameter is encountered in the script and the rest of arguments are lines of the help text.

After all parameters are processed, the main thread blocks and Sigma0 is controlled only through inter-process communication (IPC).

3.2 Entry point

The entry point of sigma0 is the symbol start, which is defined to be the same as static method Sigma0::start() in sigma0.cc. This function is called from program.h, which defines __start symbol (an entry point defined in the linker script).

3.3 Memory management

Sigma0 class iherits _free_virt members, which is a region list used to manage free virtual memory for later delegation to childs. Initially, all memory is put into this list.

3.4 Memory mapping

Memory mapping is only possible during IPC. Therefore, to map the memory from hypervisor sigma0 calls itself through "echo portal". This is implemented in map_self() method. The actual mapping is performed by hypervisor during reply to the echo call. It cannot be done during the call, because utcb->crd of receiving thread must correspond the mapping and the caller cannot easily manipulate utcb->crd of the receiving thread.

3.5 Memory allocation

Memory allocations (operator new) is implemented in library called service (file simplemalloc.cc). The implementation simply calls the function pointed out by memalloc pointer, whose value defaults to memalloc_mempool().

Sigma0 uses a more sofisticated memory allocator defined in sigma0_memalloc(). It allocate physical memory by consulting _free_phys region list and then maps the memory to its address space.

Memory freeing (memfree pointer) is not currently implemented.

3.6 Portals

Portal wrapper functions cannot be greped - they are defined with PT_FUNC() or PT_FUNC_NORETURN() macros.

3.7 Capability selector allocation

TODO How are capability selectors allocated?

0 – 0x10000reserved by program.h
0xffparent id - parent identifies us by translation of this capability to its PD
0x100 – 0x1ffMAC_CPUS??? - new parent protocol.
0x200semaphore to block the main thread after initialization is done
0x10000 – 0xffffffff(CLIENT_PT_OFFSET, CLIENT_PT_ORDER)

The following is the excerpt from a boot log on a 2-CPU system sorted by the capability selector (idx_*). Note that these events are not normally logged; these messages were only added for the purpose of this list. It lists the objects created in sigma0, their selectors and other parameters. Memory mapping selectors are not included in this list.

nova_create_pt(idx_pt=0x00000100, idx_ec=0x00000227, eip=0x0040c7a8, mtd=0x00000000, dstpd=0x00000020) // parent portal CPU0
nova_create_pt(idx_pt=0x00000101, idx_ec=0x0000024e, eip=0x0040c7a8, mtd=0x00000000, dstpd=0x00000020) // parent portal CPU1
nova_create_sm(idx_sm=0x00000200, initial=0, dstpd=0x00000020) // _cap_block (program.h)
nova_create_sm(idx_sm=0x00000201, initial=0, dstpd=0x00000020) // _lock_gsi
nova_create_sm(idx_sm=0x00000202, initial=0, dstpd=0x00000020) // _lock_mem
nova_create_ec(idx_ec=0x00000203, utcb=0x3fe000, esp=0x43dff4, cpunr=0, excpt_base=0x00000000, dstpd=0x00000020) // do_map portal
// Exception handling portals CPU0
nova_create_pt(idx_pt=0x00000204, idx_ec=0x00000203, eip=0x0041128a, mtd=0x00000000, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000205, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000206, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000207, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000208, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000209, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000020a, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000020b, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000020c, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000020d, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000020e, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000020f, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000210, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000211, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000212, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000213, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000214, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000215, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000216, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000217, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000218, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000219, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000021a, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000021b, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000021c, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000021d, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000021e, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000021f, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000220, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000221, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000222, idx_ec=0x00000203, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)

nova_create_pt(idx_pt=0x00000223, idx_ec=0x00000203, eip=0x00408755, mtd=0x0000000c, dstpd=0x00000020) // do_thread_startup
nova_create_ec(idx_ec=0x00000225, utcb=0x3fb000, esp=0x43cff4, cpunr=0, excpt_base=0x00000205, dstpd=0x00000020) // ec_echo
nova_create_ec(idx_ec=0x00000227, utcb=0x3f8000, esp=0x43bff4, cpunr=0, excpt_base=0x00000205, dstpd=0x00000020)
nova_create_sm(idx_sm=0x00000229, initial=0, dstpd=0x00000020)
nova_create_ec(idx_ec=0x0000022a, utcb=0x2fc000, esp=0x2faff4, cpunr=1, excpt_base=0x00000000, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000022b, idx_ec=0x0000022a, eip=0x0041128a, mtd=0x00000000, dstpd=0x00000020)

// Exception handling portals CPU1
nova_create_pt(idx_pt=0x0000022c, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000022d, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000022e, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000022f, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000230, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000231, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000232, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000233, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000234, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000235, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000236, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000237, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000238, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000239, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000023a, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000023b, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000023c, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000023d, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000023e, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000023f, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000240, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000241, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000242, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000243, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000244, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000245, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000246, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000247, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000248, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000249, idx_ec=0x0000022a, eip=0x00408d78, mtd=0x000effff, dstpd=0x00000020)

nova_create_pt(idx_pt=0x0000024a, idx_ec=0x0000022a, eip=0x00408755, mtd=0x0000000c, dstpd=0x00000020) // do_thread_startup
nova_create_ec(idx_ec=0x0000024c, utcb=0x2f8000, esp=0x2f6ff4, cpunr=1, excpt_base=0x0000022c, dstpd=0x00000020)
nova_create_ec(idx_ec=0x0000024e, utcb=0x2f4000, esp=0x2f2ff4, cpunr=1, excpt_base=0x0000022c, dstpd=0x00000020)
nova_create_sm(idx_sm=0x00000250, initial=0, dstpd=0x00000020)
nova_create_ec(idx_ec=0x00000251, utcb=0x2e9000, esp=0xbfbf6ff4, cpunr=0, excpt_base=0x00000205, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000252, idx_ec=0x00000251, eip=0x00402bfe, mtd=0x00000000, dstpd=0x00000020)
nova_create_ec(idx_ec=0x00000253, utcb=0x2e6000, esp=0xbfbf5ff4, cpunr=1, excpt_base=0x0000022c, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000254, idx_ec=0x00000253, eip=0x00402bfe, mtd=0x00000000, dstpd=0x00000020)
nova_create_ec(idx_ec=0x00000255, utcb=0x2e3000, esp=0xbfbf3ff4, cpunr=0, excpt_base=0x00000205, dstpd=0x00000020)
nova_create_sc(idx_sc=0x00000256, idx_ec=0x00000255, dstpd=0x00000020)
nova_create_ec(idx_ec=0x00000257, utcb=0x2e0000, esp=0xbfbf1ff4, cpunr=0, excpt_base=0x00000205, dstpd=0x00000020)
nova_create_sc(idx_sc=0x00000258, idx_ec=0x00000257, dstpd=0x00000020)
nova_create_ec(idx_ec=0x00000259, utcb=0x2dd000, esp=0xbfbf0ff4, cpunr=1, excpt_base=0x0000022c, dstpd=0x00000020)
nova_create_sc(idx_sc=0x0000025a, idx_ec=0x00000259, dstpd=0x00000020)
nova_create_ec(idx_ec=0x0000025c, utcb=0x2da000, esp=0xbfbeeff4, cpunr=0, excpt_base=0x00000000, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000025d, idx_ec=0x0000025c, eip=0x004045e6, mtd=0x00000000, dstpd=0x00000020)
nova_create_sm(idx_sm=0x0000025f, initial=0, dstpd=0x00000020)
nova_create_ec(idx_ec=0x00000261, utcb=0x2d7000, esp=0xbfbecff4, cpunr=1, excpt_base=0x00000000, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000262, idx_ec=0x00000261, eip=0x004045e6, mtd=0x00000000, dstpd=0x00000020)
nova_create_sm(idx_sm=0x00000264, initial=0, dstpd=0x00000020)
nova_create_ec(idx_ec=0x00000267, utcb=0x2d4000, esp=0xbfbe9ff4, cpunr=0, excpt_base=0x000fefe0, dstpd=0x00000020)
nova_create_pt(idx_pt=0x00000268, idx_ec=0x00000267, eip=0x00401b21, mtd=0x00000000, dstpd=0x00000020)
nova_create_sm(idx_sm=0x00000269, initial=0, dstpd=0x00000020)
nova_create_ec(idx_ec=0x0000026b, utcb=0x2d1000, esp=0xbfbe7ff4, cpunr=0, excpt_base=0x00000205, dstpd=0x00000020)
nova_create_ec(idx_ec=0x0000026c, utcb=0x2ce000, esp=0xbfbe6ff4, cpunr=1, excpt_base=0x000fefe4, dstpd=0x00000020)
nova_create_pt(idx_pt=0x0000026d, idx_ec=0x0000026c, eip=0x00401b21, mtd=0x00000000, dstpd=0x00000020)
nova_create_sm(idx_sm=0x0000026e, initial=0, dstpd=0x00000020)
nova_create_ec(idx_ec=0x00000270, utcb=0x2cb000, esp=0xbfbe4ff4, cpunr=1, excpt_base=0x0000022c, dstpd=0x00000020)
nova_create_sm(idx_sm=0x00000271, initial=0, dstpd=0x00000020)
nova_create_ec(idx_ec=0x00000272, utcb=0x2c8000, esp=0xbfbd4ff4, cpunr=0, excpt_base=0x00000205, dstpd=0x00000020)
nova_create_sc(idx_sc=0x00000273, idx_ec=0x00000272, dstpd=0x00000020)
nova_create_ec(idx_ec=0x00000274, utcb=0x2c5000, esp=0xbfbd2ff4, cpunr=0, excpt_base=0x00000205, dstpd=0x00000020)
nova_create_sc(idx_sc=0x00000275, idx_ec=0x00000274, dstpd=0x00000020)
nova_create_ec(idx_ec=0x00000276, utcb=0x2c2000, esp=0xbfbd0ff4, cpunr=0, excpt_base=0x00000205, dstpd=0x00000020)
nova_create_sc(idx_sc=0x00000277, idx_ec=0x00000276, dstpd=0x00000020)
nova_create_ec(idx_ec=0x00000278, utcb=0x2bf000, esp=0xbfbcfff4, cpunr=0, excpt_base=0x00000205, dstpd=0x00000020)
nova_create_sc(idx_sc=0x00000279, idx_ec=0x00000278, dstpd=0x00000020)
nova_create_sm(idx_sm=0x000100ff, initial=0, dstpd=0x00000020)
nova_create_sm(idx_sm=0x000fefd7, initial=0, dstpd=0x00000020)
nova_create_sm(idx_sm=0x000fefdc, initial=0, dstpd=0x00000020)
nova_create_pt(idx_pt=0x000fefee, idx_ec=0x0000026b, eip=0x004016dd, mtd=0x0000800f, dstpd=0x00000020)
nova_create_pt(idx_pt=0x000feff2, idx_ec=0x00000270, eip=0x004016dd, mtd=0x0000800f, dstpd=0x00000020)
nova_create_sm(idx_sm=0x000ff000, initial=0, dstpd=0x00000020)
nova_create_sm(idx_sm=0x000ff001, initial=0, dstpd=0x00000020)
nova_create_sm(idx_sm=0x000ff002, initial=0, dstpd=0x00000020)
nova_create_sm(idx_sm=0x000ff003, initial=0, dstpd=0x00000020)
nova_create_sm(idx_sm=0x000ff005, initial=0, dstpd=0x00000020)

c

3.8 Communication between various parts of Sigma0

The communication is currently handled by means of "devices" interconnected via various "buses". Sigma0 is also a "device". Currently, all the devices live in a single address space and therefore the bus simply pass the messages (sent by calling send_*() method) to receiving devices by calling receive() method of all devices on the bus.

Technically, the receive() methods are called by static methods inherited from StaticReceiver class template. This template generates static methods receive_static() which simply call normal member functions receive(). There might be several of these functions differentiated by the type of the message.

3.8.1 Communication diagram

The diagram below tries to express interconnection between the devices (ellipses) and the buses (rectangles). It was created semi-automatically so there might be some inaccuracies. The connections of bus_hostop and bus_hwioin/out were intentionally not included because almost every devices sends the messages there. dot

nova_com.png

3.9 TimerService

TimerService is a service that provides timers and time functionality to its clients. It currently runs within Sigma0, but it is planned to run in its own protection domain.

It listens on portal to perform requests from clients. The main services provided are OPEN, REQUEST_TIMER, REQUEST_TIME and CLONE. OPEN operation creates a semaphore which is shared with the client. This semaphore is signaled when a timer, set up with REQUEST_TIMER, fires.

TimerService internally uses motherboard and buses _mymb), similarly as Sigma0, but these seem not to be used for anything useful.

dot

timer_service.png

3.10 HostKeyboard

This driver handles keyboard (and probably even mouse) IRQs, translates the scancodes somehow and sends MessageInput messages to bus_input.

The HostKeyboard 0 is special in that it allows switching consoles and reboot host and guest systems. The HostKeyboard 0 is initialized by the following SIgma0 parameter:

hostkeyb:0,0x60,1

3.11 Console

I do not yet fully understand how console works, but it seems it might be like this:

The actual virtual consoles are implemented in HostVga. A virtual console is a set of views and the view seems to be basically a bunch of VGA and VGA memory. The emulations seems to provide both text and graphical modes. HostVga worker thread periodically (25Hz) updates the real VGA with the content of the currently selected view.

TODO How is console attached to other protection domains? What is exactly bus_console used to.

3.12 HostVga

Listens for MessageInput and uses this for switching the consoles.

3.13 Starting programs

Starting of programs other than Sigma0 (let us call them childs) is controlled by configuration scripts (text files with .nulconfig extension). The scripts have to be loaded by the bootloader into the memory. Programs can be started either automatically by using script and script_start:<num>:1 Sigma0 parameters or manually by keyboard shortcuts (by default LWin+<num>). The number <num> refers to the num-th .nulconfig script.

The actual spawning of the program is implemented in Sigma0::_start_config(). It involves the following steps:

  1. Allocation of memory and copying the executable to that memory.
  2. Allocation of a console for the the new program
  3. Preparation of a new Hypervisor Information Page (HIP) for the program.
  4. Creation of exception handling portals (page-fault and startup).

    The portal capability selectors are allocated in the range basebase + 0x400. The base is calculated as CLIENT_PT_OFFSET + (<num>-1)*(1<<CLIENT_PT_SHIFT), i.e. for the first client the range is equal to 0x10000 - 0x10400.

  5. Creation of parent protocol portals, one per CPU, staring at base + 0x100.
  6. Creation of a semaphore, that is used for later identification of the client. The capability selector for this semaphore is base + 0xfd.
  7. Creation of client protection domain, execution context and scheduling context (through Admission service).

    During the protection domain creation the capabilities in the range described above (base through base + 0x10400) are delegated to the client where they occupy selectors from 0x0 to 0x400. Therefore, the child knows implicitely how to contact the parent and how to idetify itself to the parent (see parent protocol).

3.13.1 State of a program after startup

The capability selectors 0x0 – 0x3ff in a newly created program.
SelectorSymbolCapability
0x00EPage fault handling portal (handled in Sigma0)
0x01EStartup exception handling portal (handled in Sigma0)
0x020NOVA_DEFAULT_PD_CAPChild protection domain
0x0FFParentProtocol::CAP_PARENT_IDIdentifier (a semaphore) the parent uses to distinguish between clients
0x100, 0x101, …ParentProtocol::CAP_PT_PERCPUPer-CPU portals for calling the parent using a parent protocol.

3.13.2 Controlling the startup

  • sigma0::cpu specifies the CPU where to create the new program.
  • sigma0::dma TODO

4 Semaphores

Futex-like semaphore is defined in semaphore.h as class Semaphore.

5 Service protocol (Parent protocol)

Service protocol (formerly called parent protocol) is used for communication between clients and services (also called servers). It offers the following properties:

  • Name service: Services are identified by strings such as "s0/timer".
  • Authorization: Clients must by authorized by a parent to use a particular service.
  • Anonymity: the service does not know the real identity of the client.
  • Quota: …

The service protocol consists of several closely related protocols:

  • Parent protocol covers communication between a client (or a service) and its parent.
  • Session protocol is used in communication between the client and the service.

Parent protocol (class ParentProtocol) implements the lowest level of the protocol. The message is composed in UTCB (untyped items) and its format is defined as:

Request message format
WordContent
0Operation type
1client identifier (typically one of the following: client ID, client pseudonym or session ID)
otheroperation dependent data
Response message format
WordContent
0Return value
otheroperation dependent data

IdentifyCap is a capability that is used to identify the sender in a parent. If the child calls its parent, the identity is the semaphore created by the parent in Sigma0::_start_config().

5.1 Unsorted notes

  • Although the parent protocol and the session protocol are logically different, there is some similarity between them. Namely, the parent also provides a service to clients. This service allows them to connect to other services. Because of that the CAP_PARENT_ID can be treated as the pseudonym of the child, i.e. it is used when the parent is first contacted by the child. Children uses this pseudonym to open sessions (TYPE_OPEN ) with parent. This parent-child session ID is also used as pseudonym for the other services contacted by the child.

5.2 Operations

Name of the constant (the name of the method, if it differs from the constant in anything but case and TYPE_ prefix).

5.2.1 TYPE_OPEN (get_pseudonym)

This operation serves two distinct purposes.

In parent
When a client wants to access a service, it needs to get a "pseudonym" for that service. Is uses the TYPE_OPEN operation to get the pseudonym from its parent. The parent can identify which client is asking for pseudonym by the id capability, which was delegated to him on startup. Depending on the identity and configuration, the parent either grants the client the access to the service by delegating him the "pseudonym" capability and returning 0 or it denies the access by not delegating the pseudonym and returning ERROR.
In service
When it is required by the serivce, the client uses this operation to open the session with the service. The client uses the pseudonym obtained from the parent to identify itself to the service.

5.2.2 TYPE_CLOSE (release_pseudonym)

5.2.3 TYPE_GET_PORTAL

Returns the portal for the service previously requested by TYPE_OPEN .

5.2.4 TYPE_REGISTER (register_service)

5.2.5 TYPE_UNREGISTER (unregister_service)

5.2.6 TYPE_GET_QUOTA

5.2.7 TYPE_REQ_KILL (kill)

5.3 Generic protocol

Generic protocol (class GenericProtocol) is a client-side implementation of the whole protocol for talking to the service. Its method call_server() takes care of setting up everything as well as of sending the message. Typically, service specific protocols are derived from this class.

The protocol operates on a continuous range of capability selectors as shown in the table below:

Capability selectors used by the service protocol
SelectorSymbolMeaning
base + 0CAP_PSEUDONYMIdentifies the client to the service
base + 1CAP_LOCKTODO
base + 2CAP_SERVER_SESSIONIdentifies the session to the service (one client can open multiple sessions to the service)
base + 3, …CAP_SERVER_PTPer-CPU portals to talk to the service. If the service is available only on a single CPU, there might be only a single selector.

The protocol is defined by the following rules:

  • A portal to talk to the service (CAP_SERVER_PT) is obtained from the parent by using TYPE_GET_PORTAL message.
  • A service is called by using a per-CPU portal and the client uses CAP_SERVER_SESSION capability to identify itself to the service.
  • If the identity supplied by the client is not known to the callee it should return EEXISTS error.

A typical communication sequence between a client and a service (and also a parent) looks like this:

  1. Client calls the service (using the operation of its choice) by using CAP_SERVER_PT selector and identifies itself to the service by CAP_SERVER_SESSION. In the beginning, the portal selector refers to the null capability and therefore the call returns NOVA_ECAP.
  2. The client then calls its parent through CAP_PT_PERCPU portal and sends the TYPE_GET_PORTAL message. It identifies itself with CAP_PSEUDONYM, but because it refers to the null capability, the parent (typically Sigma0 responds with EEXIST).
  3. The client calls the parent through CAP_PT_PERCPU to get the pseudonym (TYPE_OPEN) and identify itself with CAP_PARENT_ID. If the client is allowed to use the service, the parent delegates CAP_PSEUDONYM to the client.
  4. Now, the client retries from step 1, but it still gets ECAP.
  5. However, the it succeeds in getting the portal.
  6. Then it again goes to 1 to call the service. If the service is designed to use sessions (CAP_SERVER_SESSION), EEXIST is returned (6a) and the client continues by step 7, otherwise the call is finished (6b).
  7. The client sends TYPE_OPEN to the service delegates him a CAP_SERVER_SESSION.
  8. Finally, the client calls the service with the original operation and a valid CAP_SERVER_SESSION and hopefully it succeeds.

… the ID field is always the session ID. Generic protocol automatically opens the session if it is required, i.e. when the call returns EEXISTS.

Ellipses represent IPC (nova_call) and arrows represent action taken based on the return of the IPC.

generic_proto.png

5.4 Service implementation

Server side of the protocol is implemented in class s0_ParentProtocol and …

… Services can use class ClientDataStorage to store and search for client specific data.

6 Console

Applications can inherit their class from ProgramConsole class in order to have access to the console. The console is created by calling console_init(). This also initializes the Logging facility.

7 Logging

Sometimes it is usefull to have central log of multiple applications. This can be achieved by using the tracebuffer implemented in Sigma0. The tracebuffer is activated by the following command line option to Sigma0

tracebuffer:32768,1

Redirecting the logging output (Logging::printf()) of applications to the tracebuffer is achieved by running this code in the application(s):

_console_data.log = new LogProtocol(alloc_cap(LogProtocol::CAP_SERVER_PT + hip->cpu_count()));

When the second parameter in the command line is non-zero (1 in the example above), the traced outout also goes to the sigma0 output, i.e. to the VGA and/or serial line if those are enabled.

8 Admission service

Admission service is responsible for creating scheduling contexts. No other protection domain (besides Sigma0) has the right to create scheduling contexts and hence those protection domains must contact admission service and ask to create scheduling contexts for them.

8.1 Usage

The following is the example how to initialize admission service and how to create a scheduling context for an ececution context.

#include "nul/service_admission.h"
...
AdmissionProtocol * service_admission;

service_admission = new AdmissionProtocol(alloc_cap(AdmissionProtocol::CAP_SERVER_PT + hip->cpu_desc_count()));
service_admission->set_name(*utcb, "My App");

unsigned ec = nova_create_ec(...);

AdmissionProtocol::sched sched(AdmissionProtocol::sched::TYPE_PERIODIC);
service_admission->alloc_sc(*myutcb(), ec, sched, service_admission->myutcb()->head.nul_cpunr, "worker thread");

8.2 Configuration

The admission service is part of Sigma0 binary and is executed automatically without user intervention. You can configure the admission service by changing the content of base/apps/sigma0/cfg/admission.nulconfig. For instnace, you can remove the top parameter and add log parameter to get some logging output from the service.

9 Restrictions/Caveats

9.1 Static constructors (C++)

NUL does not support automatic calling of constructors of static objects.

10 How to decode crash messages

[ 0] Killed EC:0xc13dd3c0 SC:0xc0010580 V:0xe CS:0x1b EIP:0x80000c39 CR2:0xc ERR:0x4 (PT wrong CPU)
  • V: exception vector - in this case "page fault". See NOVA Microhypervisor specification, section "Event-Speciļ¬c Capability Selectors".
  • EIP: Instruction pointer (dissassemble your binary and find that address).
objdump -S myapp.nul.debug|less -p80000c39
  • CR2: Page fault address
  • ERR: The error code. In this case, the page-fault handling portal is bound to another CPU than the one where the page-fault occured.

11 Vancouver

So far, this section contains only inteconnection diagram of Vancouver. For clarity, this diagram omits connections to hostop, ioout, ioin, mem and memregion busses.

dot

vancouver_com.png

Generated by Org version 7.7 with Emacs version 23

Validate XHTML 1.0