Hi Steffen,
thank you for the elaborate reply! The text you have just written would fit pretty well on karma-vmm.org, doesn't it?. .-)
As far as I know, Afterburner also runs OS kernels on top of a microkernel on platforms that are not virtualizable. Hereby the guest kernel binary is modified to replace virtualization sensitive instructions with "hypercalls". The "hypervisor" resides in the same address space as the guest kernel, and implements an emulation for sensitive instructions, mostly using functionality of the underlying kernel.
I am far from being proficient when it comes to Afterburner but from what I understand so far, there seem to be quite a few similarities between this project and the approach you have taken with Karma. Code-wise, the Afterburner VMM (called wedge) is cleanly separated from the Linux kernel. The fact that its code is executed in the same address space as the Linux kernel rather than a different address space (as in Vancouver and Karma) is just a performance optimization.
The list of interface functions provided by the wedge is illustrative:
https://github.com/l4ka/afterburner/blob/master/afterburn-wedge/ia32/vmiCall...
Most of those functions are concerned with low-level CPU emulation. In the original form of Afterburner, they are invoked by little code stubs that replace their corresponding CPU instructions by upcalls to the wedge. (the replacement of those the critical CPU instructions with stub codes is done via a semi-automated procedure dubbed pre-virtualization) However, support for H/W virtualization has been added at a later stage, which alleviates the need for instrumenting the Linux kernel. So from a large distance, Karma reminds me of the latter variant of Afterburner. To see more clearly how your work integrates with the (quite diverse) virtualization landscape of the L4 world, it would be just great if you contrast both projects in a little more technical detail.
Vancouver is a virtual machine monitor (VMM) that runs on top of the NOVA microkernel. NOVA provides means to do page table management for
From the insights I got with porting Vancouver to Genode (on NOVA), I
got that Vancouver is in fact not inherently coupled to NOVA. It is a largely freestanding code base with only little glue between the VMM and the kernel. Technically, it should feasible to port Vancouver to other kernels with virtualization support such as Fiasco.OC.
first and second stage page tables, as well as the means to do a world switch (switch the CPU from host to guest). Vancouver uses hardware virtualization through the microkernel interface to implement memory and CPU virtualization. For everything else (platform devices, peripheral devices, 16bit code), Vancouver does emulation. Each instance of Vancouver runs exactly one virtual machine (VM). If the attacker is able to escape the VM and compromise the VMM, it is up to the microkernel to ensure that the attack remains contained.
In some sense, the Karma VMM is a mixture between Vancouver and L4Linux. Let me explain this.
Karma does CPU and memory virtualization using the interfaces of Fiasco.OC [0]. It runs as a task on top of the microkernel, and one instance of Karma drives exactly one VM. In contrast to Vancouver, Karma does no emulation at all. Instead, it implements its own custom device models to provide platform devices such as interrupt controllers. For peripheral devices, Karma relies on the L4Re infrastructure, which is very similar to L4Linux. In contrast to L4Linux, Karma requires hardware CPU virtualization (e.g. Intel VT or AMD SVM), and can make use of nested paging for hardware accelerated memory virtualization. As you
From this statement, I looks like Vancouver facilitates the same design
but provides a superset of Karma's functionality. What is the main advantage of Karma compared to Vancouver (apart from running on a different kernel)? Why have you opted to create a new VMM rather than facilitate a port of Vancouver to Fiasco.OC?
Actually we are working on reviving a technology called nested virtualization [1], where we run KVM inside the VM established by Karma. That allows us to run any OS that KVM can run (e.g. Windows).
That looks promising. I am confident that those puzzle pieces will do well together. .-)
For additional information about Karma, you can have a look into my diploma thesis, where you will also find a number of benchmarks: http://os.inf.tu-dresden.de/papers_ps/liebergeld-diplom.pdf
I would be happy if this spawns even more questions, and I am looking forward to answering those.
Please do not take my statements above as offense. I am just genuinely interested.
Cheers Norman