Hello. Does the NOVA/NUL support snapshotting of VMes? -- Sartakov A. Vasily
Hello, On 15.07.2013 13:50, Sartakov A. Vasily wrote:
Does the NOVA/NUL support snapshotting of VMes? No, however there are (were?) ongoing activities in that direction, see the links above. Maybe the author of the issues can comment on that in more detail.
Cheers, Alex. [0] https://github.com/TUD-OS/seoul/issues/8 [1] https://github.com/TUD-OS/seoul/issues/9 [2] https://github.com/TUD-OS/seoul/issues/10 [3] https://github.com/TUD-OS/seoul/issues/11 [4] https://github.com/TUD-OS/seoul/issues/12 [5] https://github.com/TUD-OS/seoul/issues/13
On Mon, 15 Jul 2013 15:12:02 +0200 Alexander Boettcher (AB) wrote: AB> On 15.07.2013 13:50, Sartakov A. Vasily wrote: AB> > Does the NOVA/NUL support snapshotting of VMes? AB> No, however there are (were?) ongoing activities in that direction, see AB> the links above. Maybe the author of the issues can comment on that in AB> more detail. VM migration support is work in progress, but looking good. We expect to release the code for it later this year, when Jacek finishes his masters thesis on the subject. Vasily - I would recommend that you look at the Github discussions that Alex linked to in his last mail, to see what kind of snapshots we implement for VM migration and whether they are suitable for your purposes. Cheers, Udo
Thank you for links, there are very useful. I have a question about VMCS. How to obtain VMCS data for migration? After creation of new VM, migration process should somehow give a context of transferring machine to NOVA, am I right? 2013/7/15 Udo Steinberg <udo@hypervisor.org>:
On Mon, 15 Jul 2013 15:12:02 +0200 Alexander Boettcher (AB) wrote:
AB> On 15.07.2013 13:50, Sartakov A. Vasily wrote: AB> > Does the NOVA/NUL support snapshotting of VMes? AB> No, however there are (were?) ongoing activities in that direction, see AB> the links above. Maybe the author of the issues can comment on that in AB> more detail.
VM migration support is work in progress, but looking good. We expect to release the code for it later this year, when Jacek finishes his masters thesis on the subject.
Vasily - I would recommend that you look at the Github discussions that Alex linked to in his last mail, to see what kind of snapshots we implement for VM migration and whether they are suitable for your purposes.
Cheers, Udo
_______________________________________________ l4-hackers mailing list l4-hackers@os.inf.tu-dresden.de http://os.inf.tu-dresden.de/mailman/listinfo/l4-hackers
-- Sartakov A. Vasily
On Thu, 18 Jul 2013 18:30:46 +0400 Sartakov A. Vasily (SAV) wrote: SAV> I have a question about VMCS. How to obtain VMCS data for migration? SAV> After creation of new VM, migration process should somehow give a SAV> context of transferring machine to NOVA, am I right? User code does not have to care about the details of VMCS/VMCB layout. Transfer of guest state to/from the microhypervisor happens by means of the UTCB data area as described in Sections 7.3 and 8.3 of the NOVA specification. In fact, user code is never given direct access to the VMCS/VMCB in order to prevent modification of host state. Cheers, Udo
Hello.
On Thu, 18 Jul 2013 18:30:46 +0400 Sartakov A. Vasily (SAV) wrote:
SAV> I have a question about VMCS. How to obtain VMCS data for migration? SAV> After creation of new VM, migration process should somehow give a SAV> context of transferring machine to NOVA, am I right?
User code does not have to care about the details of VMCS/VMCB layout. Transfer of guest state to/from the microhypervisor happens by means of the UTCB data area as described in Sections 7.3 and 8.3 of the NOVA specification. In fact, user code is never given direct access to the VMCS/VMCB in order to prevent modification of host state.
Could you please describe how to get access to a context of virtual machine? How can I modify context of virtual machine from Vancouver? When system switches into Vancouver, for example by pressing a special keyboard key, where is stored context of VM? in allocated VMCB? Can I read and modify its from Vancouver? Thank you. -- Sartakov A. Vasily sartakov@ksyslabs.org
On Fri, 23 Aug 2013 15:13:35 +0400 Sartakov A. Vasily (SAV) wrote: SAV> Could you please describe how to get access to a context of virtual machine? SAV> How can I modify context of virtual machine from Vancouver? SAV> When system switches into Vancouver, for example by pressing a special keyboard key, where is stored context of VM? in allocated VMCB? Can I read and modify its from Vancouver? If you look at git/nul/vancouver/apps/vancouver/vancouver.cc you'll see functions like: VM_FUNC(PT_VMX + 31, vmx_rdmsr, MTD_RIP_LEN | MTD_GPR_ACDB | MTD_TSC | MTD_SYSENTER | MTD_STATE, handle_vcpu(pid, true, CpuMessage::TYPE_RDMSR, tls, utcb);) This is the handler for VMX portal 31, which handles emulation of the RDMSR instruction (see NOVA specification Section 7.2 for a list of intercepts). The portal is configured to transfer a subset of the architectural state according to the MTD bits (see Section 7.4). The hypervisor stores the requested state into the UTCB, where the VMM can read and modify it. Section 7.3 shows the layout of the UTCB with all the state fields. Cheers, Udo
23.08.2013, в 15:23, Udo Steinberg <udo@hypervisor.org> написал(а):
On Fri, 23 Aug 2013 15:13:35 +0400 Sartakov A. Vasily (SAV) wrote:
SAV> Could you please describe how to get access to a context of virtual machine? SAV> How can I modify context of virtual machine from Vancouver? SAV> When system switches into Vancouver, for example by pressing a special keyboard key, where is stored context of VM? in allocated VMCB? Can I read and modify its from Vancouver?
If you look at git/nul/vancouver/apps/vancouver/vancouver.cc you'll see functions like:
VM_FUNC(PT_VMX + 31, vmx_rdmsr, MTD_RIP_LEN | MTD_GPR_ACDB | MTD_TSC | MTD_SYSENTER | MTD_STATE, handle_vcpu(pid, true, CpuMessage::TYPE_RDMSR, tls, utcb);)
This is the handler for VMX portal 31, which handles emulation of the RDMSR instruction (see NOVA specification Section 7.2 for a list of intercepts). The portal is configured to transfer a subset of the architectural state according to the MTD bits (see Section 7.4). The hypervisor stores the requested state into the UTCB, where the VMM can read and modify it. Section 7.3 shows the layout of the UTCB with all the state fields.
Thanks, now it is clear: NOVA can intercept commands from guest machine, then call VM functions via portal with saved state in UTCB. I do not understand several things about messaging: I see several reply functions and I know where they are called. for example: sigma0.cc, +1659: case MessageConsole::TYPE_START: And I see, that this message is send from function handle_console_switching hostvga.cc, +179: by calling msg1(MessageConsole::TYPE_START, num -1); I hit a key, a key is processed by host driver, driver decides that this is a command for VM switching, host driver sends message to sigma0, sigma0 does something. What about other commands, like MessageHostOp::OP_GET_MODULE? I see that all modules has a handler for this message, but I do not see who and how call them. -- Sartakov A. Vasily sartakov@ksyslabs.org
"Sartakov A. Vasily" <sartakov@ksyslabs.org> schrieb:
23.08.2013, в 15:23, Udo Steinberg <udo@hypervisor.org> написал(а):
On Fri, 23 Aug 2013 15:13:35 +0400 Sartakov A. Vasily (SAV) wrote:
SAV> Could you please describe how to get access to a context of virtual machine? SAV> How can I modify context of virtual machine from Vancouver? SAV> When system switches into Vancouver, for example by pressing a special keyboard key, where is stored context of VM? in allocated VMCB? Can I read and modify its from Vancouver?
If you look at git/nul/vancouver/apps/vancouver/vancouver.cc you'll see functions like:
VM_FUNC(PT_VMX + 31, vmx_rdmsr, MTD_RIP_LEN | MTD_GPR_ACDB | MTD_TSC | MTD_SYSENTER | MTD_STATE, handle_vcpu(pid, true, CpuMessage::TYPE_RDMSR, tls, utcb);)
This is the handler for VMX portal 31, which handles emulation of the RDMSR instruction (see NOVA specification Section 7.2 for a list of intercepts). The portal is configured to transfer a subset of the architectural state according to the MTD bits (see Section 7.4). The hypervisor stores the requested state into the UTCB, where the VMM can read and modify it. Section 7.3 shows the layout of the UTCB with all the state fields.
Thanks, now it is clear: NOVA can intercept commands from guest machine, then call VM functions via portal with saved state in UTCB.
I do not understand several things about messaging:
I see several reply functions and I know where they are called. for example:
sigma0.cc, +1659: case MessageConsole::TYPE_START:
And I see, that this message is send from function handle_console_switching
hostvga.cc, +179: by calling msg1(MessageConsole::TYPE_START, num -1);
I hit a key, a key is processed by host driver, driver decides that this is a command for VM switching, host driver sends message to sigma0, sigma0 does something.
What about other commands, like MessageHostOp::OP_GET_MODULE? I see that all modules has a handler for this message, but I do not see who and how call them.
This command is used by the vbios for Multiboot support. Check executor/vbios_multiboot.cc. HTH, Julian -- Sent from my phone. Please excuse my brevity.
Hi again. Several more questions. If I, for instance, want to have a functionality, similar to save/restore of virtual machines. For saving a virtual machine I can initiate a trap into VMM and use registers from UTCB. For VM restoring, can I fill registers in handle_cpu_init? Is that enough? What should I do with mappings (NPT in particular caused me questions), devices? What is the cpu state in receive function of vbios_multiboot? Is it a beginning state of VM, when the Munech is uploaded? -- Sartakov A. Vasily sartakov@ksyslabs.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 08/28/2013 08:50 AM, Sartakov A. Vasily wrote:
Hi again.
Several more questions.
If I, for instance, want to have a functionality, similar to save/restore of virtual machines. For saving a virtual machine I can initiate a trap into VMM and use registers from UTCB.
For VM restoring, can I fill registers in handle_cpu_init?
Probably, but that doesn't get you very far, because...
Is that enough? What should I do with mappings (NPT in particular caused me questions), devices?
... you need to save and restore device state. Jacek Galowicz has implemented this already. For a short overview of what he did check: https://github.com/TUD-OS/seoul/issues/12 I understand the VMM-specific code will be published at some point.
What is the cpu state in receive function of vbios_multiboot? Is it a beginning state of VM, when the Munech is uploaded?
This function is called whenever some code issues int 19h to (re)start the BIOS boot procedure. Initially this is called by the vBIOS itself. I am not sure what you mean with your second question. Julian -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) iEYEARECAAYFAlIfBTYACgkQ2EtjUdW3H9l4igCguK/f0DvNuF9gkxiMFfPiea36 PtEAoKf/fGaxmLt839WFRDJWX06DL7Nn =772J -----END PGP SIGNATURE-----
Hello.
Is that enough? What should I do with mappings (NPT in particular caused me questions), devices?
... you need to save and restore device state. Jacek Galowicz has implemented this already. For a short overview of what he did check: https://github.com/TUD-OS/seoul/issues/12
I understand that issue. But I do not understand how multiboot/vbios part works. Can you explain me what happens from handle_cpu_init (cpu.cc) to main function (or __start ) of munich. I.e. in range from message "reset CPU state" and first console message of Munich. I do not need full detailed description, just main idea. I stuck on very early stage of VM booting. I see intensive messaging of VBios, Lapic, RTC, Serial, I have got several traps (I count svm_invalid trap), but I seriously do not see the reason, the code that trigger traps.
I understand the VMM-specific code will be published at some point.
What is the cpu state in receive function of vbios_multiboot? Is it a beginning state of VM, when the Munech is uploaded?
This function is called whenever some code issues int 19h to (re)start the BIOS boot procedure. Initially this is called by the vBIOS itself. I am not sure what you mean with your second question.
Julian
-- Sartakov A. Vasily sartakov@ksyslabs.org
Is that enough? What should I do with mappings (NPT in particular caused me questions), devices?
... you need to save and restore device state. Jacek Galowicz has implemented this already. For a short overview of what he did check: https://github.com/TUD-OS/seoul/issues/12
I understand that issue. But I do not understand how multiboot/vbios part works. Can you explain me what happens from handle_cpu_init (cpu.cc) to main function (or __start ) of munich. I.e. in range from message "reset CPU state" and first console message of Munich. I do not need full detailed description, just main idea. I stuck on very early stage of VM booting. I see intensive messaging of VBios, Lapic, RTC, Serial, I have got several traps (I count svm_invalid trap), but I seriously do not see the reason, the code that trigger traps.
I'm wrong. nvm
I understand the VMM-specific code will be published at some point.
What is the cpu state in receive function of vbios_multiboot? Is it a beginning state of VM, when the Munech is uploaded?
This function is called whenever some code issues int 19h to (re)start the BIOS boot procedure. Initially this is called by the vBIOS itself. I am not sure what you mean with your second question.
-- Sartakov A. Vasily sartakov@ksyslabs.org
Hi there. I want to start NOVA/NUL without vga in qemu. I've added -novga, have removed vga:0x03c0 (and kbc, keyb) in PC_PS2 alias I use hostserial in sigma0.nul config string and bzImage-js has "console=ttyS0" and "console=tty" I do not have a Linux login message after booting if Linux, but I have all messages related to boot. What I might forget? Should I add another one hostserial? maybe I should something configure in QEMU? I have tried with "-serial telnet:<..> -monitor telnet:<..>" and get no success. 02.10.2013, в 21:20, Sartakov A. Vasily <sartakov@ksyslabs.org> написал(а):
Is that enough? What should I do with mappings (NPT in particular caused me questions), devices?
... you need to save and restore device state. Jacek Galowicz has implemented this already. For a short overview of what he did check: https://github.com/TUD-OS/seoul/issues/12
I understand that issue. But I do not understand how multiboot/vbios part works. Can you explain me what happens from handle_cpu_init (cpu.cc) to main function (or __start ) of munich. I.e. in range from message "reset CPU state" and first console message of Munich. I do not need full detailed description, just main idea. I stuck on very early stage of VM booting. I see intensive messaging of VBios, Lapic, RTC, Serial, I have got several traps (I count svm_invalid trap), but I seriously do not see the reason, the code that trigger traps.
I'm wrong. nvm
I understand the VMM-specific code will be published at some point.
What is the cpu state in receive function of vbios_multiboot? Is it a beginning state of VM, when the Munech is uploaded?
This function is called whenever some code issues int 19h to (re)start the BIOS boot procedure. Initially this is called by the vBIOS itself. I am not sure what you mean with your second question.
-- Sartakov A. Vasily sartakov@ksyslabs.org
-- Sartakov A. Vasily sartakov@ksyslabs.org
On 10/14/2013 12:45 PM, Sartakov A. Vasily wrote:
Hi there.
I want to start NOVA/NUL without vga in qemu. I've added -novga, have removed vga:0x03c0 (and kbc, keyb) in PC_PS2 alias I use hostserial in sigma0.nul config string and bzImage-js has "console=ttyS0" and "console=tty" I do not have a Linux login message after booting if Linux, but I have all messages related to boot.
This indicates that your setup wrt to NOVA/NUL is correct, but your virtual Linux is not spawning a getty on its serial device. You need a line such as: T0:23:respawn:/sbin/getty -w -L ttyS0 115200 vt100 in /etc/inittab to get a login prompt on the serial console. You can extract, modify and recreate your initrd with cpio. For details check for example this page: http://landley.net/writing/rootfs-howto.html If the bzImage has the initramfs image already embedded, it should work to use an initramfs with only the updated /etc/inittab file. Julian
Ah, I forgot check most simplest thing thank you. 14.10.2013, в 17:22, Julian Stecklina <jsteckli@os.inf.tu-dresden.de> написал(а):
On 10/14/2013 12:45 PM, Sartakov A. Vasily wrote:
Hi there.
I want to start NOVA/NUL without vga in qemu. I've added -novga, have removed vga:0x03c0 (and kbc, keyb) in PC_PS2 alias I use hostserial in sigma0.nul config string and bzImage-js has "console=ttyS0" and "console=tty" I do not have a Linux login message after booting if Linux, but I have all messages related to boot.
This indicates that your setup wrt to NOVA/NUL is correct, but your virtual Linux is not spawning a getty on its serial device.
You need a line such as: T0:23:respawn:/sbin/getty -w -L ttyS0 115200 vt100 in /etc/inittab to get a login prompt on the serial console.
You can extract, modify and recreate your initrd with cpio. For details check for example this page: http://landley.net/writing/rootfs-howto.html
If the bzImage has the initramfs image already embedded, it should work to use an initramfs with only the updated /etc/inittab file.
Julian
-- Sartakov A. Vasily sartakov@ksyslabs.org
Hi again According to my "ps aux" I see two getty with ttyS0 and ttyS1. Moreover I see /dev/issue welcome messages in my serial console but I still can’t write anythings, looks like console gives output only. I have tried with pty/udp/telnet on qemu side - all the same, only output. And just curios: Are there any news about publishing live migration stuff in seoul?
I want to start NOVA/NUL without vga in qemu. I've added -novga, have removed vga:0x03c0 (and kbc, keyb) in PC_PS2 alias I use hostserial in sigma0.nul config string and bzImage-js has "console=ttyS0" and "console=tty" I do not have a Linux login message after booting if Linux, but I have all messages related to boot.
This indicates that your setup wrt to NOVA/NUL is correct, but your virtual Linux is not spawning a getty on its serial device.
You need a line such as: T0:23:respawn:/sbin/getty -w -L ttyS0 115200 vt100 in /etc/inittab to get a login prompt on the serial console.
You can extract, modify and recreate your initrd with cpio. For details check for example this page: http://landley.net/writing/rootfs-howto.html
If the bzImage has the initramfs image already embedded, it should work to use an initramfs with only the updated /etc/inittab file.
Julian
-- Sartakov A. Vasily sartakov@ksyslabs.org
On 01/09/2014 04:43 PM, Sartakov A. Vasily wrote:
Hi again
According to my "ps aux" I see two getty with ttyS0 and ttyS1. Moreover I see /dev/issue welcome messages in my serial console but I still can’t write anythings, looks like console gives output only. I have tried with pty/udp/telnet on qemu side - all the same, only output.
I am confused. Are you asking about Qemu or NUL/Vancouver?
And just curios:
Are there any news about publishing live migration stuff in seoul?
https://github.com/TUD-OS/seoul/pull/39 But it is not integrated yet. Julian
09 янв. 2014 г., в 16:46, Julian Stecklina <jsteckli@os.inf.tu-dresden.de> написал(а):
On 01/09/2014 04:43 PM, Sartakov A. Vasily wrote:
Hi again
According to my "ps aux" I see two getty with ttyS0 and ttyS1. Moreover I see /dev/issue welcome messages in my serial console but I still can’t write anythings, looks like console gives output only. I have tried with pty/udp/telnet on qemu side - all the same, only output.
I am confused. Are you asking about Qemu or NUL/Vancouver?
I’m using nested virtualization. I run NOVA/NUL inside QEMU
And just curios:
Are there any news about publishing live migration stuff in seoul?
https://github.com/TUD-OS/seoul/pull/39
But it is not integrated yet.
Julian
-- Sartakov A. Vasily sartakov@ksyslabs.org
On 01/09/2014 04:48 PM, Sartakov A. Vasily wrote:
09 янв. 2014 г., в 16:46, Julian Stecklina <jsteckli@os.inf.tu-dresden.de> написал(а):
On 01/09/2014 04:43 PM, Sartakov A. Vasily wrote:
Hi again
According to my "ps aux" I see two getty with ttyS0 and ttyS1. Moreover I see /dev/issue welcome messages in my serial console but I still can’t write anythings, looks like console gives output only. I have tried with pty/udp/telnet on qemu side - all the same, only output.
I am confused. Are you asking about Qemu or NUL/Vancouver?
I’m using nested virtualization. I run NOVA/NUL inside QEMU
I am not sure whether we have ever implemented serial input to VMs. Serial was mostly used to debug NOVA/NUL. How did you configure NUL? Julian
On Thu, Jan 09, 2014 at 05:53:56PM +0100, Julian Stecklina wrote:
On 01/09/2014 04:48 PM, Sartakov A. Vasily wrote:
According to my "ps aux" I see two getty with ttyS0 and ttyS1. Moreover I see /dev/issue welcome messages in my serial console but I still can???t write anythings, looks like console gives output only. I have tried with pty/udp/telnet on qemu side - all the same, only output.
I am confused. Are you asking about Qemu or NUL/Vancouver?
I???m using nested virtualization. I run NOVA/NUL inside QEMU
I am not sure whether we have ever implemented serial input to VMs.
I had, but the functionality seems to be lost in Seoul. You could either directly assign the serial port from the host via directio.cc or forward keyboard input from the console via kbd2serial.cc. Finally there was a serial2kbd.cc to get keystrokes from the serial port. Bernhard
participants (5)
-
Alexander Boettcher -
Bernhard Kauer -
Julian Stecklina -
Sartakov A. Vasily -
Udo Steinberg