408 lines
16 KiB
ReStructuredText
408 lines
16 KiB
ReStructuredText
.. _partition_mode:
|
|
|
|
Using partition mode on UP2
|
|
###########################
|
|
|
|
ACRN hypervisor supports partition mode, in which the User OS running in a
|
|
privileged VM can bypass the ACRN hypervisor and directly access isolated
|
|
PCI devices. This tutorial provides step by step instructions on how to set up
|
|
the ACRN hypervisor partition mode on
|
|
`UP2 <https://up-board.org/upsquared/specifications/>`_ boards running two
|
|
privileged VMs as shown in :numref:`two-priv-vms`:
|
|
|
|
.. figure:: images/partition_mode_up2.png
|
|
:align: center
|
|
:name: two-priv-vms
|
|
|
|
Two privileged VMs running in partition mode
|
|
|
|
Prerequisites
|
|
*************
|
|
|
|
In this tutorial two Linux privileged VMs are started by the ACRN hypervisor.
|
|
To set up the Linux root filesystems for each VM, follow the Clear Linux OS
|
|
`bare metal installation guide <https://clearlinux.org/documentation/clear-linux/get-started/bare-metal-install#bare-metal-install>`_
|
|
to install Clear Linux OS on a **SATA disk** and a **USB flash disk** prior the setup,
|
|
as the two privileged VMs will mount the root filesystems via the SATA controller
|
|
and the USB controller respectively.
|
|
|
|
Build kernel and modules for partition mode UOS
|
|
***********************************************
|
|
|
|
#. On your development workstation, clone the ACRN kernel source tree, and
|
|
build the Linux kernel image that will be used to boot the privileged VMs:
|
|
|
|
.. code-block:: none
|
|
|
|
$ git clone https://github.com/projectacrn/acrn-kernel.git
|
|
Cloning into 'acrn-kernel'...
|
|
...
|
|
$ cd acrn-kernel
|
|
$ cp kernel_config_uos .config
|
|
|
|
$ make olddefconfig
|
|
scripts/kconfig/conf --olddefconfig Kconfig
|
|
#
|
|
# configuration written to .config
|
|
#
|
|
|
|
$ make
|
|
$ make modules_install INSTALL_MOD_PATH=out/
|
|
|
|
The last two commands built the bootable kernel image ``arch/x86/boot/bzImage``,
|
|
and the loadable kernel modules under the ``./out/`` folder. Copy these files
|
|
to a removable disk for installing on the UP2 board later.
|
|
|
|
#. Current ACRN partition mode implementation requires a multi-boot capable
|
|
bootloader to boot both ACRN hypervisor and the bootable kernel image
|
|
built from the previous step. You could install Ubuntu OS to the UP2 board
|
|
by following
|
|
`this Ubuntu tutorial <https://tutorials.ubuntu.com/tutorial/tutorial-install-ubuntu-desktop>`_.
|
|
The Ubuntu installer creates 3 disk partitions on the on-board eMMC memory.
|
|
By default, the GRUB bootloader is installed on the ESP (EFI System Partition)
|
|
partition, which will be used to bootstrap the partition mode ACRN hypervisor.
|
|
|
|
#. After installing the Ubuntu OS, power off the UP2 board, attach the SATA disk
|
|
and the USB flash disk to the board. Power on the board and make sure
|
|
it boots the Ubuntu OS from the eMMC, then copy the loadable kernel modules
|
|
built in Step 1 to the ``/lib/modules/`` folder on both the mounted SATA
|
|
disk and USB disk. For example, assuming the SATA disk and USB flash disk
|
|
are assigned to ``/dev/sda`` and ``/dev/sdb`` respectively, the following
|
|
commands set up the partition mode loadable kernel modules onto the root
|
|
filesystems to be loaded by the privileged VMs:
|
|
|
|
.. code-block:: none
|
|
|
|
# Mount the Clear Linux OS root filesystem on the SATA disk
|
|
$ sudo mount /dev/sda3 /mnt
|
|
$ sudo cp -r <kernel-modules-folder-built-in-step1>/lib/modules/* /mnt/lib/modules
|
|
$ sudo umount /mnt
|
|
|
|
# Mount the Clear Linux OS root filesystem on the USB flash disk
|
|
$ sudo mount /dev/sdb3 /mnt
|
|
$ sudo cp -r <path-to-kernel-module-folder-built-in-step1>/lib/modules/* /mnt/lib/modules
|
|
$ sudo umount /mnt
|
|
|
|
#. Copy the bootable kernel image to the ``/boot`` directory:
|
|
|
|
.. code-block:: none
|
|
|
|
$ sudo cp <path-to-kernel-image-built-in-step1>/bzImage /boot/
|
|
|
|
Enable partition mode in ACRN hypervisor
|
|
****************************************
|
|
|
|
#. Before building the ACRN hypervisor, you need to figure out the I/O address
|
|
of the serial port, and the PCI BAR addresses of the SATA controller and
|
|
the USB controller on your UP2 board.
|
|
|
|
Enter the following command to get the I/O addresses of the serial ports.
|
|
UP2 boards support two serial ports. The addresses from the command
|
|
output represent the I/O addresses of the serial port of the 10-pin side
|
|
connector and the 40-pin expansion header respectively. You will need to
|
|
connect the serial port to the development host, in order to access
|
|
the ACRN serial console to switch between privileged VMs.
|
|
|
|
.. code-block:: none
|
|
:emphasize-lines: 1
|
|
|
|
$ sudo lspci | grep UART
|
|
00:18.0 . Series HSUART Controller #1 (rev 0b)
|
|
00:18.1 . Series HSUART Controller #2 (rev 0b)
|
|
|
|
The second with ``00:18.1`` is the one on the 40-pin expansion connector.
|
|
|
|
The following command prints detailed information about all PCI buses
|
|
and devices in the system. Look up the PCI BAR addresses of the SATA
|
|
controller and the USB controller, record these addresses (highlighted below) for
|
|
editing the source code later.
|
|
|
|
.. code-block:: none
|
|
:emphasize-lines: 1,9,10,14,22
|
|
|
|
$ sudo lspci -vv
|
|
...
|
|
00:12.0 SATA controller: Intel Corporation Device 5ae3 (rev 0b) (prog-if 01 [AHCI 1.0])
|
|
Subsystem: Intel Corporation Device 7270
|
|
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
|
|
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
|
|
Latency: 0
|
|
Interrupt: pin A routed to IRQ 123
|
|
Region 0: Memory at 91514000 (32-bit, non-prefetchable) [size=8K]
|
|
Region 1: Memory at 91537000 (32-bit, non-prefetchable) [size=256]
|
|
Region 2: I/O ports at f090 [size=8]
|
|
Region 3: I/O ports at f080 [size=4]
|
|
Region 4: I/O ports at f060 [size=32]
|
|
Region 5: Memory at 91536000 (32-bit, non-prefetchable) [size=2K]
|
|
...
|
|
00:15.0 USB controller: Intel Corporation Device 5aa8 (rev 0b) (prog-if 30 [XHCI])
|
|
Subsystem: Intel Corporation Device 7270
|
|
Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
|
|
Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
|
|
Latency: 0
|
|
Interrupt: pin A routed to IRQ 122
|
|
Region 0: Memory at 91500000 (64-bit, non-prefetchable) [size=64K]
|
|
|
|
#. Clone the ACRN source code and configure the build options with
|
|
``make menuconfig`` command:
|
|
|
|
.. code-block:: none
|
|
|
|
$ git clone https://github.com/projectacrn/acrn-hypervisor.git
|
|
$ cd acrn-hypervisor/hypervisor
|
|
$ make menuconfig
|
|
|
|
Set the ``Hypervisor mode`` option to ``Partition mode``, and depending
|
|
on the serial port you are using, enter its BDF to the configuration
|
|
menu as shown in this screenshot. Finally, save the configuration.
|
|
|
|
.. figure:: images/menuconfig-partition-mode.png
|
|
:align: center
|
|
|
|
.. note::
|
|
|
|
Refer to the :ref:`getting-started-building` for more information on how
|
|
to install all the ACRN build dependencies.
|
|
|
|
#. Configure the PCI device info for each VM
|
|
|
|
PCI devices that are available to the privileged VMs
|
|
are hardcoded in the source file ``hypervisor/partition/apl-mrb/vm_description.c``.
|
|
You need to review and modify the ``vpci_vdev_array1`` and ``vpci_vdev_array2``
|
|
structures in the source code to match the PCI BAR addresses of the SATA
|
|
controller and the USB controller noted in step 1:
|
|
|
|
.. code-block:: none
|
|
:emphasize-lines: 3,39,44,49,56,72
|
|
:caption: hypervisor/partition/vm_description.c
|
|
|
|
...
|
|
static struct vpci_vdev_array vpci_vdev_array1 = {
|
|
.num_pci_vdev = 2,
|
|
|
|
.vpci_vdev_list = {
|
|
{/*vdev 0: hostbridge */
|
|
.vbdf.bits = {.b = 0x00U, .d = 0x00U, .f = 0x0U},
|
|
.ops = &pci_ops_vdev_hostbridge,
|
|
.bar = {},
|
|
.pdev = {
|
|
.bdf.bits = {.b = 0x00U, .d = 0x00U, .f = 0x0U},
|
|
}
|
|
},
|
|
|
|
{/*vdev 1: SATA controller*/
|
|
.vbdf.bits = {.b = 0x00U, .d = 0x01U, .f = 0x0U},
|
|
.ops = &pci_ops_vdev_pt,
|
|
.bar = {
|
|
[0] = {
|
|
.base = 0UL,
|
|
.size = 0x2000UL,
|
|
.type = PCIBAR_MEM32
|
|
},
|
|
[1] = {
|
|
.base = 0UL,
|
|
.size = 0x1000UL,
|
|
.type = PCIBAR_MEM32
|
|
},
|
|
[5] = {
|
|
.base = 0UL,
|
|
.size = 0x1000UL,
|
|
.type = PCIBAR_MEM32
|
|
},
|
|
},
|
|
.pdev = {
|
|
.bdf.bits = {.b = 0x00U, .d = 0x12U, .f = 0x0U},
|
|
.bar = {
|
|
[0] = {
|
|
.base = 0x91514000UL,
|
|
.size = 0x2000UL,
|
|
.type = PCIBAR_MEM32
|
|
},
|
|
[1] = {
|
|
.base = 0x91537000UL,
|
|
.size = 0x100UL,
|
|
.type = PCIBAR_MEM32
|
|
},
|
|
[5] = {
|
|
.base = 0x91536000UL,
|
|
.size = 0x800UL,
|
|
.type = PCIBAR_MEM32
|
|
},
|
|
}
|
|
...
|
|
static struct vpci_vdev_array vpci_vdev_array2 = {
|
|
.num_pci_vdev = 3,
|
|
...
|
|
{/*vdev 1: USB controller*/
|
|
.vbdf.bits = {.b = 0x00U, .d = 0x01U, .f = 0x0U},
|
|
.ops = &pci_ops_vdev_pt,
|
|
.bar = {
|
|
[0] = {
|
|
.base = 0UL,
|
|
.size = 0x10000UL,
|
|
.type = PCIBAR_MEM32
|
|
},
|
|
},
|
|
.pdev = {
|
|
.bdf.bits = {.b = 0x00U, .d = 0x15U, .f = 0x0U},
|
|
.bar = {
|
|
[0] = {
|
|
.base = 0x91500000UL,
|
|
.size = 0x10000UL,
|
|
.type = PCIBAR_MEM64
|
|
},
|
|
}
|
|
...
|
|
|
|
#. Optionally, configure the ``.bootargs`` kernel command line arguments
|
|
|
|
The kernel command line arguments used to boot the privileged VMs are
|
|
hardcoded as ``/dev/sda3`` to meet the Clear Linux OS automatic installation.
|
|
In case you plan to use your customized root
|
|
filesystem, you may optionally edit the ``root=`` parameter specified
|
|
in the ``.bootargs`` field of the ``.vm_desc_array`` structure, to
|
|
instruct the Linux kernel to mount the right disk partition:
|
|
|
|
.. code-block:: none
|
|
:emphasize-lines: 9-11,20-22
|
|
:caption: hypervisor/partition/vm_description.c
|
|
|
|
...
|
|
/* Virtual Machine descriptions */
|
|
.vm_desc_array = {
|
|
{
|
|
/* Internal variable, MUSTBE init to -1 */
|
|
.vm_hw_num_cores = VM1_NUM_CPUS,
|
|
.vm_pcpu_ids = &VM1_CPUS[0],
|
|
...
|
|
.bootargs = "root=/dev/sda3 rw rootwait noxsave maxcpus=2 nohpet console=hvc0 \
|
|
console=ttyS2 no_timer_check ignore_loglevel log_buf_len=16M \
|
|
consoleblank=0 tsc=reliable xapic_phys",
|
|
.vpci_vdev_array = &vpci_vdev_array1,
|
|
.mptable = &mptable_vm1,
|
|
},
|
|
{
|
|
/* Internal variable, MUSTBE init to -1 */
|
|
.vm_hw_num_cores = VM2_NUM_CPUS,
|
|
.vm_pcpu_ids = &VM2_CPUS[0],
|
|
...
|
|
.bootargs = "root=/dev/sda3 rw rootwait noxsave maxcpus=2 nohpet console=hvc0 \
|
|
console=ttyS2 no_timer_check ignore_loglevel log_buf_len=16M \
|
|
consoleblank=0 tsc=reliable xapic_phys",
|
|
.vpci_vdev_array = &vpci_vdev_array2,
|
|
.mptable = &mptable_vm2,
|
|
.lapic_pt = true,
|
|
},
|
|
}
|
|
|
|
#. Build the ACRN hypervisor and copy the artifact ``acrn.32.out`` to the
|
|
``/boot`` directory:
|
|
|
|
.. code-block:: none
|
|
|
|
$ make PLATFORM=sbl
|
|
...
|
|
$ sudo cp build/acrn.32.out /boot
|
|
|
|
#. Modify the ``/etc/grub.d/40_custom`` file to create a new GRUB entry
|
|
that will multi-boot the ACRN hypervisor and the UOS kernel image
|
|
|
|
Append the following configuration to the ``/etc/grub.d/40_custom`` file:
|
|
|
|
.. code-block:: none
|
|
|
|
menuentry 'ACRN Partition Mode' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-e23c76ae-b06d-4a6e-ad42-46b8eedfd7d3' {
|
|
recordfail
|
|
load_video
|
|
gfxmode $linux_gfx_mode
|
|
insmod gzio
|
|
insmod part_gpt
|
|
insmod ext2
|
|
|
|
echo 'Loading partition mode hypervisor ...'
|
|
multiboot /boot/acrn.32.out
|
|
module /boot/bzImage
|
|
}
|
|
|
|
Modify the ``/etc/default/grub`` file as follows to make the GRUB menu visible
|
|
when booting:
|
|
|
|
.. code-block:: none
|
|
|
|
#GRUB_HIDDEN_TIMEOUT=0
|
|
GRUB_HIDDEN_TIMEOUT_QUIET=false
|
|
|
|
Re-generate the GRUB configuration file and reboot the UP2 board. Select
|
|
the ``ACRN Partition Mode`` entry to boot the partition mode of the ACRN
|
|
hypervisor, the hypervisor will start the privileged VMs automatically.
|
|
|
|
.. code-block:: none
|
|
|
|
$ sudo update-grub
|
|
|
|
.. code-block:: console
|
|
:emphasize-lines: 4
|
|
|
|
Ubuntu
|
|
Advanced options for Ubuntu
|
|
System setup
|
|
*ACRN Partition Mode
|
|
|
|
Switch between privileged VMs
|
|
*****************************
|
|
|
|
Connect the serial port on the UP2 board to the development workstation.
|
|
If you set the BDF of the serial port right while building the ACRN hypervisor,
|
|
you should see the output from the ACRN serial console as below.
|
|
You could then log in to the privileged VMs by ``sos_console`` command,
|
|
and press :kbd:`CTRL+Space` keys to return to the ACRN serial console.
|
|
|
|
.. code-block:: console
|
|
:emphasize-lines: 14,31
|
|
|
|
ACRN Hypervisor
|
|
calibrate_tsc, tsc_khz=1094400
|
|
[21017289us][cpu=0][sev=2][seq=1]:HV version 0.3-unstable-2018-11-08 12:41:24-ef974d1a-dirty DBG (daily tag:acrn-2018w45.3-140000p) build by clear, start time 20997424us
|
|
[21034127us][cpu=0][sev=2][seq=2]:API version 1.0
|
|
[21039218us][cpu=0][sev=2][seq=3]:Detect processor: Intel(R) Pentium(R) CPU N4200 @ 1.10GHz
|
|
[21048422us][cpu=0][sev=2][seq=4]:hardware support HV
|
|
[21053897us][cpu=0][sev=1][seq=5]:SECURITY WARNING!!!!!!
|
|
[21059672us][cpu=0][sev=1][seq=6]:Please apply the latest CPU uCode patch!
|
|
[21074487us][cpu=0][sev=2][seq=28]:Start VM1
|
|
[21074488us][cpu=3][sev=2][seq=29]:Start VM2
|
|
[21885195us][cpu=0][sev=3][seq=34]:vlapic: Start Secondary VCPU1 for VM[1]...
|
|
[21889889us][cpu=3][sev=3][seq=35]:vlapic: Start Secondary VCPU1 for VM[2]...
|
|
ACRN:\>
|
|
ACRN:\>sos_console 1
|
|
|
|
----- Entering Guest 1 Shell -----
|
|
[ 1.997439] systemd[1]: Listening on Network Service Netlink Socket.
|
|
[ OK ] Listening on Network Service Netlink Socket.
|
|
[ 1.999347] systemd[1]: Created slice system-serial\x2dgetty.slice.
|
|
[ OK ] Created slice system-serial\x2dgetty.slice.
|
|
[ OK ] Listening on Journal Socket (/dev/log).
|
|
...
|
|
clr-932c8a3012ec4dc6af53790b7afbf6ba login: root
|
|
Password:
|
|
root@clr-932c8a3012ec4dc6af53790b7afbf6ba ~ # lspci
|
|
00:00.0 Host bridge: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series Host Bridge (rev 0b)
|
|
00:01.0 SATA controller: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series SATA AHCI Controller (rev 0b)
|
|
root@clr-932c8a3012ec4dc6af53790b7afbf6ba ~ #
|
|
|
|
---Entering ACRN SHELL---
|
|
ACRN:\>sos_console 2
|
|
|
|
----- Entering Guest 2 Shell -----
|
|
[ 1.490122] usb 1-4: new full-speed USB device number 2 using xhci_hcd
|
|
[ 1.621311] usb 1-4: not running at top speed; connect to a high speed hub
|
|
[ 1.627824] usb 1-4: New USB device found, idVendor=058f, idProduct=6387, bcdDevice= 1.01
|
|
[ 1.628438] usb 1-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
|
|
...
|
|
clr-2e8082cd4fc24d57a3c2d3db43368d36 login: root
|
|
Password:
|
|
root@clr-2e8082cd4fc24d57a3c2d3db43368d36 ~ # lspci
|
|
00:00.0 Host bridge: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series Host Bridge (rev 0b)
|
|
00:01.0 USB controller: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series USB xHCI (rev 0b)
|
|
00:02.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)
|
|
root@clr-2e8082cd4fc24d57a3c2d3db43368d36 ~ #
|