From e8c20e7a63bd06e6e4fa8186fdf50d75918aec22 Mon Sep 17 00:00:00 2001 From: Tao Yuhong Date: Tue, 17 Nov 2020 06:06:02 -0500 Subject: [PATCH] Doc: pci-vuart-guide update tutorials/vuart_configuration.rst, add pci-vuart guide Signed-off-by: Tao Yuhong Signed-off-by: David B. Kinder --- doc/tutorials/vuart_configuration.rst | 192 ++++++++++++++++++++++++++ 1 file changed, 192 insertions(+) diff --git a/doc/tutorials/vuart_configuration.rst b/doc/tutorials/vuart_configuration.rst index fb69f3ac9..1af51c725 100644 --- a/doc/tutorials/vuart_configuration.rst +++ b/doc/tutorials/vuart_configuration.rst @@ -48,6 +48,8 @@ Console enable list | | (vUART enable) | (vUART enable) | RTVM | | +-----------------+-----------------------+--------------------+----------------+----------------+ +.. _how-to-configure-a-console-port: + How to configure a console port =============================== @@ -71,6 +73,8 @@ Example: .irq = COM1_IRQ, }, +.. _how-to-configure-a-communication-port: + How to configure a communication port ===================================== @@ -214,3 +218,191 @@ started, as shown in the diagram below: the hypervisor is not sufficient. Currently, we recommend that you use the configuration in the figure 3 data flow. This may be refined in the future. + +Use PCI-vUART +############# + +PCI Interface of ACRN vUART +=========================== + +When you set :ref:`vuart[0] and vuart[1] `, the ACRN +hypervisor emulates virtual legacy serial devices (I/O port and IRQ) for +VMs. So ``vuart[0]`` and ``vuart[1]`` are legacy vUARTs. ACRN +hypervisor can also emulate virtual PCI serial devices (BDF, MMIO +registers and MSIX capability). These virtual PCI serial devices are +called PCI-vUART, and have an advantage in device enumeration for the +guest OS. It is easy to add new PCI-vUART ports to a VM. + +.. _index-of-vuart: + +Index of vUART +============== + +ACRN hypervisor supports PCI-vUARTs and legacy vUARTs as ACRN vUARTs. +Each vUART port has its own ``vuart_idx``. ACRN hypervisor supports up +to 8 vUART for each VM, from ``vuart_idx=0`` to ``vuart_idx=7``. +Suppose we use vUART0 for a port with ``vuart_idx=0``, vUART1 for +``vuart_idx=1``, and so on. + +Please pay attention to these points: + +* vUART0 is the console port, vUART1-vUART7 are inter-VM communication ports. +* Each communication port must set the connection to another communication vUART port of another VM. +* When legacy ``vuart[0]`` is available, it is vUART0. A PCI-vUART can't + be vUART0 unless ``vuart[0]`` is not set. +* When legacy ``vuart[1]`` is available, it is vUART1. PCI-vUART can't + be vUART1 unless ``vuart[1]`` is not set. + +Setup ACRN vUART Using Configuration Tools +========================================== + +When you set up ACRN VM configurations with PCI-vUART, it is better to +use the ACRN configuration tools because of all the PCI resources required: BDF number, +address and size of mmio registers, and address and size of MSIX entry +tables. These settings can't conflict with another PCI device. Furthermore, +whether PCI-vUART can use ``vuart_idx=0`` and ``vuart_idx=1`` depends on legacy +vUART settings. Configuration tools will override your settings in +:ref:`How to Configure a Console Port ` +and :ref:`How to Configure a Communication Port +`. + +You can configure both Legacy vUART and PCI-vUART in +``./misc/vm_configs/xmls/config-xmls//.xml``. For +example, if VM0 has a legacy vUART0 and a PCI-vUART1, VM1 has no legacy +vUART but has a PCI-vUART0 and a PCI-vUART1, VM0's PCI-vUART1 and VM1's +PCI-vUART1 are connected to each other. You should configure then like this: + +.. code-block:: none + + + + VUART_LEGACY_PIO /* vuart[0] is console port */ + COM1_BASE /* vuart[0] is used */ + COM1_IRQ + + + VUART_LEGACY_PIO + INVALID_COM_BASE /* vuart[1] is not used */ + + + INVALID_PCI_BASE /* PCI-vUART0 can't be used, because vuart[0] */ + + + PCI_VUART /* PCI-vUART1 is communication port, connect to vUART1 of VM1 */ + 1 + 1 + + + + + + VUART_LEGACY_PIO + INVALID_COM_BASE /* vuart[0] is not used */ + + + VUART_LEGACY_PIO + INVALID_COM_BASE /* vuart[1] is not used */ + + + PCI_VUART /* PCI-vUART0 is console port */ + + + PCI_VUART /* PCI-vUART1 is communication port, connect to vUART1 of VM0 */ + 0 + 1 + + + +The ACRN vUART related XML fields: + + - ``id`` in ````, value of ``vuart_idx``, ``id=0`` is for + legacy ``vuart[0]`` configuration, ``id=1`` is for ``vuart[1]``. + - ``type`` in ````, type is always ``VUART_LEGACY_PIO`` + for legacy vUART. + - ``base`` in ````, if use the legacy vUART port, set + COM1_BASE for ``vuart[0]``, set ``COM2_BASE`` for ``vuart[1]``. + ``INVALID_COM_BASE`` means do not use the legacy vUART port. + - ``irq`` in ````, if you use the legacy vUART port, set + ``COM1_IRQ`` for ``vuart[0]``, set ``COM2_IRQ`` for ``vuart[1]``. + - ``id`` in ```` and ````, + ``vuart_idx`` for PCI-vUART + - ``base`` in ```` and ````, + ``PCI_VUART`` means use this PCI-vUART, ``INVALID_PCI_BASE`` means do + not use this PCI-VUART. + - ``target_vm_id`` and ``target_uart_id``, connection settings for this + vUART port. + +Run the command to build ACRN with this XML configuration file:: + + make BOARD_FILE=$PWD/misc/acrn-config/xmls/board-xmls/.xml \ + SCENARIO_FILE=$PWD/misc/acrn-config/xmls/config-xmls//.xml + +The configuration tools will test your settings, and check :ref:`vUART +Rules ` for compilation issue. After compiling, you can find +``./misc/vm_configs/scenarios///pci_dev.c`` has been +changed by the configuration tools based on the XML settings, something like: + +.. code-block:: none + + struct acrn_vm_pci_dev_config vm0_pci_devs[] = { + { + .emu_type = PCI_DEV_TYPE_HVEMUL, + .vbdf.bits = {.b = 0x00U, .d = 0x05U, .f = 0x00U}, + .vdev_ops = &vmcs9900_ops, + .vbar_base[0] = 0x80003000, + .vbar_base[1] = 0x80004000, + .vuart_idx = 1, /* PCI-vUART1 of VM0 */ + .t_vuart.vm_id = 1U, /* connected to VM1's vUART1 */ + .t_vuart.vuart_id = 1U, + }, + } + +This struct shows a PCI-vUART with ``vuart_idx=1``, ``BDF 00:05.0``, its +a PCI-vUART1 of +VM0, and it is connected to VM1's vUART1 port. When VM0 wants to communicate +with VM1, it can use ``/dev/ttyS*``, the character device file of +VM0's PCI-vUART1. Usually, legacy ``vuart[0]`` is ``ttyS0`` in VM, and +``vuart[1]`` is ``ttyS1``. So we hope PCI-vUART0 is ``ttyS0``, +PCI-VUART1 is ``ttyS1`` and so on through +PCI-vUART7 is ``ttyS7``, but that is not true. We can use BDF to identify +PCI-vUART in VM. + +If you run ``dmesg | grep tty`` at a VM shell, you may see: + +.. code-block:: none + + [ 1.276891] 0000:00:05.0: ttyS4 at MMIO 0xa1414000 (irq = 124, base_baud = 115200) is a 16550A + +We know for VM0 guest OS, ``ttyS4`` has BDF 00:05.0 and is PCI-vUART1. +VM0 can communicate with VM1 by reading from or writing to ``/dev/ttyS4``. + +If VM0 and VM1 are pre-launched VMs, or Service VM, ACRN hypervisor will +create PCI-vUART virtual devices automatically. For post-launched VMs, +created by ``acrn-dm``, an additional ``acrn-dm`` option is needed +to create a PCI-vUART virtual device: + +.. code-block:: none + + -s ,uart,vuart_idx: + +Kernel Config for Legacy vUART +============================== + +When ACRN hypervisor passthroughs a local APIC to a VM, there is IRQ +injection issue for legacy vUART. The kernel driver must work in +polling mode to avoid the problem. The VM kernel should have these config +symbols set: + +.. code-block:: none + + CONFIG_SERIAL_8250_EXTENDED=y + CONFIG_SERIAL_8250_DETECT_IRQ=y + +Kernel Cmdline for PCI-vUART console +==================================== + +When an ACRN VM does not have a legacy ``vuart[0]`` but has a +PCI-vUART0, you can use PCI-vUART0 for VM serial input/output. Check +which tty has the BDF of PCI-vUART0; usually it is not ``/dev/ttyS0``. +For example, if ``/dev/ttyS4`` is PCI-vUART0, you must set +``console=/dev/ttyS4`` in the kernel cmdline.