From 279a3995b6d60e2796e56491e144f9f1a07fcdba Mon Sep 17 00:00:00 2001
From: Yuan Liu <yuan1.liu@intel.com>
Date: Tue, 9 Jun 2020 13:40:59 +0800
Subject: [PATCH] doc: add inter-vm communication example

Add the example for two post-launched VMs communication

Signed-off-by: Yuan Liu <yuan1.liu@intel.com>
---
 doc/developer-guides/hld/ivshmem-hld.rst | 55 ++++++++++++++++++++++++
 1 file changed, 55 insertions(+)

diff --git a/doc/developer-guides/hld/ivshmem-hld.rst b/doc/developer-guides/hld/ivshmem-hld.rst
index 4af8945d9..4c225d7b7 100644
--- a/doc/developer-guides/hld/ivshmem-hld.rst
+++ b/doc/developer-guides/hld/ivshmem-hld.rst
@@ -111,3 +111,58 @@ where
 
 -  ``shm_size`` - Specify a shared memory size. The two communicating
    VMs must define the same size.
+
+
+Inter-VM Communication Example
+******************************
+
+Using inter-vm communication between two linux-based post-launched VMs (VM1 and
+VM2) as an example.
+
+Firstly, we need to add a new virtual PCI device for both of VMs, the device type
+is ``ivshmem``, shared memory name is test and shared memory size is 4096 bytes.
+
+.. note:: both VMs should have same shared memory name and size
+
+- VM1 Launch Script Sample::
+
+	acrn-dm -A -m $mem_size -s 0:0,hostbridge
+	-s 2,pci-gvt -G "$2"
+	-s 5,virtio-console,@stdio:stdio_port
+	-s 6,virtio-hyper_dmabuf
+	-s 3,virtio-blk,/home/clear/uos/uos1.img
+	-s 4,virtio-net,tap0
+	``-s 6,ivshmem,test,4096``
+	-s 7,virtio-rnd
+	--ovmf /usr/share/acrn/bios/OVMF.fd
+	$vm_name
+
+
+- VM2 Launch Script Sample::
+
+	acrn-dm -A -m $mem_size -s 0:0,hostbridge
+	-s 2,pci-gvt -G "$2"
+	-s 3,virtio-blk,/home/clear/uos/uos2.img
+	-s 4,virtio-net,tap0
+	``-s 5,ivshmem,test,4096``
+	--ovmf /usr/share/acrn/bios/OVMF.fd
+	$vm_name
+
+Secondly, boot two VMs and use ``lspci`` to check the virtual device is ready for each VM.
+
+-  For VM1, it is ``00:06.0 RAM memory: Red Hat, Inc. Inter-VM shared memory (rev 01)``
+-  For VM2, it is ``00:05.0 RAM memory: Red Hat, Inc. Inter-VM shared memory (rev 01)``
+
+Thirdly, using below commands to probe the device::
+
+  sudo modprobe uio
+  sudo modprobe uio_pci_generic
+  sudo echo "1af4 1110" > /sys/bus/pci/drivers/uio_pci_generic/new_id
+
+Finally, user application can get shared memory base address from ``ivshmem`` device
+BAR resource (``/sys/class/uio/uioX/device/resource2``) and shared memory size from
+``ivshmem`` device config resource (``/sys/class/uio/uioX/device/config``).
+
+.. note:: the X in uioX is a number that can be retrieved using the ``ls`` command.
+	VM1 can use ``ls -lh /sys/bus/pci/devices/0000\:00\:06.0/uio`` and VM2 can use
+	``ls -lh /sys/bus/pci/devices/0000\:00\:05.0/uio``.