Splitting the definitions of a post-launched VM into two files, namely the
scenario XML and launch XML, introduces duplicated field in both files and
leads to a high probability of having inconsistencies between them (see
issue #7156 as an example). Further more, this split has also adds much
complexity to the configurator which has to either hide some of the items
from user interfaces or synchronize different fields upon changes.
The advantage of the split, however, is not widely adopted. Having a
separate XML capturing the VM definition tweakable in the service VM at
runtime seems to give users more flexibility to redefine a VM without
recompiling the hypervisor. But that is not a common practice in the
industry segment; instead it is preferred to have a static scenario
definition to make sure that all resources are allocated carefully in a
fixed manner in order for better determinism.
As a result, this patch merges the fields in launch XMLs into the schema of
scenario XMLs. Some fields are post-launched VM specific and thus added,
while the others have similar items in scenario XMLs today.
The launch script generator is also updated accordingly to generate launch
scripts from the new scenario XMLs (which now contain the same amount of
information as previous launch XMLs).
Tracked-On: #6690
Signed-off-by: Junjie Mao <junjie.mao@intel.com>
A shared memory region can be provided either by the hypervisor or by the
device model. Before recent schema changes this is telled by the "hv:/" or
"dm:/" prefix.
This patch adds another node under an IVSHMEM region to represent the
provider, following the practice that information in the old-school
encodings is split and put as separate XML nodes.
Tracked-On: #6690
Signed-off-by: Junjie Mao <junjie.mao@intel.com>
This patch drops a few config items which are no longer needed, including:
- vm.os_config.name
- vm.UEFI_OS_LOADER_NAME
- vm.pci_dev_num
Tracked-On: #6690
Signed-off-by: Junjie Mao <junjie.mao@intel.com>
As is recommended by UX/DX reviews, the per-VM console virtual UART is now
limited to the following choices:
- Disabled
- a COM port from COM1 to COM4
- PCI based
This patch converts the schema of scenario XMLs to integrate this
recommendation and add logic in the scenario upgrader to migrate data from
old scenario XMLs.
v1 -> v2:
* Update the static allocators and C source transformers according to the
new console vUART config item.
Tracked-On: #6690
Signed-off-by: Junjie Mao <junjie.mao@intel.com>
Instead of using a Boolean variable indicating whether a build is for debug
or release, it is more intuitive to specify the build types as "debug" or
"release".
This patch converts the config item RELEASE to BUILD_TYPE which takes
"debug" or "release" as of now.
The generated header and makefile still uses RELEASE, and the command line
option RELEASE=<y or n> is also preserved.
Tracked-On: #6690
Signed-off-by: Junjie Mao <junjie.mao@intel.com>
We have redesign the vuart and the UI for user, so the config tool
should change the schema and xform for the new xml, then change the
static_allocators to alloc irq and io_port for new connection.
This patch add a new vuart connection type and change the xforms to
adapter the new type.
Tracked-On: #6690
Reviewed-by: Junjie Mao <junjie.mao@intel.com>
Signed-off-by: Chenli Wei <chenli.wei@linux.intel.com>
The concept of guest_flags is hard to understand for users.
So turn guest_flags into several parameters in config tool
user interface, list as below:
GUEST_FLAG_LAPIC_PASSTHROUGH ---> lapic_passthrough
GUEST_FLAG_IO_COMPLETION_POLLING ---> io_completion_polling
GUEST_FLAG_VCAT_ENABLED ---> virtual_cat_support
GUEST_FLAG_SECURE_WORLD_ENABLED ---> secure_world_support
GUEST_FLAG_HIDE_MTRR ---> hide_mtrr_support
GUEST_FLAG_NVMX_ENABLED ---> nested_virtualization_support
GUEST_FLAG_SECURITY_VM ---> security_vm
GUEST_FLAG_RT ---> vm_type(RTVM)
GUEST_FLAG_TEE ---> vm_type(TEE_VM)
GUEST_FLAG_REE ---> vm_type(REE_VM)
In addition, HV global parameter NVMX_ENABLE is removed
from user interface, when config tool detects more than
one VM with nested_virtualization_support, NVMX_ENABLE is
assigned as 'y' automatically.
v1->v2:
*Rebase on the latest xml schema checking change
*Remove "all rights reserved" from the license header in guest_flags.py
v2->v3:
*Change the name of the new config items to CAPITAL_CASE style
*Combine guest flag policy to an XPATH in guest_flags.py
*Use count() to directly deduce NVMX_ENABLED in config_common.xsl and
update `boolean-by-key-value` to process 'true'
v3->v4:
*Change the name of the new config items to lower_case style
*Change guest_flag_node to allocation_vm_node in guest_flags.py
*Separate value case and key case for boolean-by-key-value
Tracked-On: #6690
Signed-off-by: hangliu1 <hang1.liu@linux.intel.com>
Reviewed-by: Junjie Mao <junjie.mao@intel.com>
Remove the logic to parse SERIAL_CONSOLE and append to bootargs. Specify the console in bootargs directly.
Tracked-On: #7127
Signed-off-by: Yang,Yu-chu <yu-chu.yang@intel.com>
Reviewed-by: Junjie Mao <junjie.mao@intel.com>
Acked-by: Anthony Xu <anthony.xu@intel.com>
This patch includes:
1.add load_order(PRE_LAUNCHED_VM/SERVICE_VM/POST_LAUNCHED_VM) parameter
2.change vm_type parameter to values as RTVM, STANDARD_VM, TEE, REE
TEE and REE are hide in UI.
3.deduce vm severity in vm_configuration from vm_type and load_order
This patch not includes:
change for scenario_config and functions called by scenario_config about checking
v2->v3:
*Refine template load_order
v1->v2:
*Change variable name from vm_type to load_order
*Change LoadOptionType to LoadOrderType
*Change VMOptionsType to VMType
*Add TEE_VM/REE_VM description
*Refine acrn:is-pre-launched-vm
Tracked-On: #6690
Signed-off-by: hangliu1 <hang1.liu@linux.intel.com>
Reviewed-by: Junjie Mao <junjie.mao@intel.com>
Since PR #7113 has landed, we also remove PLATFORM_RAM_SIZE in the related
Python code, schema and all existing scenario XML files.
Tracked-On: #6690
Signed-off-by: Kunhui-Li <kunhuix.li@intel.com>
According to the new design of DX, implement ivshmem configuration
and generate hv related files.
Tracked-On: #6690
Signed-off-by: Yuanyuan Zhao <yuanyuan.zhao@linux.intel.com>
Reviewed-by: Junjie Mao <junjie.mao@intel.com>
Manually specifying MAX_IOAPIC_NUM and MAX_IOAPIC_LINES in a scenario XML
is still usable as a workaround on boards without IOAPIC (e.g. emulated
ones) or when the board inspector does not collect IOAPIC information
correctly.
This patch takes user-specified values of those items if
provided. Otherwise the automatically calculated values will be used
instead, as is done today.
Tracked-On: #6986
Signed-off-by: Junjie Mao <junjie.mao@intel.com>
Today the config.h/config.mk generators do not put a newline after an entry
without a value, which causes the generated files ill-formed.
This patch fixes the issue by putting a newline at the end of an entry
unconditionally.
Tracked-On: #6355
Signed-off-by: Junjie Mao <junjie.mao@intel.com>
In XSL a string variable can be nul, the empty string or a non-empty
string. While `$var != ''` ensures that `var` is a non-empty string,
`$var = ''` is true only when `var` holds the empty string. In other words,
`$var = ''` and `$var != ''` can be both false if `var` evaluates to nul.
As a result, the config.h/config.mk generating scripts does not use the
given default value if the explicit value is nul.
This patch fixes the incorrect comparison in the XSL scripts that handles
default values, so that the default value behaves as expected.
Tracked-On: #6355
Signed-off-by: Junjie Mao <junjie.mao@intel.com>
While the terms to refer to CPU topology levels are originally borrowed
from MSR listings in SDMs (i.e. thread, core, die and package), we now
align the topology information with the CPUID topology enumeration
leaf (which does not have a "package" level) in the
implementation. However, some hard-coded level types are not updated
accordingly, leading to strangely organized nodes in the generated board
XML.
This patch refines the CPU extractor of the board inspector by removing
such hard code. Also the XPATH counting the total number of threads is
refined to tolerate the variance of reported intermediate levels.
Tracked-On: #6689
Signed-off-by: Junjie Mao <junjie.mao@intel.com>
1. remove the board_private tag in the schema and all existing scenario XML files,
and remove the related value check about board_private.rootfs and bootargs.
2. merge board_private.rootfs and board_private.bootargs to os_config.bootargs.
and no change to the related contents of the .c/.h files except the order of
define SERVICE_VM_ROOTFS.
3. update the schema to make os_config.bootargs configurable for service VM in UI.
Tracked-On: #6690
Signed-off-by: Kunhui-Li <kunhuix.li@intel.com>
Generate MAX_IOAPIC_NUM and MAX_IOAPIC_LINE based on board.xml automatically.
Tracked-On: #6986
Signed-off-by: Yang,Yu-chu <yu-chu.yang@intel.com>
Reviewed-by: Junjie Mao <junjie.mao@intel.com>
Acked-by: Anthony Xu <anthony.xu@intel.com>
Extract the max pci bus number from board information and generate the
common configuration macro ACFG_MAX_PCI_BUS_NUM automatically.
Tracked-On: #6942
Signed-off-by: Yang,Yu-chu <yu-chu.yang@intel.com>
This patch adds an option CONFIG_KEEP_IRQ_DISABLED to hv (default n) and
config-tool so that when this option is 'y', all interrupts in hv root
mode will be permanently disabled.
With this option to be 'y', all interrupts received in root mode will be
handled in external interrupt vmexit after next VM entry. The postpone
latency is negligible. This new configuration is a requirement from x86
TEE's secure/non-secure interrupt flow support. Many race conditions can be
avoided when keeping IRQ off.
v5:
Rename CONFIG_ACRN_KEEP_IRQ_DISABLED to CONFIG_KEEP_IRQ_DISABLED
v4:
Change CPU_IRQ_ENABLE/DISABLE to
CPU_IRQ_ENABLE_ON_CONFIG/DISABLE_ON_CONFIG and guard them using
CONFIG_ACRN_KEEP_IRQ_DISABLED
v3:
CONFIG_ACRN_DISABLE_INTERRUPT -> CONFIG_ACRN_KEEP_IRQ_DISABLED
Add more comment in commit message
Tracked-On: #6571
Signed-off-by: Yifan Liu <yifan1.liu@intel.com>
Reviewed-by: Wang, Yu1 <yu1.wang@intel.com>
Acked-by: Anthony Xu <anthony.xu@intel.com>
1. remove LOG_DESTINATION in the related python code, schema and
all existing scenario XML files.
2. for MEM_LOGLEVEL, NPK_LOGLEVEL and CONSOLE_LOGLEVEL,
update the loglevel range to [0, 5] from [0, 6].
Tracked-On: #6934
Signed-off-by: Kunhui-Li <kunhuix.li@intel.com>
Acked-by: Anthony Xu <anthony.xu@intel.com>
remove LOG_BUF_SIZE in the related python code, schema and
all existing scenario XML files.
Tracked-On: #6690
Signed-off-by: Kunhui-Li <kunhuix.li@intel.com>
Acked-by: Anthony Xu <anthony.xu@intel.com>
Because it is not configurable, config tool might not need to
define DM_OWNED_GUEST_FLAG_MASK. So we remove it in tool.
Tracked-On: #6366
Signed-off-by: Kunhui-Li <kunhuix.li@intel.com>
GUEST_FLAG_SECURITY_VM only for pre-launch vm, should not be
programmed by device model, so we remove it in DM_OWNED_GUEST_FLAG_MASK.
Tracked-On: #6366
Signed-off-by: Kunhui-Li <kunhuix.li@intel.com>
Get the integrated GPU bdf by looking for device from board.xml and extract the
BDF from device address.
Tracked-On: #6855
Signed-off-by: Yang,Yu-chu <yu-chu.yang@intel.com>
remove CONFIG_MAX_IR_ENTRIES the related python code, schema and
all existing scenario XML files since PR #6809 have changed it.
Tracked-On: #6745
Signed-off-by: Kunhui-Li <kunhuix.li@intel.com>
remove LOW_RAM_SIZE in the related python code, schema and
all existing scenario XMLs because PR #6791 have removed it.
Tracked-On: #6805
Signed-off-by: Kunhui-Li <kunhuix.li@intel.com>
Add a configuration to support companion VM.
Tracked-On: #6571
Signed-off-by: Jie Deng <jie.deng@intel.com>
Reviewed-by: Wang, Yu1 <yu1.wang@intel.com>
1. remove CONFIG_KATA_VM, CONFIG_MAX_KATA_VM_NUM and KATA_VM
related code.
2. remove KATA_VM in scenario xml files.
3. remove KATA vm type in UI.
4. remove UUID DB.
5. remove uuid in the launch script generation logic.
6. use vmname to identify a vm. (eg: <name>ACRN_SOS_VM</name>)
add the check logic to ensure vm name is a must-set item, not duplicated,
and the string length is [1-15] and the whitespace is not allowed.
7. enlarge the max VM num to 32.
Tracked-On: #6685
Signed-off-by: Kunhui-Li <kunhuix.li@intel.com>
Rename function name to get-common-clos-count and refine the
calculation. Some platforms have odd clos counts. Using "floor" to get
integer numbers for macros.
Tracked-On: #6755
Signed-off-by: Yang,Yu-chu <yu-chu.yang@intel.com>
The config of MAX_EFI_MMAP_ENTRIES has been removed in hypervisor, so remove
this config in config_tools;
Tracked-On: #6442
Signed-off-by: Victor Sun <victor.sun@intel.com>
Rename SOS_VM type to SERVICE_VM
rename UOS to User VM in XML description
rename uos_thread_pid to user_vm_thread_pid
rename devname_uos to devname_user_vm
rename uosid to user_vmid
rename UOS_ACK to USER_VM_ACK
rename SOS_VM_CONFIG_CPU_AFFINITY to SERVICE_VM_CONFIG_CPU_AFFINITY
rename SOS_COM to SERVICE_VM_COM
rename SOS_UART1_VALID_NUM" to SERVICE_VM_UART1_VALID_NUM
rename SOS_BOOTARGS_DIFF to SERVICE_VM_BOOTARGS_DIFF
rename uos to user_vm in launch script and xml
Tracked-On: #6744
Signed-off-by: Liu Long <long.liu@linux.intel.com>
Reviewed-by: Geoffroy Van Cutsem <geoffroy.vancutsem@intel.com>
Rename SOS_VM_NUM to SERVICE_VM_NUM.
rename SOS_SOCKET_PORT to SERVICE_VM_SOCKET_PORT.
rename PROCESS_RUN_IN_SOS to PROCESS_RUN_IN_SERVICE_VM.
rename PCI_DEV_TYPE_SOSEMUL to PCI_DEV_TYPE_SERVICE_VM_EMUL.
rename SHUTDOWN_REQ_FROM_SOS to SHUTDOWN_REQ_FROM_SERVICE_VM.
rename PROCESS_RUN_IN_SOS to PROCESS_RUN_IN_SERVICE_VM.
rename SHUTDOWN_REQ_FROM_UOS to SHUTDOWN_REQ_FROM_USER_VM.
rename UOS_SOCKET_PORT to USER_VM_SOCKET_PORT.
rename SOS_CONSOLE to SERVICE_VM_OS_CONSOLE.
rename SOS_LCS_SOCK to SERVICE_VM_LCS_SOCK.
rename SOS_VM_BOOTARGS to SERVICE_VM_OS_BOOTARGS.
rename SOS_ROOTFS to SERVICE_VM_ROOTFS.
rename SOS_IDLE to SERVICE_VM_IDLE.
rename SEVERITY_SOS to SEVERITY_SERVICE_VM.
rename SOS_VM_UUID to SERVICE_VM_UUID.
rename SOS_REQ to SERVICE_VM_REQ.
rename RTCT_NATIVE_FILE_PATH_IN_SOS to RTCT_NATIVE_FILE_PATH_IN_SERVICE_VM.
rename CBC_REQ_T_UOS_ACTIVE to CBC_REQ_T_USER_VM_ACTIVE.
rename CBC_REQ_T_UOS_INACTIVE to CBC_REQ_T_USER_VM_INACTIV.
rename uos_active to user_vm_active.
Tracked-On: #6744
Signed-off-by: Liu Long <long.liu@linux.intel.com>
Reviewed-by: Geoffroy Van Cutsem <geoffroy.vancutsem@intel.com>
For vCAT, it may need to store more than MAX_VCPUS_PER_VM of closids,
change clos in vm_config.h to a pointer to accommodate this situation
Rename clos to pclosids
pclosids now is a pointer to an array of physical CLOSIDs that is defined
in vm_configurations.c by vmconfig. The number of elements in the array
must be equal to the value given by num_pclosids
Add max_type_pcbm (type: l2 or l3) to struct acrn_vm_config, which stores a bitmask
that selects/covers all the physical cache ways assigned to the VM
Change vmsr.c to accommodate this amended data structure
Change the config-tools to generate vm_configurations.c, and fill in the num_closids
and clos pointers based on the information from the scenario file.
Now vm_configurations.c.xsl generates all the clos related code so remove the same
code from misc_cfg.h.xsl.
Examples:
Scenario file:
<RDT>
<RDT_ENABLED>y</RDT_ENABLED>
<CDP_ENABLED>n</CDP_ENABLED>
<VCAT_ENABLED>y</VCAT_ENABLED>
<CLOS_MASK>0x7ff</CLOS_MASK>
<CLOS_MASK>0x7ff</CLOS_MASK>
<CLOS_MASK>0x7ff</CLOS_MASK>
<CLOS_MASK>0xff800</CLOS_MASK>
<CLOS_MASK>0xff800</CLOS_MASK>
<CLOS_MASK>0xff800</CLOS_MASK>
<CLOS_MASK>0xff800</CLOS_MASK>
<CLOS_MASK>0xff800</CLOS_MASK>
/RDT>
<vm id="0">
<guest_flags>
<guest_flag>GUEST_FLAG_VCAT_ENABLED</guest_flag>
</guest_flags>
<clos>
<vcpu_clos>3</vcpu_clos>
<vcpu_clos>4</vcpu_clos>
<vcpu_clos>5</vcpu_clos>
<vcpu_clos>6</vcpu_clos>
<vcpu_clos>7</vcpu_clos>
</clos>
</vm>
<vm id="1">
<clos>
<vcpu_clos>1</vcpu_clos>
<vcpu_clos>2</vcpu_clos>
</clos>
</vm>
vm_configurations.c (generated by config-tools) with the above vCAT config:
static uint16_t vm0_vcpu_clos[5U] = {3U, 4U, 5U, 6U, 7U};
static uint16_t vm1_vcpu_clos[2U] = {1U, 2U};
struct acrn_vm_config vm_configs[CONFIG_MAX_VM_NUM] = {
{
.guest_flags = (GUEST_FLAG_VCAT_ENABLED),
.pclosids = vm0_vcpu_clos,
.num_pclosids = 5U,
.max_l3_pcbm = 0xff800U,
},
{
.pclosids = vm1_vcpu_clos,
.num_pclosids = 2U,
},
};
Tracked-On: #5917
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
Add the VCAT_ENABLED element to RDTType so that user can enable/disable vCAT globally
Add the GUEST_FLAG_VCAT_ENABLED guest flag to enable/disable vCAT per-VM.
Currently we have the following per-VM clos element in scenario file for RDT use:
<clos>
<vcpu_clos>0</vcpu_clos>
<vcpu_clos>0</vcpu_clos>
</clos>
When the GUEST_FLAG_VCAT_ENABLED guest flag is not specified, clos is for RDT use,
vcpu_clos is per-CPU and it configures each CPU in VMs to a desired CLOS ID.
When the GUEST_FLAG_VCAT_ENABLED guest flag is specified, vCAT is enabled for this VM,
clos is for vCAT use, vcpu_clos is not per-CPU anymore in this case, just a list of
physical CLOSIDs (minimum 2) that are assigned to VMs for vCAT use. Each vcpu_clos
will be mapped to a virtual CLOSID, the first vcpu_clos is mapped to virtual CLOSID
0 and the second is mapped to virtual CLOSID 1, etc
Add xs:assert to prevent any problems with invalid configuration data for vCAT:
If any GUEST_FLAG_VCAT_ENABLED guest flag is specified, both RDT_ENABLED and VCAT_ENABLED
must be 'y'
If VCAT_ENABLED is 'y', RDT_ENABLED must be 'y' and CDP_ENABLED must be 'n'
For a vCAT VM, vcpu_clos cannot be set to CLOSID 0, CLOSID 0 is reserved to be used by hypervisor
For a vCAT VM, number of clos/vcpu_clos elements must be greater than 1
For a vCAT VM, each clos/vcpu_clos must be less than L2/L3 COS_MAX
For a vCAT VM, its clos/vcpu_clos elements cannot contain duplicate values
There should not be any CLOS IDs overlap between a vCAT VM and any other VMs
Tracked-On: #5917
Signed-off-by: dongshen <dongsheng.x.zhang@intel.com>
Acked-by: Eddie Dong <eddie.dong@intel.com>
1. remove HV_RAM_SIZE and CONFIG_HV_RAM_SIZE in the related python
code, schema and all existing scenario XMLs because PR 6664 has
changed it in HV side.
2. set HV_RAM_START default value to 2M.
Tracked-On: #6663
Signed-off-by: Kunhui-Li <kunhuix.li@intel.com>
Expand the capacity of legacy vuarts per VM. This change is applied to
manual scenario xml editing only.
A SOS VM can choose io port 0x3F8, 0x2F8, 0x3E8, 0x2E8 by selecting
SOS_COM1_BASE, SOS_COM2_BASE, SOS_COM3_BASE, SOS_COM4_BASE respectively.
Non SOS VM can choose io port 0x3F8, 0x2F8, 0x3E8, 0x2E8 by selecting
COM1_BASE, COM2_BASE, COM3_BASE, COM4_BASE respectively.
For any type of VM, selecting "CONFIG_COM_BASE" allows configuration tool
to pick an available io port from hardcoded list:
['0xA000', '0xA010', '0xA020', '0xA030', '0xA040', '0xA050', '0xA060', '0xA070']
A SOS VM can choose irq 4 by selecting SOS_COM1_IRQ and SOS_COM3_IRQ, and choose irq 3 by selecting SOS_COM2_IRQ and SOS_COM4_IRQ.
Non SOS VM can choose irq 4 by selecting COM1_IRQ and COM3_IRQ, and choose irq 3 by selecting COM2_IRQ and COM4_IRQ.
For SOS VM, selecting "CONFIG_COM_IRQ" allows configuration tool
to pick an available irq based on AVAILABLE_IRQ_INFO. For non SOS VM, it
will allocate an available irq from [1, 15].
Tracked-On: #6652
Signed-off-by: Yang,Yu-chu <yu-chu.yang@intel.com>
This patch adds a new priority based scheduler to support
vCPU scheduling based on their pre-configured priorities.
A vCPU can be running only if there is no higher priority
vCPU running on the same pCPU.
Tracked-On: #6571
Signed-off-by: Jie Deng <jie.deng@intel.com>
Simplify the statement by TPM2 enablement detection and present TPM2 device.
Remove the function and move the statement to misc_cfg.h.xsl
Tracked-On: #6562
Signed-off-by: Yang,Yu-chu <yu-chu.yang@intel.com>
The whitespaces between delimiter saperated elemnts of a list causes
acrn:get-common-clos-max and acrn:find-list-min returns unexpected
numbers.
Translate the whitespaces to nothing and rewrite acrn:find-list-min and
acrn:get-common-clos-max.
Tracked-On: #6515
Signed-off-by: Yang,Yu-chu <yu-chu.yang@intel.com>
This patch renames the GUEST_FLAG_TPM2_FIXUP to
GUEST_FLAG_SECURITY_VM.
v2 -> v3:
The "FIXUP" suffix is removed.
Tracked-On: #6320
Signed-off-by: Yifan Liu <yifan1.liu@intel.com>
MAXIMUM_PA_WIDTH will be calculated from board information.
Tracked-On: #6357
Signed-off-by: Liang Yi <yi.liang@intel.com>
Signed-off-by: Junjie Mao <junjie.mao@intel.com>
It is used to specify the maximum number of EFI memmap entries.
On some platforms, like Tiger Lake, the number of EFI memmap entries
becomes 268 when the BIOS settings are changed.
The current value of MAX_EFI_MMAP_ENTRIES (256) defined in hypervisor
is not big enough to cover such cases.
As the number of EFI memmap entries depends on the platforms and the
BIOS settings, this patch introduces a new entry MAX_EFI_MMAP_ENTRIES
in configurations so that it can be adjusted for different cases.
Tracked-On: #6442
Signed-off-by: Shiqing Gao <shiqing.gao@intel.com>
Junjie Mao