diff --git a/doc/developer-guides/hld/hld-hypervisor.rst b/doc/developer-guides/hld/hld-hypervisor.rst index 57b343d21..bb435dd9c 100644 --- a/doc/developer-guides/hld/hld-hypervisor.rst +++ b/doc/developer-guides/hld/hld-hypervisor.rst @@ -23,3 +23,4 @@ Hypervisor high-level design Console, Shell, and vUART Hypercall / VHM upcall Compile-time configuration + RDT support, CAT diff --git a/doc/developer-guides/hld/hv-rdt.rst b/doc/developer-guides/hld/hv-rdt.rst new file mode 100644 index 000000000..fac751e20 --- /dev/null +++ b/doc/developer-guides/hld/hv-rdt.rst @@ -0,0 +1,47 @@ +.. _hv_rdt: + +RDT Allocation Feature Supported by Hypervisor +############################################## + +The hypervisor allows to use RDT (Resource Director Technology) allocation features to optimize performance of VMs. There are 2 sub-features: CAT (Cache Allocation Technology) and MBA(Memory Bandwidth Allocation), CAT is for cache resources and MBA is for memory bandwidth resources. Code and Data Prioritization (CDP) is an extension of CAT. Only CAT is enabled due to the feature availability on ACRN supported platform. In ACRN, the CAT is configured via the "VM-Configuration", the resources allocated for VMs are determined in the VM configuration. + +CAT Support in ACRN +******************* + +Introduction of CAT Capabilities +================================ + +On a platform which supports CAT, each CPU can mask last-level-cache (LLC) with a cache mask, the masked cache ways cannot be evicted by this CPU. In terms of SDM, please see chapter 17, volume 3, CAT capabilities are enumerated via CPUID, and configured via MSR registers, these are: + +* CPUID.10H contains CAT capabilities, such as cache mask type(L2/L3), mask bit + length, number of masks. + +* Cache masks are set in IA32_type_MASK_n, each one of these MSRs can + hold a cache mask. They are shared by CPUs who share the same LLC; CPU sets + the RMID field of its IA32_PQR_ASSOC MSR with class-of-service (CLOS) ID, to + select a cache mask to take effect. + +Objective of CAT +================ + +CAT feature in hypervisor can isolate cache for a VM from other VMs. It can also isolate the cache usage between VMX root mode and VMX non-root mode. Generally, certain cache resources will be allocated for the RT VMs in order to reduce the performance interference through the shared cache access from the neighbour VMs. + +CAT Workflow +============= + +The hypervisor enumerates CAT capabilities and setup cache mask arrays; It also sets up CLOS for VMs and hypervisor itself per the "vm configuration". + +* The CAT capabilities are enumerated on boot-strap processor (BSP), at the + PCPU pre-initialize stage. The global data structure cat_cap_info holds the + result. +* If CAT is supported, then setup cache masks array on all APs, at the PCPU + post-initialize stage. The mask values are written to IA32_type_MASK_n. In + fact, for CPUs which share LLC, they share the same IA32_type_MASK_n MSRs too, + only need to do that on one CPU of them. The hypervisor does not detect + hierarchy of LLCs. +* If CAT is supported. The CLOS of a VM will be stored into its vCPU + msr_store_area data structure guest part. It will be loaded to + MSR IA32_PQR_ASSOC at each VM entry. +* If CAT is supported, The CLOS of hypervisor is stored for all VMs, in their + vCPU msr_store_area data structure host part. It will be loaded to MSR + IA32_PQR_ASSOC at each VM exit.