WIXLIB

30VXen: converting a paravirtual (PV) guest into a fullyvirtual (FV/HVM) guest 268MANAGING VIRTUAL MACHINES WITH QEMU 27231QEMU overview 27332Setting up a KVM VM Host Server 27432.1CPU support for virtualization 27432.2Required software 27432.3KVM host-specific features 276Using the host storage with virtio-scsi 276 Accelerated networkingwith vhost-net 277 Scaling network performance with multiqueue virtionet 278 VFIO: secure direct access to devices 279 VirtFS: sharingdirectories between host and guests 281 KSM: sharing memory pagesbetween guests 28233Guest installation 28433.1Basic installation with qemu-system-ARCH 28433.2Managing disk images with qemu-img 285General information on qemu-img invocation 286 Creating, converting, andchecking disk images 287 Managing snapshots of virtual machines withqemu-img 292 Manipulate disk images effectively 29434Running virtual machines with qemu-systemARCH 29934.1Basic qemu-system-ARCH invocation 29934.2General qemu-system-ARCH options 299Basic virtual hardware 301 Storing and reading configuration of virtualdevices 302 Guest real-time clock 30334.3Using devices in QEMU 303Block devices 304 Graphic devices and display options 309 USBdevices 311 Character devices 313xiiiVirtualization Guide

34.4Networking in QEMU 315Defining a network interface card 315 User-modenetworking 316 Bridged networking 31834.5Viewing a VM Guest with VNC 321Secure VNC connections 32335Virtual machine administration using QEMUmonitor 32635.1Accessing monitor console 32635.2Getting information about the guest system 32735.3Changing VNC password 32935.4Managing devices 33035.5Controlling keyboard and mouse 33135.6Changing available memory 33135.7Dumping virtual machine memory 33235.8Managing virtual machine snapshots 33335.9Suspending and resuming virtual machine execution 33435.10Live migration 33435.11QMP - QEMU machine protocol 336Access QMP via standard input/output 336 Access QMP viatelnet 337 Access QMP via Unix socket 338 Access QMP via libvirt'svirsh command 339Glossary 340AA.1A.2xivConfiguring GPU Pass-Through for NVIDIA cards 350Introduction 350Prerequisites 350Virtualization Guide

A.3Configuring the host 350Verify the host environment 350 Enable IOMMU 351 Blacklist theNouveau driver 352 Configure VFIO and isolate the GPU used for passthrough 352 Load the VFIO driver 352 Disable MSR for MicrosoftWindows guests 353 Install and enable UEFI firmware 353 Reboot thehost machine 354A.4Configuring the guest 354Requirements for the guest configuration 355 Install the graphic carddriver 355BxvGNU licenses 358Virtualization Guide

Preface1 Available documentationOnline documentationThe online documentation for this product is available at http://doc.opensuse.org/ .Browse or download the documentation in various formats.Note: Latest updatesThe latest documentation updates are usually available in the English version of thedocumentation.In your systemFor o ine use, nd documentation in your installed system under /usr/share/doc . Manycommands are also described in detail in their manual pages. To view them, run man ,followed by a speci c command name. If the man command is not installed on your system,install it with sudo zypper install man .2 Improving the documentationYour feedback and contributions to this documentation are welcome! Several channels areavailable:Bug reportsReport issues with the documentation at https://bugzilla.opensuse.org/ . To simplify thisprocess, you can use the Report Documentation Bug links next to headlines in the HTMLversion of this document. These preselect the right product and category in Bugzilla andadd a link to the current section. You can start typing your bug report right away. ABugzilla account is required.ContributionsTo contribute to this documentation, use the Edit Source links next to headlines in theHTML version of this document. They take you to the source code on GitHub, where youcan open a pull request. A GitHub account is required.xviAvailable documentationopenSUSE Leap 15.3

usedforthisdocumentation, see the repository's README E.adoc).MailAlternatively, you can report errors and send feedback concerning the documentation todoc-team@suse.com . Make sure to include the document title, the product version andthe publication date of the documentation. Refer to the relevant section number and title(or include the URL) and provide a concise description of the problem.HelpIf you need further help on openSUSE Leap, see https://en.opensuse.org/Portal:Support .3 Documentation conventionsThe following notices and typographical conventions are used in this documentation:/etc/passwd : directory names and le namesPLACEHOLDER : replace PLACEHOLDER with the actual valuePATH : the environment variable PATHls , --help : commands, options, and parametersuser : users or groupspackage name : name of a packageAlt,Alt– F1 : a key to press or a key combination; keys are shown in uppercase as ona keyboardFile, File Save As: menu items, buttonsDancing Penguins (Chapter Penguins, Another Manual): This is a reference to a chapter inanother manual.Commands that must be run with root privileges. Often you can also pre x thesecommands with the sudo command to run them as non-privileged user.root # commandtux sudo commandxviiDocumentation conventionsopenSUSE Leap 15.3

Commands that can be run by non-privileged users.tux commandNoticesWarning: Warning noticeVital information you must be aware of before proceeding. Warns you about securityissues, potential loss of data, damage to hardware, or physical hazards.Important: Important noticeImportant information you should be aware of before proceeding.Note: Note noticeAdditional information, for example about di erences in software versions.Tip: Tip noticeHelpful information, like a guideline or a piece of practical advice.xviiiDocumentation conventionsopenSUSE Leap 15.3

I Introduction1Virtualization technology 22Virtualization scenarios 63Introduction to Xen virtualization 84Introduction to KVM virtualization 115Virtualization tools 136Installation of virtualization components 18

1 Virtualization technologyVirtualization is a technology that provides a way for a machine (Host) to runanother operating system (guest virtual machines) on top of the host operatingsystem.1.1 OverviewopenSUSE Leap includes the latest open source virtualization technologies, Xen and KVM. Withthese hypervisors, openSUSE Leap can be used to provision, de-provision, install, monitor andmanage multiple virtual machines (VM Guests) on a single physical system (for more informationsee Hypervisor). openSUSE Leap can create virtual machines running both modi ed, highly tuned,paravirtualized operating systems and fully virtualized unmodi ed operating systems.The primary component of the operating system that enables virtualization is a hypervisor (orvirtual machine manager), which is a layer of software that runs directly on server hardware.It controls platform resources, sharing them among multiple VM Guests and their operatingsystems by presenting virtualized hardware interfaces to each VM Guest.openSUSE is a Linux server operating system that o ers two types of hypervisors: Xen and KVM.openSUSE Leap with Xen or KVM acts as a virtualization host server (VHS) that supports VMGuests with its own guest operating systems. The SUSE VM Guest architecture consists of ahypervisor and management components that constitute the VHS, which runs many applicationhosting VM Guests.In Xen, the management components run in a privileged VM Guest often called Dom0. In KVM,where the Linux kernel acts as the hypervisor, the management components run directly onthe VHS.1.2 Virtualization benefitsVirtualization brings a lot of advantages while providing the same service as a hardware server.2OverviewopenSUSE Leap 15.3

First, it reduces the cost of your infrastructure. Servers are mainly used to provide a service toa customer, and a virtualized operating system can provide the same service, with:Less hardware: You can run several operating system on one host, so all hardwaremaintenance will be reduced.Less power/cooling: Less hardware means you do not need to invest more in electric power,backup power, and cooling if you need more service.Save space: Your data center space will be saved because you do not need more hardwareservers (less servers than service running).Less management: Using a VM Guest simpli es the administration of your infrastructure.Agility and productivity: Virtualization provides migration capabilities, live migration andsnapshots. These features reduce downtime, and bring an easy way to move your servicefrom one place to another without any service interruption.1.3 Virtualization modesGuest operating systems are hosted on virtual machines in either full virtualization (FV) modeor paravirtual (PV) mode. Each virtualization mode has advantages and disadvantages.Full virtualization mode lets virtual machines run unmodi ed operating systems,such as Windows* Server 2003. It can use either Binary Translation or hardware-assisted virtualization technology, such as AMD* Virtualization or Intel* VirtualizationTechnology. Using hardware assistance allows for better performance on processors thatsupport it.To be able to run under paravirtual mode, guest operating systems usually need tobe modi ed for the virtualization environment. However, operating systems running inparavirtual mode have better performance than those running under full virtualization.Operating systems currently modi ed to run in paravirtual mode are called paravirtualizedoperating systems and include openSUSE Leap and NetWare 6.5 SP8.3Virtualization modesopenSUSE Leap 15.3

1.4 I/O virtualizationVM Guests not only share CPU and memory resources of the host system, but also the I/O subsystem. Because software I/O virtualization techniques deliver less performance thanbare metal, hardware solutions that deliver almost “native” performance have been developedrecently. openSUSE Leap supports the following I/O virtualization techniques:Full virtualizationFully Virtualized (FV) drivers emulate widely supported real devices, which can be usedwith an existing driver in the VM Guest. The guest is also called Hardware Virtual Machine(HVM). Since the physical device on the VM Host Server may di er from the emulated one,the hypervisor needs to process all I/O operations before handing them over to the physicaldevice. Therefore all I/O operations need to traverse two software layers, a process thatnot only signi cantly impacts I/O performance, but also consumes CPU time.ParavirtualizationParavirtualization (PV) allows direct communication between the hypervisor and theVM Guest. With less overhead involved, performance is much better than with fullvirtualization. However, paravirtualization requires either the guest operating system tobe modi ed to support the paravirtualization API or paravirtualized drivers.PVHVMThis type of virtualization enhances HVM (see Full virtualization) with paravirtualized (PV)drivers, and PV interrupt and timer handling.VFIOVFIO stands for Virtual Function I/O and is a new user-level driver framework for Linux. Itreplaces the traditional KVM PCI Pass-Through device assignment. The VFIO driver exposesdirect device access to user space in a secure memory (IOMMU) protected environment.With VFIO, a VM Guest can directly access hardware devices on the VM Host Server (passthrough), avoiding performance issues caused by emulation in performance critical paths.This method does not allow to share devices—each device can only be assigned to a singleVM Guest. VFIO needs to be supported by the VM Host Server CPU, chipset and the BIOS/EFI.Compared to the legacy KVM PCI device assignment, VFIO has the following advantages:Resource access is compatible with secure boot.Device is isolated and its memory access protected.4I/O virtualizationopenSUSE Leap 15.3

O ers a user space device driver with more exible device ownership model.Is independent of KVM technology, and not bound to x86 architecture only.In openSUSE Leap the USB and PCI pass-through methods of device assignment areconsidered deprecated and are superseded by the VFIO model.SR-IOVThe latest I/O virtualization technique, Single Root I/O Virtualization SR-IOV combines thebene ts of the aforementioned techniques—performance and the ability to share a devicewith several VM Guests. SR-IOV requires special I/O devices, that are capable of replicatingresources so they appear as multiple separate devices. Each such “pseudo” device can bedirectly used by a single guest. However, for network cards for example the number ofconcurrent queues that can be used is limited, potentially reducing performance for theVM Guest compared to paravirtualized drivers. On the VM Host Server, SR-IOV must besupported by the I/O device, the CPU and chipset, the BIOS/EFI and the hypervisor—forsetup instructions see Section 13.12, “Assigning a host PCI device to a VM Guest”.Important: Requirements for VFIO and SR-IOVTo be able to use the VFIO and SR-IOV features, the VM Host Server needs to ful ll thefollowing requirements:IOMMU needs to be enabled in the BIOS/EFI.For Intel CPUs, the kernel parameter intel iommu on needs to be provided on thekernel command line. For more information, see Book “Reference”, Chapter 12 “Theboot loader GRUB 2”, Section 12.3.3.2 “Kernel Parameters tab”.The VFIO infrastructure needs to be available. This can be achieved by loading thekernel module vfio pci . For more information, see Book “Reference”, Chapter 10“The systemd daemon”, Section 10.6.4 “Loading kernel modules”.5I/O virtualizationopenSUSE Leap 15.3

2 Virtualization scenariosVirtualization provides several useful capabilities to your organization: moree cient hardware use, support for legacy software, operating system isolation, livemigration, disaster recovery, and load balancing.2.1 Server consolidationMany servers can be replaced by one big physical server, so that hardware is consolidated, andguest operating systems are converted to virtual machines. This also supports running legacysoftware on new hardware.Better usage of resources that were not running at 100%Fewer server locations neededMore e cient use of computer resources: multiple workloads on the same serverSimpli cation of data center infrastructureSimpli es moving workloads to other hosts, avoiding service downtimeFaster and agile virtual machine provisioning.Multiple guest operating systems can run on a single hostImportantServer consolidation requires special attention to the following points:Maintenance windows should be carefully plannedStorage is key: it must be able to support migration and growing disk usageYou must verify that your servers can support the additional workloads6Server consolidationopenSUSE Leap 15.3

2.2 IsolationGuest operating systems are fully isolated from the host running them. Therefore, if there areproblems inside virtual machines, the host is not harmed. Also, problems inside one VM do nota ect other VMs. No data is shared between VMs.Secure Boot can be used for VMs.KSM should be avoided. For more details on KSM, refer to KSM.Individual CPU cores can be assigned to VMs.Hyper-threading (HT) should be disabled to avoid potential security issues.VM should not share network, storage, or network hardware.Use of advanced hypervisor features such as PCI pass-through or NUMA will adverselya ect VM migration capabilities.Use of paravirtualization and virtio drivers will generally improve VM performance ande ciency.AMD provides some speci c features regarding the security of virtualization.2.3 Disaster recoveryThe hypervisor can make snapshots of VMs, enabling restoration to a known good state, or toany desired earlier state. Since Virtualized OSes are less dependent on hardware con gurationthan those running directly on bare metal, these snapshots can be restored onto di erent serverhardware so long as it is running the same hypervisor.2.4 Dynamic load balancingLive migration provides a simple way to load-balance your services across your infrastructure,by moving VMs from busy hosts to those with spare capacity, on demand.7IsolationopenSUSE Leap 15.3

3 Introduction to Xen virtualizationThis chapter introduces and explains the components and technologies you need to understandto set up and manage a Xen-based virtualization environment.3.1 Basic componentsThe basic components of a Xen-based virtualization environment are the Xen hypervisor, theDom0, any number of other VM Guests, and the tools, commands, and con guration les that letyou manage virtualization. Collectively, the physical computer running all these components iscalled a VM Host Server because together these components form a platform for hosting virtualmachines.The Xen hypervisorThe Xen hypervisor, sometimes simply called a virtual machine monitor, is an open sourcesoftware program that coordinates the low-level interaction between virtual machines andphysical hardware.The Dom0The virtual machine host environment, also called Dom0 or controlling domain, iscomposed of several components, such as:openSUSE Leap provides a graphical and a command line environment to managethe virtual machine host components and its virtual machines.NoteThe term “Dom0” refers to a special domain that provides the managementenvironment. This may be run either in graphical or in command line mode.The xl tool stack based on the xenlight library (libxl). Use it to manage Xen guestdomains.QEMU—an open source software that emulates a full computer system, including aprocessor and various peripherals. It provides the ability to host operating systemsin both full virtualization or paravirtualization mode.Xen-based virtual machines8Basic componentsopenSUSE Leap 15.3

A Xen-based virtual machine, also called a VM Guest or DomU , consists of the followingcomponents:At least one virtual disk that contains a bootable operating system. The virtual diskcan be based on a le, partition, volume, or other type of block device.A con guration le for each guest domain. It is a text le following the syntaxdescribed in the manual page man 5 xl.conf .Several network devices, connected to the virtual network provided by thecontrolling domain.Management tools, commands, and configuration filesThere is a combination of GUI tools, commands, and con guration les to help you manageand customize your virtualization environment.3.2 Xen virtualization architectureThe following graphic depicts a virtual machine host w

File system pass-through feature is incompatible with migration. The VM Host Server and VM Guest need to have proper timekeeping installed. See Chapter 18, VM Guest clock settings. No physical devices can be passed from host to guest. Live migration is currently not supported when using devices with PCI pass-through or SR-IOV. If live migration .