WIXLIB

28Administrative tasks 27528.1The boot loader program 27528.2Sparse image files and disk space 27628.3Migrating Xen VM Guest systems 277Detecting CPU features 278 Preparing block devices formigrations 279 Migrating VM Guest systems 28028.4Monitoring Xen 280Monitor Xen with xentop 280 Additional tools 28128.529Providing host information for VM Guest systems 282XenStore: configuration database shared betweendomains 28429.1Introduction 28429.2File system interface 284XenStore commands 285 /vm 285 /local/domain/ domid 2873030.1Xen HA with remote storage 28930.2Xen HA with local storage 29030.3Xen HA and private bridges 29131VxivXen as a high-availability virtualization host 289Xen: converting a paravirtual (PV) guest into a fullyvirtual (FV/HVM) guest 292MANAGING VIRTUAL MACHINES WITH QEMU 29632QEMU overview 29733Setting up a KVM VM Host Server 29833.1CPU support for virtualization 29833.2Required software 298Virtualization Guide

33.3KVM host-specific features 300Using the host storage with virtio-scsi 300 Accelerated networkingwith vhost-net 301 Scaling network performance with multiqueue virtionet 302 VFIO: secure direct access to devices 303 VirtFS: sharingdirectories between host and guests 305 KSM: sharing memory pagesbetween guests 30634Guest installation 30834.1Basic installation with qemu-system-ARCH 30834.2Managing disk images with qemu-img 309General information on qemu-img invocation 310 Creating, converting, andchecking disk images 311 Managing snapshots of virtual machines withqemu-img 316 Manipulate disk images effectively 31835Running virtual machines with qemu-systemARCH 32335.1Basic qemu-system-ARCH invocation 32335.2General qemu-system-ARCH options 324Basic virtual hardware 325 Storing and reading configuration of virtualdevices 327 Guest real-time clock 32835.3Using devices in QEMU 328Block devices 329 Graphic devices and display options 334 USBdevices 336 Character devices 33835.4Networking in QEMU 340Defining a network interface card 340 User-modenetworking 341 Bridged networking 34335.5Viewing a VM Guest with VNC 346Secure VNC connections 3483636.1xvVirtual machine administration using QEMUmonitor 351Accessing monitor console 351Virtualization Guide

36.2Getting information about the guest system 35236.3Changing VNC password 35436.4Managing devices 35536.5Controlling keyboard and mouse 35636.6Changing available memory 35636.7Dumping virtual machine memory 35736.8Managing virtual machine snapshots 35836.9Suspending and resuming virtual machine execution 35936.10Live migration 35936.11QMP - QEMU machine protocol 361Access QMP via standard input/output 361 Access QMP viatelnet 362 Access QMP via Unix socket 363 Access QMP via libvirt'svirsh command 364VITROUBLESHOOTING 36537Integrated help and package documentation 36638Gathering system information and logs 36738.1libvirt log controls 367Glossary 369xviAVirtual machine drivers 379BConfiguring GPU Pass-Through for NVIDIA cards 380B.1Introduction 380B.2Prerequisites 380Virtualization Guide

B.3Configuring the host 380Verify the host environment 380 Enable IOMMU 381 Blacklist theNouveau driver 382 Configure VFIO and isolate the GPU used for passthrough 382 Load the VFIO driver 382 Disable MSR for MicrosoftWindows guests 383 Install and enable UEFI firmware 383 Reboot thehost machine 384B.4Configuring the guest 384Requirements for the guest configuration 385 Install the graphic carddriver 385CC.1XM, XL toolstacks, and the libvirt framework 388Xen toolstacks 388Upgrading from xend/xm to xl/libxl 388 XL design 389 Checklist beforeupgrade 390C.2Import Xen domain configuration into libvirt 390C.3Differences between the xm and xl applications 392Notation conventions 392 New global options 393 Unchangedoptions 393 Removed options 397 Changed options 400 Newoptions 414C.4External links 415C.5Saving a Xen guest configuration in an xm compatible format 416DxviiGNU licenses 417Virtualization Guide

Preface1 Available documentationOnline documentationThe online documentation for this product is available at https://documentation.suse.com/#sles. Browse or download the documentation in various formats.Find the online documentation for other products at https://documentation.suse.com/ .Note: Latest updatesThe latest documentation updates are usually available in the English version of thedocumentation.Release notesFor release notes, see https://www.suse.com/releasenotes/ .In your systemFor offline use, nd documentation in your installed system under /usr/share/doc . Manycommands are also described in detail in their manual pages. To view them, run man ,followed by a specific command name. If the man command is not installed on your system,install it with sudo zypper install man .2 Improving the documentationYour feedback and contributions to this documentation are welcome. The following channelsfor giving feedback are available:Service requests and supportFor services and support options available for your product, see https://www.suse.com/support/.To open a service request, you need a SUSE subscription registered at SUSE CustomerCenter. Go to https://scc.suse.com/support/requests , log in, and click Create New.Bug reportsxviiiAvailable documentationSLES 15 SP4

Report issues with the documentation at https://bugzilla.suse.com/ . To simplify thisprocess, you can use the Report an issue link in the HTML version of this document. Position the cursor in the affected sentence and in the Give feedback section of the right-handnavigation panel, click Report an issue. This preselects the right product and category inBugzilla and adds a link to the current section. You can start typing your bug report rightaway. A Bugzilla account is required.ContributionsTo contribute to this documentation, use the Edit source document link in the HTML versionof this document. Position the cursor in the affected sentence and in the Give feedbacksection of the right-hand navigation panel, click Edit source document. This takes you to thesource code on GitHub, where you can open a pull request. A GitHub account is required.Note: Edit source document only available for EnglishThe Edit source document links are only available for the English version of eachdocument. For all other languages, use the Report an issue link as described above.For more information about the documentation environment used for this documentation,see the repository's README at docMailYou can also report errors and send feedback concerning the documentation to doc-team@suse.com . Include the document title, the product version, and the publication dateof the document. Additionally, include the relevant section number and title (or providethe URL), and provide a concise description of the problem.3 Documentation conventionsThe following notices and typographical conventions are used in this documentation:/etc/passwd : directory names and le namesPLACEHOLDER : replace PLACEHOLDER with the actual valuePATH : the environment variable PATHls , --help : commands, options, and parametersuser : users or groupsxixDocumentation conventionsSLES 15 SP4

package name : name of a packageAlt,Alt– F1 : a key to press or a key combination; keys are shown in uppercase as ona keyboardFile, File Save As: menu items, buttonsAMD/IntelThis paragraph is only relevant for the AMD64/Intel 64 architecture. The ar-rows mark the beginning and the end of the text block.IBM Z, POWERThis paragraph is only relevant for the architectures IBM Z and POWER .The arrows mark the beginning and the end of the text block.Dancing Penguins (Chapter Penguins, Another Manual): This is a reference to a chapter inanother manual.Commands that must be run with root privileges. Often you can also prefix these commands with the sudo command to run them as non-privileged user.# command sudo commandCommands that can be run by non-privileged users. commandNoticesWarning: Warning noticeVital information you must be aware of before proceeding. Warns you about securityissues, potential loss of data, damage to hardware, or physical hazards.Important: Important noticeImportant information you should be aware of before proceeding.Note: Note noticeAdditional information, for example about differences in software versions.xxDocumentation conventionsSLES 15 SP4

Tip: Tip noticeHelpful information, like a guideline or a piece of practical advice.4 SupportFind the support statement for SUSE Linux Enterprise Server and general information abouttechnology previews below. For details about the product lifecycle, see Book “Upgrade Guide”,Chapter 2 “Lifecycle and support”.If you are entitled to support, nd details on how to collect information for a support ticket inBook “Administration Guide”, Chapter 47 “Gathering system information for support”.4.1Support statement for SUSE Linux Enterprise ServerTo receive support, you need an appropriate subscription with SUSE. To view the specific supportofferings available to you, go to https://www.suse.com/support/and select your product.The support levels are defined as follows:L1Problem determination, which means technical support designed to provide compatibilityinformation, usage support, ongoing maintenance, information gathering, and basic troubleshooting using available documentation.L2Problem isolation, which means technical support designed to analyze data, reproducecustomer problems, isolate problem areas, and provide a resolution for problems not resolved by Level 1, or prepare for Level 3.L3Problem resolution, which means technical support designed to resolve problems by en-gaging engineering to resolve product defects which have been identified by Level 2 Support.xxiSupportSLES 15 SP4

For contracted customers and partners, SUSE Linux Enterprise Server is delivered with L3 support for all packages, except for the following:technology previews.sound, graphics, fonts, and artwork.packages that require an additional customer contract.some packages shipped as part of the module Workstation Extension are L2-supported only.packages with names ending in -devel (containing header les and similar developerresources) will only be supported together with their main packages.SUSE will only support the usage of original packages. That is, packages that are unchangedand not recompiled.4.2Technology previewsTechnology previews are packages, stacks, or features delivered by SUSE to provide glimpsesinto upcoming innovations. The previews are included for your convenience to give you thechance to test new technologies within your environment. We would appreciate your feedback!If you test a technology preview, contact your SUSE representative and let them know aboutyour experience and use cases. Your input is helpful for future development.However, technology previews come with the following limitations:Technology previews are still in development. Therefore, they may be functionally incomplete, unstable, or in other ways not suitable for production use.Technology previews are not supported.Technology previews may only be available for specific hardware architectures.Details and functionality of technology previews are subject to change. As a result, upgrading to subsequent releases of a technology preview may be impossible and require afresh installation.Technology previews can be dropped at any time. For example, if SUSE discovers that apreview does not meet the customer or market needs, or does not prove to comply withenterprise standards. SUSE does not commit to providing a supported version of such technologies in the future.xxiiTechnology previewsSLES 15 SP4

For an overview of technology previews shipped with your product, see the release notes gy previewsSLES 15 SP4

I Introduction1Virtualization technology 22Virtualization scenarios 63Introduction to Xen virtualization 84Introduction to KVM virtualization 115Virtualization tools 136Installation of virtualization components 177Virtualization limits and support 25

1 Virtualization technologyVirtualization is a technology that provides a way for a machine (Host) to run an-other operating system (guest virtual machines) on top of the host operating system.1.1 OverviewSUSE Linux Enterprise Server includes the latest open source virtualization technologies, Xenand KVM. With these hypervisors, SUSE Linux Enterprise Server can be used to provision, de-provision, install, monitor and manage multiple virtual machines (VM Guests) on a single physical system (for more information see Hypervisor). SUSE Linux Enterprise Server can create vir-tual machines running both modified, highly tuned, paravirtualized operating systems and fullyvirtualized unmodified operating systems.The primary component of the operating system that enables virtualization is a hypervisor (orvirtual machine manager), which is a layer of software that runs directly on server hardware.It controls platform resources, sharing them among multiple VM Guests and their operatingsystems by presenting virtualized hardware interfaces to each VM Guest.SUSE Linux Enterprise is an enterprise-class Linux server operating system that offers two typesof hypervisors: Xen and KVM.SUSE Linux Enterprise Server with Xen or KVM acts as a virtualization host server (VHS) thatsupports VM Guests with its own guest operating systems. The SUSE VM Guest architectureconsists of a hypervisor and management components that constitute the VHS, which runs manyapplication-hosting VM Guests.In Xen, the management components run in a privileged VM Guest often called Dom0. In KVM,where the Linux kernel acts as the hypervisor, the management components run directly onthe VHS.1.2 Virtualization benefitsVirtualization brings a lot of advantages while providing the same service as a hardware server.2OverviewSLES 15 SP4

First, it reduces the cost of your infrastructure. Servers are mainly used to provide a service toa customer, and a virtualized operating system can provide the same service, with:Less hardware: You can run several operating system on one host, so all hardware maintenance will be reduced.Less power/cooling: Less hardware means you do not need to invest more in electric power,backup power, and cooling if you need more service.Save space: Your data center space will be saved because you do not need more hardwareservers (less servers than service running).Less management: Using a VM Guest simplifies the administration of your infrastructure.Agility and productivity: Virtualization provides migration capabilities, live migration andsnapshots. These features reduce downtime, and bring an easy way to move your servicefrom one place to another without any service interruption.1.3 Virtualization modesGuest operating systems are hosted on virtual machines in either full virtualization (FV) modeor paravirtual (PV) mode. Each virtualization mode has advantages and disadvantages.Full virtualization mode lets virtual machines run unmodified operating systems, such asWindows* Server 2003. It can use either Binary Translation or hardware-assisted virtualiza-tion technology, such as AMD* Virtualization or Intel* Virtualization Technology. Usinghardware assistance allows for better performance on processors that support it.Some guest operating systems hosted in full virtualization mode can be configured to usedrivers from the SUSE Virtual Machine Drivers Pack (VMDP) instead of drivers originatingfrom the operating system. Running virtual machine drivers improves performance dra-matically on guest operating systems, such as Windows Server 2003. For more information, see Appendix A, Virtual machine drivers.To be able to run under paravirtual mode, guest operating systems usually need to bemodified for the virtualization environment. However, operating systems running in paravirtual mode have better performance than those running under full virtualization.Operating systems currently modified to run in paravirtual mode are called paravirtualizedoperating systems and include SUSE Linux Enterprise Server.3Virtualization modesSLES 15 SP4

1.4 I/O virtualizationVM Guests not only share CPU and memory resources of the host system, but also the I/O sub-system. Because software I/O virtualization techniques deliver less performance than bare metal, hardware solutions that deliver almost “native” performance have been developed recently.SUSE Linux Enterprise Server supports the following I/O virtualization techniques:Full virtualizationFully Virtualized (FV) drivers emulate widely supported real devices, which can be usedwith an existing driver in the VM Guest. The guest is also called Hardware Virtual Machine(HVM). Since the physical device on the VM Host Server may differ from the emulated one,the hypervisor needs to process all I/O operations before handing them over to the physicaldevice. Therefore all I/O operations need to traverse two software layers, a process thatnot only significantly impacts I/O performance, but also consumes CPU time.ParavirtualizationParavirtualization (PV) allows direct communication between the hypervisor and the VMGuest. With less overhead involved, performance is much better than with full virtualiza-tion. However, paravirtualization requires either the guest operating system to be modifiedto support the paravirtualization API or paravirtualized drivers. See Section 7.4.1, “Availabilityof paravirtualized drivers” for a list of guest operating systems supporting paravirtualization.PVHVMThis type of virtualization enhances HVM (see Full virtualization) with paravirtualized (PV)drivers, and PV interrupt and timer handling.VFIOVFIO stands for Virtual Function I/O and is a new user-level driver framework for Linux. Itreplaces the traditional KVM PCI Pass-Through device assignment. The VFIO driver exposesdirect device access to user space in a secure memory (IOMMU) protected environment.With VFIO, a VM Guest can directly access hardware devices on the VM Host Server (passthrough), avoiding performance issues caused by emulation in performance critical paths.This method does not allow to share devices—each device can only be assigned to a singleVM Guest. VFIO needs to be supported by the VM Host Server CPU, chipset and the BIOS/EFI.4I/O virtualizationSLES 15 SP4

Compared to the legacy KVM PCI device assignment, VFIO has the following advantages:Resour

1 Virtualization technology Virtualization is a technology that provides a way for a machine (Host) to run an-other operating system (guest virtual machines) on top of the host operating system. 1.1 Overview SUSE Linux Enterprise Server includes the latest open source virtualization technologies, Xen and KVM.