MOS Deployment Guide - Mirantis

Transcription

MOS Deployment Guideversion latest

MOS Deployment GuideMOS UssuriContentsCopyright notice1Preface2About this documentation set2Intended audience2Technology Preview support scope3Documentation history3Conventions3Introduction5Plan the deployment6Provision a Container Cloud bare metal management cluster7Configure host operating system8Configure kernel modules8Configure networking8Configure networking on a server with three NICsConfigure networking on a server with five NICsConfigure hosts files for a deployment with TF91215Create a MOS managed cluster16Add bare metal hosts to the MOS cluster17Add machines to the MOS cluster18Deploy a Ceph cluster19Deploy OpenStack21Deploy an OpenStack cluster21Advanced OpenStack configuration (optional)26Enable LVM ephemeral storage27Enable DPDK with OVS29Enable SR-IOV with OVS31Advanced configuration for OpenStack compute nodes33Enable huge pages for OpenStack34Configure CPU isolation for an instance35Configure custom CPU topologies36Configure PCI passthrough for guests37 2021, Mirantis Inc.Page i

MOS Deployment GuideAccess OpenStack after deploymentMOS Ussuri38Configure DNS to access OpenStack38Access your OpenStack environment41Access OpenStack using the Kubernetes built-in admin CLI42Access an OpenStack environment through Horizon42Access OpenStack through CLI from your local machine42Troubleshoot an OpenStack deployment44Debugging the HelmBundle controller44Verify the Helm releases statuses44Verify the status of a HelmBundle release45Debugging the OpenStack ControllerThe openstack-operator pod is missingDebugging the OsDpl CR454646The osdpl has DEPLOYED false46Some pods are stuck in Init46Some HelmBundles are not present47Deploy Tungsten Fabric48Tungsten Fabric deployment prerequisites48Deploy Tungsten Fabric49Advanced Tungsten Fabric configuration (optional)51Enable DPDK for Tungsten Fabric52Enable SR-IOV for Tungsten Fabric54Specify custom vRouter settings56Access the Tungsten Fabric web UI57Troubleshoot the Tungsten Fabric deployment58Enable debug logs for the Tungsten Fabric services58Troubleshoot access to the Tungsten Fabric web UI59 2021, Mirantis Inc.Page ii

MOS Deployment GuideMOS UssuriCopyright notice2021 Mirantis, Inc. All rights reserved.This product is protected by U.S. and international copyright and intellectual property laws. Nopart of this publication may be reproduced in any written, electronic, recording, or photocopyingform without written permission of Mirantis, Inc.Mirantis, Inc. reserves the right to modify the content of this document at any time without priornotice. Functionality described in the document may not be available at the moment. Thedocument contains the latest information at the time of publication.Mirantis, Inc. and the Mirantis Logo are trademarks of Mirantis, Inc. and/or its affiliates in theUnited States an other countries. Third party trademarks, service marks, and names mentionedin this document are the properties of their respective owners. 2021, Mirantis Inc.Page 1

MOS Deployment GuideMOS UssuriPreface About this documentation set Intended audience Technology Preview support scope Documentation history ConventionsAbout this documentation setThis documentation provides information on how to deploy and operate a Mirantis OpenStack forKubernetes (MOS) environment. The documentation is intended to help operators to understandthe core concepts of the product. The documentation provides sufficient information to deployand operate the solution.The information provided in this documentation set is being constantly improved and amendedbased on the feedback and kind requests from the consumers of MOS.The following table lists the guides included in the documentation set you are reading:Guides listGuidePurposeMOS ReferenceArchitectureLearn the fundamentals of MOS reference architecture toappropriately plan your deploymentMOS Deployment GuideDeploy a MOS environment of a preferred configuration usingsupported deployment profiles tailored to the demands of specificbusiness casesMOS Operations GuideOperate your MOS environmentMOS Release notesLearn about new features and bug fixes in the current MOS versionThe MOS documentation home page contains references to all guides included in thisdocumentation set. For your convenience, we provide all guides in HTML (default), single-pageHTML, PDF, and ePUB formats. To use the preferred format of a guide, select the required optionfrom the Formats menu next to the guide title.Intended audienceThis documentation is intended for engineers who have the basic knowledge of Linux,virtualization and containerization technologies, Kubernetes API and CLI, Helm and Helm charts,Mirantis Kubernetes Engine (MKE), and OpenStack. 2021, Mirantis Inc.Page 2

MOS Deployment GuideMOS UssuriTechnology Preview support scopeThis documentation set includes description of the Technology Preview features. A TechnologyPreview feature provide early access to upcoming product innovations, allowing customers toexperience the functionality and provide feedback during the development process. TechnologyPreview features may be privately or publicly available and neither are intended for productionuse. While Mirantis will provide support for such features through official channels, normalService Level Agreements do not apply. Customers may be supported by Mirantis CustomerSupport or Mirantis Field Support.As Mirantis considers making future iterations of Technology Preview features generallyavailable, we will attempt to resolve any issues that customers experience when using thesefeatures.During the development of a Technology Preview feature, additional components may becomeavailable to the public for testing. Because Technology Preview features are being underdevelopment, Mirantis cannot guarantee the stability of such features. As a result, if you areusing Technology Preview features, you may not be able to seamlessly upgrade to subsequentreleases of that feature. Mirantis makes no guarantees that Technology Preview features will begraduated to a generally available product release.The Mirantis Customer Success Organization may create bug reports on behalf of support casesfiled by customers. These bug reports will then be forwarded to the Mirantis Product team forpossible inclusion in a future release.Documentation historyThe following table contains the released revision of the documentation set you are reading:Release dateDescriptionNovember 05, 2020MOS GA releaseDecember 23, 2020MOS GA Update releaseMarch 01, 2021MOS 21.1ConventionsThis documentation set uses the following conventions in the HTML format:Documentation conventionsConventionDescriptionboldface fontInline CLI tools and commands, titles of the procedures andsystem response examples, table titlesmonospaced fontFiles names and paths, Helm charts parameters and their values,names of packages, nodes names and labels, and so onitalic fontInformation that distinguishes some concept or termLinksExternal links and cross-references, footnotes 2021, Mirantis Inc.Page 3

MOS Deployment GuideMOS UssuriMain menu menu itemGUI elements that include any part of interactive user interfaceand menu navigationSuperscriptSome extra, brief informationMessages of a generic meaning that may be useful for the userNoteThe Note blockInformation that prevents a user from mistakes and undesirableconsequences when following the proceduresCaution!The Caution blockWarningThe Warning blockMessages that include details that can be easily missed, butshould not be ignored by the user and are valuable beforeproceedingList of references that may be helpful for understanding of somerelated tools, concepts, and so onSeealsoThe See also blockLearn moreUsed in the Release Notes to wrap a list of internal references tothe reference architecture, deployment and operation proceduresspecific to a newly implemented product featureThe Learn moreblock 2021, Mirantis Inc.Page 4

MOS Deployment GuideMOS UssuriIntroductionMirantis OpenStack for Kubernetes (MOS) enables the operator to create, scale, update, andupgrade OpenStack deployments on Kubernetes through a declarative API.The Kubernetes built-in features, such as flexibility, scalability, and declarative resourcedefinition make MOS a robust solution. 2021, Mirantis Inc.Page 5

MOS Deployment GuideMOS UssuriPlan the deploymentThe detailed plan of any Mirantis OpenStack for Kubernetes (MOS) deployment is determined ona per-cloud basis. For the MOS reference architecture and design overview, see MOS ReferenceArchitecture.Also, read through Mirantis Container Cloud Reference Architecture: Container Cloud bare metalas a MOS managed cluster is deployed on top of a baremetal-based Container Cloudmanagement cluster.NoteOne of the industry best practices is to verify every new update or configuration change ina non-customer-facing environment before applying it to production. Therefore, Mirantisrecommends having a staging cloud, deployed and maintained along with the productionclouds. The recommendation is especially applicable to the environments that: Receive updates often and use continuous delivery. For example, any non-isolateddeployment of Mirantis Container Cloud and Mirantis OpenStack for Kubernetes(MOS). Have significant deviations from the reference architecture or third party extensionsinstalled. Are managed under the Mirantis OpsCare program. Run business-critical workloads where even the slightest application downtime isunacceptable.A typical staging cloud is a complete copy of the production environment including thehardware and software configurations, but with a bare minimum of compute and storagecapacity. 2021, Mirantis Inc.Page 6

MOS Deployment GuideMOS UssuriProvision a Container Cloud bare metalmanagement clusterThe bare metal management system enables the Infrastructure Operator to deploy ContainerCloud on a set of bare metal servers. It also enables Container Cloud to deploy MOS managedclusters on bare metal servers without a pre-provisioned operating system.To provision your bare metal management cluster, refer to Mirantis Container Cloud DeploymentGuide: Deploy a baremetal-based management cluster 2021, Mirantis Inc.Page 7

MOS Deployment GuideMOS UssuriConfigure host operating systemBefore you proceed with the OpenStack deployment, you need to configure the host operatingsystem to run OpenStack.To calculate the required number of hosts, read through Hardware requirements.Configure kernel modulesThe OpenStack Nova and Neutron services may require additional kernel modules to be loadedfor normal operation. This section instructs you on how to enable the required kernel modules.To enable KVMA kernel-based Virtual Machine (KVM) is a virtualization module in the Linux kernel that enablesthe kernel to function as a hypervisor. The KVM-based virtualization is recommended oncompute hosts to provide high performance VMs. See OpenStack Ussuri official documentation:Enable KVM for details.NoteSome systems require that you enable the VT support in the system BIOS. To enable theVT support, refer to the BIOS guides of the manufacturer of your server motherboard.Configure networkingMirantis OpenStack for Kubernetes (MOS) typically requires common/PXE, management, tenant,and storage types of networks. For the details, refer to MOS Reference Architecture: Networking.This section describes how to configure persistent networking on hosts with three and five NICsusing the Mirantis Container Cloud L2 Templates. 2021, Mirantis Inc.Page 8

MOS Deployment GuideMOS UssuriConfigure networking on a server with three NICsThis section provides recommended configuration for networking on servers with three NICs thatinclude: eno1 that is used as a PXE interface ens3f1 and ens3f2 that are used for bond0To configure networking:1. Create subnets. For example:--apiVersion: ipam.mirantis.com/v1alpha1kind: Subnetmetadata:labels:kaas.mirantis.com/provider: baremetalkaas.mirantis.com/region: region-onename: storage-backendnamespace: managed-nsspec:cidr: 10.12.0.0/24--apiVersion: ipam.mirantis.com/v1alpha1kind: Subnetmetadata:labels:kaas.mirantis.com/provider: baremetalkaas.mirantis.com/region: region-onename: storage-frontendnamespace: managed-nsspec:cidr: 10.12.1.0/242. Create the openstack-example-3nic L2 template.Caution!The bootstrapping engine automatically assigns an IP address to the PXE nic 0 NIC. Toprevent the IP duplication during updates, do not assign the IP address manually.Example template:apiVersion: ipam.mirantis.com/v1alpha1kind: L2Templatemetadata:labels: 2021, Mirantis Inc.Page 9

MOS Deployment GuideMOS Ussurikaas.mirantis.com/provider: baremetalkaas.mirantis.com/region: region-onename: openstack-example-3nicnamespace: managed-nsspec:clusterRef: managed-clusterautoIfMappingPrio:- provision- eno- ens- enpnpTemplate: version: 2ethernets:{{nic 0}}:# IMPORTANT: Do not assign an IP address here explicitly# to prevent IP duplication issues. The IP will be assigned# automatically by the bootstrapping engine.# addresses: []match:macaddress: {{mac 0}}set-name: {{nic 0}}mtu: 1500{{nic 1}}:dhcp4: falsedhcp6: falsematch:macaddress: {{mac 1}}set-name: {{nic 1}}mtu: 9000{{nic 2}}:dhcp4: falsedhcp6: falsematch:macaddress: {{mac 2}}set-name: {{nic 2}}mtu: 9000bonds:bond0:mtu: 9000interfaces:- {{nic 1}}- {{nic 2}}vlans:pr-floating:id: 403link: bond0stor-frontend: 2021, Mirantis Inc.Page 10

MOS Deployment GuideMOS Ussuriid: 404link: bond0addresses:- {{ip "stor-frontend:storage-frontend"}}mtu: 9000stor-backend:id: 405link: bond0addresses:- {{ip "stor-backend:storage-backend"}}mtu: 9000 2021, Mirantis Inc.Page 11

MOS Deployment GuideMOS UssuriConfigure networking on a server with five NICsThis section provides recommended configuration for networking on servers with five NICs thatinclude: eno1 that is used as a PXE interface ens3f1 and ens3f2 that are used for bond0 ens5f1 and ens5f2 that are used for bond1To configure networking:1. Create subnets. For example:--apiVersion: ipam.mirantis.com/v1alpha1kind: Subnetmetadata:labels:kaas.mirantis.com/provider: baremetalkaas.mirantis.com/region: region-onename: storage-backendnamespace: managed-nsspec:cidr: 10.12.0.0/24--apiVersion: ipam.mirantis.com/v1alpha1kind: Subnetmetadata:labels:kaas.mirantis.com/provider: baremetalkaas.mirantis.com/region: region-onename: storage-frontendnamespace: managed-nsspec:cidr: 10.12.1.0/24--apiVersion: ipam.mirantis.com/v1alpha1kind: Subnetmetadata:labels:kaas.mirantis.com/provider: baremetalkaas.mirantis.com/region: region-onename: tenant-tunnelnamespace: managed-nsspec:cidr: 10.12.3.0/242. Create the openstack-example-5nic L2 template. 2021, Mirantis Inc.Page 12

MOS Deployment GuideMOS UssuriCaution!The bootstrapping engine automatically assigns an IP address to the PXE nic 0 NIC. Toprevent the IP duplication during updates, do not assign the IP address manually.Example template:apiVersion: ipam.mirantis.com/v1alpha1kind: er: baremetalkaas.mirantis.com/region: region-onename: openstack-example-5nicnamespace: managed-nsspec:clusterRef: managed-clusterautoIfMappingPrio:- provision- eno- ens- enpnpTemplate: version: 2ethernets:{{nic 0}}:# IMPORTANT: Do not assign an IP address here explicitly# to prevent IP duplication issues. The IP will be assigned# automatically by the bootstrapping engine.# addresses: []match:macaddress: {{mac 0}}set-name: {{nic 0}}mtu: 1500{{nic 1}}:dhcp4: falsedhcp6: falsematch:macaddress: {{mac 1}}set-name: {{nic 1}}mtu: 9000{{nic 2}}:dhcp4: falsedhcp6: falsematch:macaddress: {{mac 2}}set-name: {{nic 2}} 2021, Mirantis Inc.Page 13

MOS Deployment GuideMOS Ussurimtu: 9000{{nic 3}}:dhcp4: falsedhcp6: falsematch:macaddress: {{mac 3}}set-name: {{nic 3}}mtu: 9000{{nic 4}}:dhcp4: falsedhcp6: falsematch:macaddress: {{mac 4}}set-name: {{nic 4}}mtu: 9000bonds:bond0:mtu: 9000interfaces:- {{nic 1}}- {{nic 2}}bond1:mtu: 9000interfaces:- {{nic 3}}- {{nic 4}}vlans:pr-floating:id: 403link: bond1stor-frontend:id: 404link: bond0addresses:- {{ip "stor-frontend:storage-frontend"}}mtu: 9000stor-backend:id: 405link: bond1addresses:- {{ip "stor-backend:storage-backend"}}mtu: 9000tenant-tunnel:id: 406link: bond1addresses:- {{ip "tenant-tunnel:tenant-tunnel"}}mtu: 9000 2021, Mirantis Inc.Page 14

MOS Deployment GuideMOS UssuriConfigure hosts files for a deployment with TFTungstenFabric services use the /etc/hosts file as a source of information for servicesconfiguration and registration in the database.Caution!For a MOS with Tungsten Fabric deployment, the proper hosts file configuration is arequirement.To configure the hosts file:On each node that will be used for running TungstenFabric services, verify that the /etc/hosts filehas an entry with a local hostname and appropriate IP address from the Management networkaccording to MOS Reference Architecture: Networking. For example:127.0.0.1 localhost management-ip node-hostname . 2021, Mirantis Inc.Page 15

MOS Deployment GuideMOS UssuriCreate a MOS managed clusterAfter you prepare the host operating system as described in Configure host operating system,proceed to creating a MOS cluster.A MOS cluster is deployed as a Container Cloud managed cluster through the Container Cloudweb UI. For a detailed procedure, refer to Mirantis Container Cloud Operations Guide: Create amanaged cluster.To properly configure StackLight, consider the recommendations described in Elasticsearch andPrometheus storage sizing.NoteOnce you have created a MOS managed cluster, some StackLight alerts may raise asfalse-positive until you deploy the MOS OpenStack environment. 2021, Mirantis Inc.Page 16

MOS Deployment GuideMOS UssuriAdd bare metal hosts to the MOS clusterAfter you create a MOS managed cluster and configure operating systems for the bare metalhosts, proceed with adding the bare metal hosts to your deployment through the MirantisContainer Cloud web UI.For the detailed procedure, refer to Mirantis Container Cloud Operations Guide: Add a bare metalhost.NoteTo calculate the required number of hosts, read through Hardware requirements. 2021, Mirantis Inc.Page 17

MOS Deployment GuideMOS UssuriAdd machines to the MOS clusterAfter you add bare metal hosts to the MOS managed cluster, create Kubernetes machines inyour cluster using the Mirantis Container Cloud web UI.For the detailed procedure, refer to Mirantis Container Cloud Operations Guide: Add a machine.When adding the machines, verify that you label the Kubernetes nodes according to theOpenStack node roles:OpenStack node rolesNoderoleDescriptionOpenStackcontrolplaneHosts the OpenStack control plane servicessuch as database, messaging, API,schedulers, conductors, L3 and L2 agents.OpenStack computeHosts the OpenStack compute servicessuch as libvirt and L2 agents.Kubernetes labelsopenstack-control-plane enabledopenstack-gateway enabledopenvswitch enabledopenstack-compute-node enabledopenvswitch enabled (for adeploymentwithOpenvSwitch as a back end fornetworking)Minimalcount3VariesNoteMOS allows collocating components on the same host, for example, Ceph OSD andOpenStack compute. For details, see Mirantis Container Cloud Operations Guide: EnableCeph tolerations and resources management. 2021, Mirantis Inc.Page 18

MOS Deployment GuideMOS UssuriDeploy a Ceph clusterThis section describes how to deploy a Ceph cluster in a MOS managed cluster.For Ceph cluster limitations, see Mirantis Container Cloud Reference Architecture: Limitations.To deploy a Ceph cluster:1. Deploy Ceph in the same Kubernetes cluster as described in Mirantis Container CloudOperations Guide: Add a Ceph cluster.2. Open the KaaSCephCluster CR for editing as described in Mirantis Container CloudOperations Guide: Ceph advanced configuration.3. Verify that the following snippet is present in the KaaSCephCluster configuration:network:clusterNet: 10.10.10.0/24publicNet: 10.10.11.0/244. Configure the pools for Ima

This documentation provides information on how to deploy and operate a Mirantis OpenStack for Kubernetes (MOS) environment. The documentation is intended to help operators to understand the core concepts of the product. The documentation provides