MongoDB With SUSE Rancher - SUSE Rancher, RKE, MongoDB

Transcription

SUSE Best PracticesMongoDB with SUSE RancherGetting StartedSUSE Rancher, RKE, MongoDBSamip Parikh, Solution Architect, SUSE1MongoDB with SUSE Rancher

This document provides recommendations for deploying and managing ahighly available MongoDB NoSQL database on a SUSE Rancher Kubernetescluster.Disclaimer: Documents published as part of the SUSE Best Practices andthe Technical Reference Documentation series have been contributed voluntarily by SUSE employees and third parties. They are meant to serve as examples of how particular actions can be performed. They have been compiled with utmost attention to detail. However, this does not guaranteecomplete accuracy. SUSE cannot verify that actions described in these documents do what is claimed or whether actions described have unintendedconsequences. SUSE LLC, its affiliates, the authors, and the translators maynot be held liable for possible errors or the consequences thereof.Publication Date: 2021-04-29Contents21Motivation 42Technical overview 53Value of HA for data 54Setting up a cluster with SUSE Rancher 65Storage considerations 76Creating a persistent volume 117Deploying MongoDB Kubernetes Operator 118Deploying MongoDB Ops Manager resource 139Loading and querying the database 1410Simulating node failure & restoration 1611Summary 18MongoDB with SUSE Rancher

312Additional resources 1813Legal notice 1914GNU Free Documentation License 20MongoDB with SUSE Rancher

1 MotivationAgility is the name of the game in modern application development. This is driving developerstoward more agile, cloud native methodologies that focus on microservices architectures andstreamlined workflows. Container technologies, like Kubernetes, embody this agile approachand help enable cloud native transformation.SUSE Rancher simplifies Kubernetes management, empowering you to take control of your ITlandscape and create an agile data platform that accelerates achievement of your goals. Rancherenables you to manage dynamic, robust, multi-cluster Kubernetes environments and supportsany CNCF-certified Kubernetes distribution. With built-in resilience and scalability, unified security and policy management, and a rich catalog of shared tools and services, Rancher helpsyou accelerate development-to-production and innovate everywhere.MongoDB is a cloud native database technology that brings all the power of the traditional,relational database and provides a more natural way to work with modern data. MongoDB offersa variety of compelling features that make it a natural partner for a SUSE Rancher agile dataplatform. These include:Flexible Data ModelMongoDB’s dynamic schema is ideal for handling changing requirements and continuous delivery. You can seamlessly roll out new features without having to update existingrecords — a process that can take weeks for traditional, relational databases. DevOps teamscan quickly model data against an ever-changing environment and roll these changes intoproduction. This results in faster time to market and faster time to value.ResilienceMongoDB’s replica sets have built-in redundancy, providing greater resilience and enhancing disaster recovery capabilities. Administrators can even isolate operational workloadsfrom analytical reporting in single database cluster to ensure sufficient resources are allocated to handle demand.Monitoring and AutomationHeterogeneous services increase the level of complexity and can stall productivity. Tech-nology that handles monitoring and automation is critical to keeping DevOps teams productive as their environments evolve. MongoDB Ops Manager features visualization, cus-tom dashboards, and automated alerting. It tracks, reports, processes, and visualizes 100 key database and systems-health metrics, including operations counters, CPU utilization,replication status, and node status.4MongoDB with SUSE Rancher

ScalabilityMongoDB’s auto-sharding automatically partitions and distributes the database acrossnodes, serving IT infrastructures that require dynamic, high-performance capabilities. Distribution can even span different geographic regions. MongoDB is ideally suited to scaleout architectures.With enterprise-grade products, proactive support, and success-focused services and training,SUSE and MongoDB deliver an agile data platform that helps organizations achieve their cloudnative goals.2 Technical overviewSUSE Rancher is a lightweight Kubernetes installer that supports installation on bare-metal andvirtualized servers. Rancher solves a common issue in the Kubernetes community: installationcomplexity. With Rancher, Kubernetes installation is simplified, regardless of what operatingsystems and platforms you are running.This document reviews considerations for deploying and managing a highly available, MongoDBNoSQL database on a SUSE Rancher Kubernetes cluster.In practice, the process is as follows:Install a Kubernetes cluster through Rancher Kubernetes EngineInstall a cloud native storage solution on KubernetesDeploy MongoDB Enterprise Kubernetes Operator er/)Configure a storage class and define storage requirements via OperatorTest failover by killing or cordoning nodes in your cluster3 Value of HA for dataOne of the primary benefits of running a Kubernetes environment is flexibility, the ability toeasily adapt to varying circumstances. Traditional database deployments exist in fairly staticconfigurations and environments. The beauty of running a data-oriented service on Kuberneteslies in maintaining stability while enabling adaptability to meet real-world situations.5MongoDB with SUSE Rancher

Imagine a scenario where your e-commerce site is consistently taking 100 orders per day. Sud-denly, a viral marketing event occurs, and your site is pushed to 5000 orders per day for a day.This increase could easily lead to data overload – or worse, corruption or downtime, whichcould result to considerable loss of revenue. Having a way to design for such failure scenariosand maintain resilient operations is a tangible market advantage.MongoDB can run in a single node configuration and in a clustered configuration using replicasets (not to be confused with Kubernetes Stateful Sets). A replica set is a group of MongoDBinstances that maintain the same data. A replica set contains several data-bearing nodes andoptionally one arbiter node. Of the data-bearing nodes, one and only one member is deemedthe primary node, while the other nodes are deemed secondary nodes. Resiliency of the datais achieved, as illustrated below.In this configuration, the failover process generally completes within a minute. It may takeabout 30 seconds for the members of a replica set to declare a primary inaccessible. One of theremaining secondaries will then be enabled as the "new primary". The election itself may takeanother 10 to 30 seconds. During this time, the data will be preserved in a virtually seamlessway for dependent services.4 Setting up a cluster with SUSE RancherSUSE Rancher is a tool to install and configure Kubernetes in a choice of environments includingbare metal, virtual machines, and IaaS. Rancher is a complete container management platformbuilt on upstream Kubernetes. It consists of three major components:A certified Kubernetes Distribution – Rancher Kubernetes Engine (RKE)A Kubernetes Management platform (Rancher)Application Catalog and management (Third-party)6MongoDB with SUSE Rancher

Rancher has the capabilities to manage any Kubernetes cluster from a central location, via theRancher server. As illustrated below, Rancher can manage any Kubernetes flavor and is notrestricted to RKE.For reference, see Rancher deployment guides for specific details on installation. By the end ofthis step, you should have a cluster with one master and three worker nodes.5 Storage considerationsWhen deploying an application that needs to retain data, you need to create persistent storage.Persistent storage allows you to store application data external from the pod running your ap-plication. This storage practice allows you to maintain application data, even if the application’spod fails.7MongoDB with SUSE Rancher

A variety of storage options exist and can be used to create an HA data solution with Rancher.Some considerations you may need to follow for your storage solution include:Volumes as persistent storage for the distributed stateful applicationsPartitioned block storage for Kubernetes volumes with or without a cloud providerReplicated block storage across multiple nodes and data centers to increase availabilitySecondary data backup storage (for example, NFS or S3)Cross-cluster disaster recovery volumesRecurring volume snapshotsRecurring backups to secondary storageNon-disruptive upgradesSome common storage solutions to consider are as follows:Longhorn (https://longhorn.io)Distributed block storage system for Kubernetes. Originally developed by Rancher Labs.Currently sandbox project of the Cloud Native Computing FoundationOpenEBS (https://openebs.io)Open source, CNCF Sandbox storage with flexible storage engine options - requires thirdparty integrationCeph (https://ceph.io)Powerful, open source, general purpose storage in the CNCF Sandbox – requires thirdparty integrationPortworx (https://portworx.com)Proprietary solution with Rancher certified integration - installation steps can be foundhere her/rancher-2.x)5.1Setting up your storageBefore proceeding, be sure that you understand the Kubernetes concepts of persistent volumes, persistent volume claims, and storage classes. For more information, refer to How Per-sistent Storage Works admin/volumes-and-storage/how-storage-works)8in the Rancher documentation.MongoDB with SUSE Rancher

The workflow for setting up existing storage is as follows:1. Ensure you have access to set up your persistent storage. This may be storage in an infra-structure provider, or it could be your own storage.2. Add a persistent volume (PV) that refers to the persistent storage.3. Add a persistent volume claim (PVC) that refers to the PV.4. Mount the PVC as a volume in your workload.For further details and prerequisites, read the Rancher documentation section Setting Up Existing Storage ge).The overall workflow for provisioning new storage is as follows:1. Add a StorageClass and configure it to use your storage provider. The StorageClass couldrefer to storage in an infrastructure provider, or it could refer to your own storage.2. Add a persistent volume claim (PVC) that refers to the storage class.3. Mount the PVC as a volume for your workload.See section Dynamically Provisioning New Storage in Rancher )requisites.5.2for details and pre-Creating a storage class for MongoDBWhen the Kubernetes cluster is running and storage is configured, it is time to deploy a highlyavailable MongoDB database.MongoDB resources are created in Kubernetes as custom resources. After you create or update aMongoDB Kubernetes resource specification, you direct MongoDB Kubernetes Operator to applythis specification to your Kubernetes environment. Kubernetes Operator creates the definedStatefulSets, services and other Kubernetes resources. After the Operator finishes creating thoseobjects, it updates the Ops Manager deployment configuration to reflect changes.The following example shows a resource specification for a replica set y/#term-replica-set)configuration:apiVersion: mongodb.com/v1kind: MongoDB9MongoDB with SUSE Rancher

metadata:name: my-replica-setspec:members: 3version: "4.2.2-ent"service: my-serviceopsManager: # Alias of cloudManagerconfigMapRef:name: my-projectcredentials: my-credentialspersistent: truetype: ReplicaSetpodSpec:cpu: "0.25"memory: "512M"persistence:multiple:data:storage: "10Gi"journal:storage: "1Gi"labelSelector:matchLabels:app: "my-app"logs:storage: "500M"storageClass: standardpodAntiAffinityTopologyKey: uringExecution:- labelSelector:matchExpressions:- key: securityoperator: Invalues:- S1topologyKey: deSelectorTerms:- matchExpressions:- key: kubernetes.io/e2e-az-nameoperator: Invalues:- e2e-az1- e2e-az2podTemplate:10MongoDB with SUSE Rancher

metadata:labels:label1: edDuringSchedulingIgnoredDuringExecution:- podAffinityTerm:topologyKey: "mykey"weight: 50security:tls:enabled: trueauthentication:enabled: truemodes: ["X509"]internalCluster: "X509"additionalMongodConfig:net:ssl:mode: preferSSLFull details can be found here er/reference/k8s-operator-specification).6 Creating a persistent volumeYou can now create a persistent volume claim (PVC) based on the storage class. Dynamic pro-visioning will be created without explicitly provisioning a persistent volume (PV). As part ofdeployment, the Kubernetes operator creates persistent volumes stent-volumes)for the Ops Manager StatefulSets. The Kubernetes containeruses persistent volumes to maintain the cluster state between restarts.7 Deploying MongoDB Kubernetes OperatorKubernetes needs help creating and managing stateful applications like databases. The typicallifecycle events of a MongoDB cluster may include provisioning storage and computing power,configuring network connections, setting up users, and more. This is where the MongoDB Enterprise Kubernetes Operator comes in. It translates the human knowledge of how to create aMongoDB instance into a scalable, repeatable, and standardized methodology. And it does this11MongoDB with SUSE Rancher

by using the built-in Kubernetes API and tools. To use the operator, you simply need to provideit with the specifications for your MongoDB cluster. The operator uses this information to directKubernetes into performing all the required steps to achieve the end state.The general commands for deploying the MongoDB Enterprise Operator are:kubectl describe deployments mongodb-enterprise-operator -n namespace helm install chart-name helm chart \--values helm chart/values.yaml \The next step after deploying the operator is to create the database using a yaml le, such as:apiVersion: mongodb.com/v1kind: MongoDBmetadata:name: my-standalone spec:version: "4.2.2-ent"opsManager:configMapRef:name: configMap.metadata.name # Must match metadata.name in ConfigMap filecredentials: mycredentials type: Standalonepersistent: trueNow, you can deploy the database and check the status of the deployment with:kubectl apply -f standalone-conf .yamlkubectl get mdb resource-name -o yamlAt this point, MongoDB has been deployed via the operator. Additional settings canbe applied to create replica le/tutorial/deploy-replica-set/)sets to further enhance data availability. Also, sharded clusters can be createdto enable greater throughput across a distributed system.12MongoDB with SUSE Rancher

Additionally, see specific documentation ernetes)and steps for full installation and configuration options.8 Deploying MongoDB Ops Manager resourceKubernetes needs help creating and managing stateful applications like databases. The typicallifecycle events of a MongoDB cluster may include provisioning, storage and compute. Ops Manager can be deployed via the MongoDB Kubernetes Operator to manage MongoDB resourcesin a cluster. The Operator manages the lifecycle of each of these deployments differently. TheOperator manages Ops Manager deployments using the Ops Manager custom resource. The Op-erator watches the custom resource’s specification for changes. When the specification changes,the Operator validates the changes and makes the appropriate updates to the resources in thecluster. Ops Manager custom resources specification defines the following components: Application Database, Ops Manager application, and Backup Daemon. Summarized instructions tocreate Ops Manager are below, but full details can be found here er/tutorial/deploy-om-container).To begin, run the following command to execute all kubectl commands in the namespace youcreated:kubectl config set-context (kubectl config current-context) --namespace namespace Create Ops Manager object as below:apiVersion: mongodb.com/v1kind: MongoDBOpsManagermetadata:name: myopsmanager spec:replicas: 1version: opsmanagerversion adminCredentials: adminusercredentials # Should match metadata.nameexternalConnectivity:type: LoadBalancerapplicationDatabase:members: 3version: mongodbversion persistent: true13MongoDB with SUSE Rancher

9 Loading and querying the databaseWhen the database has been created, you can populate it with some sample data.First, nd the pod that is running MongoDB:POD kubectl get pods -l app mongo grep Running grep 1/1 awk '{print 1}' Then, access the MongoDB shell on that POD instance: kubectl exec -it POD mongoMongoDB shell version v4.0.0connecting to: mongodb://127.0.0.1:27017MongoDB server version: 4.0.0Welcome to the MongoDB shell.Now, using the MongoDB shell, you can populate a collection:db.ships.insert({name:'USS ps.insert({name:'USS :50,codes:[10,17,19]})db.ships.insert({name:'IKS Buruk',operator:' 110,120]})db.ships.insert({name:'IKS Somraw',operator:' :'Romulan a',operator:'Romulan ,codes:[251,251,220]})And you can run some operations on the MongoDB collection.For example, nd one arbitrary document:db.ships.findOne(){" id" : ObjectId("5b5c16221108c314d4c000cd"),"name" : "USS Enterprise-D","operator" : "Starfleet","type" : "Explorer","class" : "Galaxy","crew" : 750,"codes" : [14MongoDB with SUSE Rancher

10,11,12]}You can also nd ALL documents and apply some basic formatting:db.ships.find().pretty(){" id" : ObjectId("5b5c16221108c314d4c000d1"),"name" : "IKS Somraw","operator" : " Klingon Empire","class" : "Raptor","crew" : 50,"codes" : [101,111,120]}{" id" : ObjectId("5b5c16221108c314d4c000d2"),"name" : "Scimitar","operator" : "Romulan Star Empire","type" : "Warbird","class" : "Warbird","crew" : 25,"codes" : [201,211,220]}And you can get a list of the names of the ships:db.ships.find({}, {name:true, id:false}){ "name" : "USS Enterprise-D" }{ "name" : "USS Prometheus" }{ "name" : "USS Defiant" }{ "name" : "IKS Buruk" }{ "name" : "IKS Somraw" }{ "name" : "Scimitar" }{ "name" : "Narada" }15MongoDB with SUSE Rancher

10 Simulating node failure & restorationNext, simulate a node failure by cordoning o the node on which MongoDB is running: NODE kubectl get pods -l app mongo -o wide grep -v NAME awk '{print 7}' kubectl cordon {NODE} node/ip-172-31-29-132.compute.internal cordonedThe above command disables scheduling on one of the nodes. Check this with: kubectl get .compute.internalv1.13.4Now, go ahead and delete the MongoDB pod: POD kubectl get pods -l app mongo -o wide grep -v NAME awk '{print 1}' kubectl delete pod {POD}pod "mongo-68cc69bc95-7q96h" deletedWhen the pod is deleted, it is relocated to the node with the replicated data, even when thatnode is in a different zone. Each pod is rescheduled on the exact node where the data is stored.Verify this: kubectl get pods -l app mongo -o 21.compute.internalNote that a new pod has been created and scheduled in a different node.Next, uncordon the node to bring it back to action: kubectl uncordon {NODE}node/ip-172-31-29-132.compute.internal uncordonedFinally, verify that the data is still available.16MongoDB with SUSE Rancher

To do so, nd the pod name and run the ‘exec’ command, then access the Mongo shell:POD kubectl get pods -l app mongo grep Running grep 1/1 awk '{print 1}' kubectl exec -it POD mongoMongoDB shell version v4.0.0connecting to: mongodb://127.0.0.1:27017MongoDB server version: 4.0.0Welcome to the MongoDB shell.After that, query the collection to verify that the data is intact.Find one arbitrary document:db.ships.findOne(){" id" : ObjectId("5b5c16221108c314d4c000cd"),"name" : "USS Enterprise-D","operator" : "Starfleet","type" : "Explorer","class" : "Galaxy","crew" : 750,"codes" : [10,11,12]}Now, nd all documents and apply formatting:db.ships.find().pretty() .{" id" : ObjectId("5b5c16221108c314d4c000d1"),"name" : "IKS Somraw","operator" : " Klingon Empire","class" : "Raptor","crew" : 50,"codes" : [101,111,120]}{" id" : ObjectId("5b5c16221108c314d4c000d2"),"name" : "Scimitar",17MongoDB with SUSE Rancher

"operator" : "Romulan Star Empire","type" : "Warbird","class" : "Warbird","crew" : 25,"codes" : [201,211,220]}For further validation, you can run all the same initial steps to compare that data is still availablewith originally queried values.11 SummaryMongoDB with SUSE Rancher makes it possible to easily build, deploy, and manage resilient,scalable data services.12 Additional resourcesFor more information, visit:Rancher best practices guide ctices/)Rancher troubleshooting tips hooting/)MongoDB best practices rator)18MongoDB with SUSE Rancher

13 Legal noticeCopyright 2006–2022 SUSE LLC and contributors. All rights reserved.Permission is granted to copy, distribute and/or modify this document under the terms of theGNU Free Documentation License, Version 1.2 or (at your option) version 1.3; with the InvariantSection being this copyright notice and license. A copy of the license version 1.2 is included inthe section entitled "GNU Free Documentation License".SUSE, the SUSE logo and YaST are registered trademarks of SUSE LLC in the United States andother countries. For SUSE trademarks, see https://www.suse.com/company/legal/ .Linux is a registered trademark of Linus Torvalds. All other names or trademarks mentioned inthis document may be trademarks or registered trademarks of their respective owners.Documents published as part of the SUSE Best Practices series have been contributed voluntarilyby SUSE employees and third parties. They are meant to serve as examples of how particularactions can be performed. They have been compiled with utmost attention to detail. However,this does not guarantee complete accuracy. SUSE cannot verify that actions described in thesedocuments do what is claimed or whether actions described have unintended consequences.SUSE LLC, its affiliates, the authors, and the translators may not be held liable for possible errorsor the consequences thereof.Below we draw your attention to the license under which the articles are published.19MongoDB with SUSE Rancher

14 GNU Free Documentation LicenseCopyright 2000, 2001, 2002 Free Software Foundation, Inc. 51 Franklin St, Fifth Floor, Boston,MA 02110-1301 USA. Everyone is permitted to copy and distribute verbatim copies of thislicense document, but changing it is not allowed.0. PREAMBLEThe purpose of this License is to make a manual, textbook, or other functional and useful docu-ment "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily,this License preserves for the author and publisher a way to get credit for their work, while notbeing considered responsible for modifications made by others.This License is a kind of "copyleft", which means that derivative works of the document mustthemselves be free in the same sense. It complements the GNU General Public License, whichis a copyleft license designed for free software.We have designed this License in order to use it for manuals for free software, because freesoftware needs free documentation: a free program should come with manuals providing thesame freedoms that the software does. But this License is not limited to software manuals; itcan be used for any textual work, regardless of subject matter or whether it is published as aprinted book. We recommend this License principally for works whose purpose is instructionor reference.1. APPLICABILITY AND DEFINITIONSThis License applies to any manual or other work, in any medium, that contains a notice placedby the copyright holder saying it can be distributed under the terms of this License. Such anotice grants a world-wide, royalty-free license, unlimited in duration, to use that work underthe conditions stated herein. The "Document", below, refers to any such manual or work. Anymember of the public is a licensee, and is addressed as "you". You accept the license if you copy,modify or distribute the work in a way requiring permission under copyright law.A "Modified Version" of the Document means any work containing the Document or a portionof it, either copied verbatim, or with modifications and/or translated into another language.20MongoDB with SUSE Rancher

A "Secondary Section" is a named appendix or a front-matter section of the Document that dealsexclusively with the relationship of the publishers or authors of the Document to the Document’soverall subject (or to related matters) and contains nothing that could fall directly within thatoverall subject. (Thus, if the Document is in part a textbook of mathematics, a Secondary Sectionmay not explain any mathematics.) The relationship could be a matter of historical connectionwith the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.The "Invariant Sections" are certain Secondary Sections whose titles are designated, as beingthose of Invariant Sections, in the notice that says that the Document is released under thisLicense. If a section does not t the above definition of Secondary then it is not allowed to bedesignated as Invariant. The Document may contain zero Invariant Sections. If the Documentdoes not identify any Invariant Sections then there are none.The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or BackCover Texts, in the notice that says that the Document is released under this License. A FrontCover Text may be at most 5 words, and a Back-Cover Text may be at most 25 words.A "Transparent" copy of the Document means a machine-readable copy, represented in a formatwhose specification is available to the general public, that is suitable for revising the documentstraightforwardly with generic text editors or (for images composed of pixels) generic paintprograms or (for drawings) some widely available drawing editor, and that is suitable for inputto text formatters or for automatic translation to a variety of formats suitable for input to textformatters. A copy made in an otherwise Transparent le format whose markup, or absence ofmarkup, has been arranged to thwart or discourage subsequent modification by readers is notTransparent. An image format is not Transparent if used for any substantial amount of text. Acopy that is not "Transparent" is called "Opaque".Examples of suitable formats for Transparent copies include plain ASCII without markup, Tex-info input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML, PostScript or PDF designed for human modification. Examplesof transparent image formats include PNG, XCF and JPG. Opaque formats include proprietaryformats that can be read and edited only by proprietary word processors, SGML or XML forwhich the DTD and/or processing tools are not generally available, and the machine-generatedHTML, PostScript or PDF produced by some word processors for output purposes only.The "Title Page" means, for a printed book, the title page itself, plus such following pages as areneeded to hold, legibly, the material this License requires to appear in the title page. For worksin formats which do not have any title page as such, "Title Page" means the text near the mostprominent appearance of the work’s title, preceding the beginning of the body of the text.21MongoDB with SUSE Rancher

A section "Entitled XYZ" means a named subunit of the Document whose title either is preciselyXYZ or contains XYZ in parentheses following text that translates XYZ in another language.(Here XYZ stands for a specific section name mentioned below, such as "Acknowledgements","Dedications", "Endorsements", or "History".) To "Preserve the Title" of such a section when youmodify the Document means that it remains a section "Entitled XYZ" according to this definition.The Document may include Warranty Disclaimers next to the notice which states that this Li-cense applies to the Document. These Warranty Disclaimers are considered to be included byreferenc

highly available MongoDB NoSQL database on a SUSE Rancher Kubernetes cluster. Disclaimer: Documents published as part of the SUSE Best Practices and the Technical Reference Documentation series have been contributed volun-tarily by SUSE employees and third parties. They are meant to serve as ex-amples of how particular actions can be performed.