April 2020 Kasten (K10) On AWS Cloud

Transcription

Amazon Web Services – K10 on the AWS CloudApril 2020Kasten (K10) on AWS CloudQuick Start Reference DeploymentApril 2020Kasten IOAWS Quick Start Reference TeamContentsOverview. 2Kasten K10 on AWS. 2Cost . 4Architecture . 4Planning the deployment . 5Specialized knowledge . 6AWS account . 6Technical requirements . 6Resource requirements . 6Deployment prerequisites. 6Deployment steps . 7Step 1. Create an IAM Policy . 7Step 2. Create an IAM Role . 7Step 3. Choose K10 installation type . 8Step 4. Validating the Install . 8Step 5. Accessing the K10 Dashboard . 8K10 Workflows . 8Support . 26Additional resources. 261

Amazon Web Services – K10 on the AWS CloudApril 2020This Quick Start was created by Kasten in collaboration with Amazon Web Services(AWS).Quick Starts are automated reference deployments that use AWS CloudFormationtemplates to deploy key technologies on AWS, following AWS best practices.OverviewThis Quick Start reference deployment guide provides step-by-step instructionsfor deploying K10 data management platform in Kubernetes on the AWS Cloud.This Quick Start is for users who want to use AWS and its products and services, such asAmazon Elastic Kubernetes Service (Amazon EKS), Amazon Elastic Block Store (AmazonEBS) Amazon Elastic File System (Amazon EFS) and Amazon Simple Storage Service(Amazon S3) to launch Kubernetes and protect data for their container environment.This reference deployment uses the Amazon EKS Architecture Quick Start as a foundationto provide a fully managed, highly available, and certified Kubernetes-conformant controlplane for K10 platform. It is meant to be low touch.Kasten K10 on AWSThe K10 data management platform, purpose-built for Kubernetes, provides enterpriseoperations teams an easy-to-use, scalable, and secure system for backup/restore, disasterrecovery, and mobility of Kubernetes applications.K10’s application-centric approach and deep integrations with relational and NoSQLdatabases, Kubernetes distributions, and all clouds provides teams the freedom ofinfrastructure choice without sacrificing operational simplicity. Policy-driven andextensible, K10 provides a native Kubernetes API and includes features such as fullspectrum consistency, database integrations, automatic application discovery, multi-cloudmobility, and a powerful web-based user interface.2

Amazon Web Services – K10 on the AWS CloudApril 2020Figure 1: K10 platform on AWSK10 use-cases on AWSGiven K10’s extensive ecosystem support, you have the flexibility to choose environments(public/ private/ hybrid cloud/ on-prem) and Kubernetes distributions (cloud vendormanaged or self managed) in support of three principal use cases: Backup and Restore,Disaster Recovery, and Application Mobility.Figure 2: K10 use-cases on AWS 3Data Protection: Snapshots are the basis of persistent data capture in K10. Theyare usually used in the context of disk volumes (PVC/PVs) used by the applicationbut can also apply to application-level data capture (e.g., by leveraging Kasten’s openand extensible application blueprints). Given the limitations of snapshots, it is oftenadvisable to set up backups of your application stack. However, even if yoursnapshots are durable, backups might still be useful in a variety of use casesincluding lowering costs with K10’s data deduplication or backing your snapshots upin a different infrastructure provider for cross-cloud delivery. Once applications have

Amazon Web Services – K10 on the AWS CloudApril 2020been protected via a policy or a manual action, it is possible to restore them in-placeor clone them to a different namespace. Seamless Migration: The ability to move an application across clusters is anextremely powerful feature that enables a variety of use cases including DisasterRecovery (DR), Test/Dev with realistic data sets, and performance testing inisolated environments. In particular, the K10 platform is built to supportapplication migration and mobility in a variety of different and terCross-Account: (e.g., AWS accounts, Google Cloud projects)Cross-Region: (e.g., US-East to US-West)Cross-Cloud: (e.g., Azure to AWS) End-to-End Security: K10 platform seamlessly integrates into a customer’senvironment with their authentication tool (OIDC, Centrify, LDAP, SAML,Kerberos, etc.,) and offers a variety of different ways to secure access to itsdashboard and APIs. K10 supports flexible permissions model which allowsscoping of user permissions to perform K10 actions only within the context ofspecified applications. To facilitate role-based access for users, K10 leveragesKubernetes ClusterRoles and Bindings, which are user-configurable. K10 usesenterprise-grade AES-256 algorithm and TLS/SSL protocols to encrypt all data atrest and in-flight. Deep Kubernetes integration: K10 platform can manage the entire statefullifecycle of an application—from optimal scheduling decisions based on theproximity to data, to Kubernetes-driven backup, restore, and migrationfunctionality.This Quick Start deploys K10 on an Amazon EKS cluster across Availability Zones within15 minutes to enable data protection.CostYou are responsible for the cost of the AWS services used while running this QuickStart reference deployment. There is no additional cost for using the Quick Start.For cost estimates, see the pricing pages for each AWS service you will be using. Prices aresubject to change.4

Amazon Web Services – K10 on the AWS CloudApril 2020Tip After you deploy the Quick Start, we recommend that you enable the AWS Costand Usage Report to track costs associated with the Quick Start. This report deliversbilling metrics to an S3 bucket in your account. It provides cost estimates based onusage throughout each month, and finalizes the data at the end of the month. Formore information about the report, see the AWS documentation.For more information on the Kasten K10 platform license types (including free editions),features included, and how to upgrade or transfer your license, please visit Kasten.ArchitectureDeploying this Quick Start for a new virtual private cloud (VPC) with defaultparameters builds the following K10 platform in the AWS Cloud. The diagram showsthree Availability Zones, leveraging multiple AWS services.Figure 3: Quick Start architecture for K10 on AWSMore detailed K10 architecture diagram is shown below.5

Amazon Web Services – K10 on the AWS CloudApril 2020Figure 4: K10 ArchitectureThe Quick Start sets up the K10 platform in an Amazon EKS cluster to provide dataprotection to applications within this cluster.Planning the deploymentSpecialized knowledgeBefore you deploy this Quick Start, we recommend that you become familiar withthe following AWS services. (If you are new to AWS, see Getting Started with AWS.)Amazon Elastic Block Store (Amazon EBS)Amazon Elastic Kubernetes Service (Amazon EKS)Amazon Simple Storage Service (Amazon S3)Amazon Elastic File System (Amazon EFS)Amazon Identity & Access Management (Amazon IAM)AWS accountIf you don’t already have an AWS account, create one at https://aws.amazon.com byfollowing the on-screen instructions. Part of the sign-up process involves receiving a phonecall and entering a PIN using the phone keypad.Your AWS account is automatically signed up for all AWS services. You are charged only forthe services you use.6

Amazon Web Services – K10 on the AWS CloudApril 2020Technical requirementsAWS account configurationKnowledge of KubernetesKnowledge of how to use KubectlResource requirementsBelow is our guidance of EKS cluster resources for your K10 installation. These numberscan vary based on the number of applications protected by K10.CPU (cores) 15mMemory 750Mi1m 1millicore (1/1000 AWS CPU core)Note: Subject to change as we continue to improve resource utilizationDeployment prerequisitesThis Quick Start provides steps to ensure a smooth and successful K10 installation in yourEKS cluster(s). Follow these instructions prior to installing K10. Pre-flight checks Helm Package Manager (v2.11.0 ) Service Account Amazon EKS ClusterNote Amazon EKS isn’t currently supported in all AWS Regions. For a currentlist of supported Regions, see the AWS Regions and Endpoints webpage.Deployment stepsStep 1. Create an IAM Policya) An IAM Policy specifies permissions the role will grant. The set of permissions neededby K10 for integrating against different AWS services are described here.Step 2. Create an IAM RoleUsing IAM Roles is an AWS recommended best practice. As such, K10 can be installedand accessed with two different IAM Roles.7

Amazon Web Services – K10 on the AWS CloudApril 2020IAM Role for the K10 product - This is for the K10 product to gain access to and useAWS resources that need to be protected/managed (EBS Volumes/Snapshots, S3Bucket). This will only be used by the K10 service account and no other user shouldbe granted access to this.IAM Role used to access K10 Dashboard – This is for users to assume an IAM Roleto login to K10 (and get a token for). This is mapped internally in Kubernetes to aCluster Role. This role does not need any IAM Policy attached and we don’t grantany access to AWS resources.a) First enable OIDC on your EKS cluster by running the command below. Moredetails can be found here. eksctl utils associate-iam-oidc-provider --cluster {EKS CLUSTER NAME} --approveb) An IAM Role can be created using a few different AWS tools: eksctl AWSManagment Console AWS CLI. Using eksctl is the easiest (command shown below).For SERVICE ACCOUNT NAMESPACE use kasten-io (or the namespace you installed K10 in)For SERVICE ACCOUNT NAME use k10-k10. eksctl create iamserviceaccount \--name service account name \--namespace service account namespace \--cluster cluster name \--attach-policy-arn IAM policy ARN \--approve \--override-existing-serviceaccountsStep 3. Choose K10 installation typea) You can install K10 on AWS in multiple ways, AWS Marketplace, AWS EKS or any otherKubernetes distribution running on EC2.i.To install via Marketplace, go here and choose the deployment type.ii.To install as a Helm chart, you can use IAM Credentials or assume an IAM Role.Instructions for both options are outlined here.e.g.: Helm command to install K10 with IAM Role (recommended) helm install kasten/k10 --name k10 --namespace kasten-io \--set secrets.awsIamRole " {AWS IAM ROLE ARN}"8

Amazon Web Services – K10 on the AWS CloudApril 2020e.g.: Helm command to install K10 with IAM Keys helm install kasten/k10 --name k10 --namespace kasten-io \--set secrets.awsAccessKeyId " {AWS ACCESS KEY ID}" \--set secrets.awsSecretAccessKey " {AWS SECRET ACCESS KEY}"Each deployment takes 10-15 minutes to complete.Step 3. Validating the Installa) To validate that K10 has been installed properly, the following command can be run inK10’s namespace (the install default is kasten-io) to watch for the status of all K10 pods.e.g.: Kubectl command to validate install kubectl get pods --namespace kasten-io --watchNote In the unlikely event scenario that pods are stuck in any other state, pleasefollow the support documentation to debug further.Step 4. Accessing the K10 Dashboarda) For local access, K10 Dashboard can be accessed by enabling port-forwarding. kubectl --namespace kasten-io port-forward service/gateway 8080:8000Dashboard URL: http://127.0.0.1:8080/k10/#/To enable authentication and access Dashboard via other options, please refer to thedocumentation here.K10 Workflows – Backup, Restore & DRIn this example, we will walk through how to use Kasten K10 to backup and restorePostgreSQL databases operating in a Kubernetes environment on Amazon AWS. The fullymanaged Amazon Elastic Kubernetes Service (EKS) makes deploying and managingcontainerized applications easy and offers serverless Kubernetes, an integrated continuousintegration and continuous delivery (CI/CD) experience, and enterprise-grade security andgovernance.PostgreSQL (often referred to as Postgres), is an Open Source relational database, popularin the cloud-native community.Kasten’s K10 data management platform, it is a secure software-only product that has beenpurpose-built for Kubernetes and provides operations teams an easy-to-use, scalable, and9

Amazon Web Services – K10 on the AWS CloudApril 2020secure system for backup/restore, disaster recovery, and mobility of Kubernetesapplications.We assume that you already have an EKS cluster set up (if not, you can follow instructionshere to deploy an EKS cluster).The instruction in this section are organized in four sections:1. Installing Kasten K10 on your EKS cluster2. Installing PostgreSQL3. Backup and restore workflow using Kasten K104. Disaster Recovery workflow using Kasten K10 to a different clusterStep 1. Installing Kasten K10 on your EKS clusterDetailed instructions for installing K10 are available in the K10 documentation. In thissection, a “happy path” install is used for demo purposes. Before proceeding with theinstall, Deployment prerequisites need to be satisfied.To install K10, follow instructions under Deployment Steps.Step 2. Installing PostgreSQLUse the commands below to create a namespace called postgresql and install PostgreSQLinto your EKS cluster. helm repo add stable elm repo updatekubectl create namespace postgresqlhelm install --namespace postgresql --name postgres stable/postgresqlTo validate the PostgreSQL install, use the command below in the postgresql namespaceto confirm that all PostgreSQL pods display a status of Running within a couple ofminutes. kubectl get pods --namespace postgresqlK10 automatically discovers the instance of PostgreSQL. Following the successful install ofPostgreSQL, click on the Applications card on the K10 dashboard to see the discoveredPostgreSQL instance.10

Amazon Web Services – K10 on the AWS CloudApril 2020Step 3. Backup and Restore Workflow using Kasten K10In this section, we will use K10’s default backup mechanism which relies on taking volumesnapshots. Click on the Applications card in the K10 dashboard and either create a backuppolicy or, for experimentation, simple create a restore point to do a full manual backup.Check the progress of the backup action in the main K10 dashboard.11

Amazon Web Services – K10 on the AWS CloudApril 2020Completion of the backup action will result in the creation of a restore point (a set ofconfiguration and data artifacts) which can be used to restore from. To restore from therestore point, go to the Applications card and click on restore button for the postgresql application. Here you should see all the available restore points.12

Amazon Web Services – K10 on the AWS CloudApril 2020Click on the restore point. This will open the Restore panel where you can view andmodify the restore parameters. Click on the Restore button to restore the associateddata and specs.13

Amazon Web Services – K10 on the AWS CloudApril 2020Check the progress of the restore action in the main K10 dashboard.The data and application configuration have been successfully restored.Using Backup policiesNote that the workflow demonstrated above used a manual backup. You can also createpolicies to execute backups on a scheduled basis. Policies are extremely configurable. Youcan set the backup schedule and snapshot retention schedule independently for finegrained control over how often backups are performed and how much total storage theyconsume.To try this out click on Create New Policy on the application card on the dashboard.14

Amazon Web Services – K10 on the AWS CloudApril 2020When a policy that applies to an application successfully executes a backup, theapplication’s compliance with the policy is reported in the application card. In thescreenshot below, we can see that our postgresql application is now compliant with allpolicies.Step 4. Disaster Recovery Workflow using Kasten K10We can use the existing restorepoint to enable Disaster Recovery (DR). Specifically, weneed to first backup snapshots to an S3 target via an Export Profile. Once it has beenuploaded, we can create an Import Profile on the destination cluster to recover theapplication to. Note that the S3 target must reside in the same region as the destinationcluster. Similar to the above, we can leverage policies to automate this workflow at a userdefined schedule and retention.To create an S3 target, click on Settings (top right of the dashboard)15

Amazon Web Services – K10 on the AWS CloudApril 2020Select Mobility Profiles, then click on New Profile. Provide a name, select Export andchoose Amazon S3. You can choose to enter AWS credentials or use the IAM Role youinstalled K10 with.16

Amazon Web Services – K10 on the AWS CloudApril 2020Go back to the Dashboard, click on the Policies card, click on the policy we created, andselect Enable Backups via Snapshot Exports . Choose the appropriate profile that pointsto the S3 target.Once you’ve made changes and confirm, click on Show import details to copy the string.This is used as a handshake on the destination cluster to import from the appropriatebackup.17

Amazon Web Services – K10 on the AWS CloudApril 2020Go back to the Dashboard and you should see an Export job with a successful upload to theS3 target.18

Amazon Web Services – K10 on the AWS CloudApril 2020Recovery steps to import data on the destination clusterSimilar to the above steps, after installing K10 on the destination cluster, login to thedashboard and go to the Settings tab. Select New Profile of type Import , enter thedetails (as done previously for the Export Profile ).19

Amazon Web Services – K10 on the AWS CloudApril 2020Go back to the dashboard and select the Policies card. Choose to create a new policy of typeImport. Select Restore After Import and paste the string under Config Data for Import .Once you go back to the dashboard, you will see an Import job followed by a Restore job tosimulate a successful disaster recovery.20

Amazon Web Services – K10 on the AWS CloudApril 2020SupportAll certified Kubernetes distributions greater than v1.12 and OpenShift 3.11 are supportedby the current K10 release.If you have questions, need support, or would like an invite to our support Slack channel,please send email to our support team. You can also call us at 1 (415) 851-1767.Additional resourcesAWS serviceseksctlaws-iam-authenticatorQuick Start reference deploymentsAWS Quick Start home pageAmazon EKS Architecture Quick StartKastenKasten documentation21

Amazon Web Services – K10 on the AWS CloudApril 2020 2020, Amazon Web Services, Inc. or its affiliates, and Kasten, Inc. All rights reserved.NoticesThis document is provided for informational purposes only. It represents AWS’s current product offeringsand practices as of the date of issue of this document, which are subject to change without notice.Customers are responsible for making their own independent assessment of the information in thisdocument and any use of AWS’s products or services, each of which is provided “as is” without warranty ofany kind, whether express or implied. This document does not create any warranties, representations,contractual commitments, conditions or assurances from AWS, its affiliates, suppliers or licensors. Theresponsibilities and liabilities of AWS to its customers are controlled by AWS agreements, and thisdocument is not part of, nor does it modify, any agreement between AWS and its customers.The software included with this paper is licensed under the Apache License, Version 2.0 (the "License"). Youmay not use this file except in compliance with the License. A copy of the License is located athttp://aws.amazon.com/apache2.0/ or in the "license" file accompanying this file. This code is distributed onan "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions and limitations under the License.22

Amazon Elastic Kubernetes Service (Amazon EKS) Amazon Simple Storage Service (Amazon S3) Amazon Elastic File System (Amazon EFS) Amazon Identity & Access Management (Amazon IAM) AWS account If you don't already have an AWS account, create one at https://aws.amazon.com by following the on-screen instructions.