Transcription
Qlik Sense on AWSDeployment Guide .7.8.9.10.Qlik Sense: A PrimerQlik on AWS - high level ArchitectureInstallationScalability and SizingConnectivitya.How to get startedb.How to connectc.How to build an appReference ArchitectureStorage and Access PatternsSecurity and AuthenticationOperationsa.Using the Qlik Management Consoleb.Importing/exporting applicationsc.Creating new Streamsd.Scheduling taskse.Publishing applications11. Designs for Deploying Qlik Sense on AWS12.13.14.15.a.Multi-node deploymentsb.Load Balancingc.High Availability and Disaster Recovery StrategyHigh Performance StorageConclusionContributorsNotes
AbstractAmazon Web Services (AWS) is a flexible, cost-effective, easy-to-use cloud computing platform. Qlik Sense is aplatform that delivers visual analytics to where they deliver matter the most – the point at which decisions aremade. Qlik’s platform is widely deployed on AWS Cloud infrastructure to support analytics and visual analysis atvarious levels within organizations. Running Qlik Sense solutions on Amazon Elastic Compute Cloud (AmazonEC2) is a great option for organizations of any size that want to deliver SaaS (software as a service) solutions tointernal and/or external users with cost effective, high-performing operations on any number of data sources.SaaS solutions based on Qlik’s platform and delivered through AWS Cloud infrastructure enable individuals andteams to easily create and share applications with a wide number of users across multiple devices.This Deployment Guide provides an overview on how to deploy and implement the Qlik Sense platform on theAWS Cloud infrastructure including architectural considerations and configuration steps that explain how QlikSense can leverage various AWS services such as Amazon Redshift, Amazon Elastic Compute Cloud (AmazonEC2), Amazon Virtual Private Cloud (Amazon VPC), AWS Directory Service, and Amazon Relational DatabaseServices (RDS) to run scalable, secure and reliable analytical solutions.IntroductionAWS provides an infrastructure with all the services and computing power needed to provide global solutionsthat can reliably perform as needed, regardless of unexpected changes in demand and concurrency acrossvarious levels of any organization. Qlik Sense is a next-generation visual analytics platform that empowerseveryone to see the whole story that lives within their data. It supports a full spectrum of BI use cases includingself-service visualization for everyone, centrally deployed guided analytics apps and dashboards, embeddedanalytics, and reporting, all within a governed framework that drives enterprise scalability and trust for IT. Thisgives everyone the ability to answer not just ‘what happened’, but ‘why’ and ‘what it is likely to happen next’. Qlikis the first and only analytics software vendor that can do this by providing a fully integrated, simple and intuitivevisual analytics platform, powered by Qlik’s patented in-memory associative data indexing engine that deliversanalytics and therefore insight right to the point of decision. Qlik delivers a complete analytics solution, including: Simplified visual data preparation Associative engine for business discovery Cloud-based upstream data sourcing and delivery Self-service & governed data visualization Centralized guided analytics Collaborative analytics and reporting Embedded analytics in any application or web page AWS InstallQlik Sense installation Guide for AWS 2
Qlik: A PrimerQlik is a Business Intelligence platform which provides “Managed Analysis” and Data Visualization capabilitiesallowing people to navigate and filter freely on any of the data elements in a rich multi-sheet dashboard. Allvisible objects are recalculated after a user performs selections (highlighted in green) causing associated data tobe clearly distinguishable (highlighted in white) from non-associated data (highlighted in grey). This is usuallyreferred as ‘the associative experience’ and it is the foundation for the data discovery capabilities within Qlik.Basically, the ability to understand the data and make new discoveries in a visual way. Qlik is able to delivervisual analytics in a variety of ways, each a category in its own right. Self-Service Data Visualization empowers all users to create and explore visual analytics in a selfservice manner, whether it’s a local spreadsheet or large datasets from multiple systems-of-record. Forsome, this could mean starting from scratch to look at ad-hoc combinations of data to investigate ahunch, or confirm a theory. For others, it simply means re-using pre-constructed and re-usable analyticsapplications, with the ability to modify to suit their individual needs. The advantage with Qlik is that Qliksolutions don’t trade off agility for trust and scalability. Users have easy-to-use, drag-and-drop selfservice capabilities, while still operating in a framework of security and trusted data models that lead totrusted insights. Guided Analytics provides data driven applications and dashboards -typically developed by IT ortechnical business analysts-, for consumption by others within their day-to-day operational processes.An example of guided analytics app would be an interactive dashboard where users can search andfilter information in a structured manner, and use pre-defined navigation options to move to differentviews to support better decision making with less complexity for the end user. Embedded Analytics provides the ability to insert analytics and visualizations into any workflow orapplication. This furthers the goal of delivering analytics and insight exactly where and when it isneeded, to improve understanding and decision making throughout the organization. Custom analytics application provides the ability to create custom mashups that leverage out-of-thebox as well as custom design visualizations for various types o f analytics. These custom applicationsand mashups can be easily deployed as standalone solutions and/or as part of a larger applicationwhile still providing the same associative experience as a traditional Qlik application. Collaborative Analytics (Reporting) deliver information and insights to people in a repeatable,portable format even when they are outside the BI system. Reporting is one of the oldest forms of BIand the reports are typically static, with limited or no interactivity, and are often printed for use off-line(i.e. financial reports). Today, reporting provides limited value, but when combined with visual analyticslinked to the actual underlying data it drives increased efficiencies, higher collaboration and delivers thepower of visual analytics to all of the business stakeholders. The ability to connect reporting withdynamic interactive visualization gives your data a story to tell, rather than being locked in a staticreport.Qlik Sense installation Guide for AWS 3
Qlik Sense Server is a suite of component services that are installed as part of the Server. While suchcomponents can be installed across various nodes, this document covers the scenario where all of thosecomponents are installed within the same node. In other words, a single node architecture. The image belowdepicts some of the main component services that are part of the Qlik Sense Service.In order to learn more about such component services and how to architect a Qlik Sense environment, visit thefollowing tent/Server/ServerArchitecture-Services.htmQlik on AWS – High Level ArchitectureCloud based solutions are prevalent in almost every single Industry and Qlik’s customers are eager to implementQlik Sense environments on AWS infrastructure so they can rapidly deploy new applications in a simple, costeffective and scalable manner. The difficult and lengthy procurement process for on-premises equipment makesit hard to meet deployment timelines, but customers can spin up new AWS infrastructure on demand in minutes.For those customers, the AWS based approach is ideal for a SaaS solutions. This guide will help customerspursuing this approach so that they understand the different aspects of AWS and Qlik that will be involved in thesetup and configuration of the environment.Qlik Sense installation Guide for AWS 4
Some of the characteristics that tend to be similar across Organizations that go with Qlik and AWSimplementations are:I.II.III.Pre-requisites: Tight timeline to develop and deploy Qlik Sense applications Appropriate Licensing from Qlik is available Proactive support and stewardship from IT (i.e. Active Directory extract which can be used byan LDIF, Direct Connect, etc.)Competencies Good understanding of AWS architecture and services (i.e. AWS EC2, AWS VPC, AWSElastic IP, AWS S3, AWS RDS, AWS EMR, AWS Redshift, AWS Cloud formation, etc.) withinthe team Internal resources that can maintain and manage the Qlik and AWS environments areavailable Ability to setup a QA environment within AWS and appropriate time for testing is availableConstraints Data complexity is not too high Volume and frequency of data updates will not interfere with the cloud -based approach (i.e. clear strategy on nightly incremental feeds)Qlik Sense installation Guide for AWS 5
AWS is an excellent platform for running many advanced data systems in the cloud. Characteristics of cloudcomputing solutions built on AWS, such as scalability, security and tight integration with IT resources are keybenefits when architecting and running Qlik applications. A general understanding of these characteristics canhelp you make the best architecture decisions for your analytical system. Understanding of this guide will beeasier if you are already familiar with the following topics:I.Amazon EC2 and Amazon VPCII.Qlik Sense ServerIII.Qlik Sense ConfigurationIV.Windows Server 2012V.VI.Windows Server Active Directory and DNSAmazon RedshiftWhile this document covers general steps for Qlik Sense product installations and software configuration tasks,careful consideration on configuration topics is strongly recommended when you are planning and deploying QlikSense Solutions on AWS Cloud in Production environments. For more advanced scenarios, consult with QlikProfessional Services and/or one of Qlik’s Solution Integrator partners. For generic Qlik Sense softwareconfiguration guidance and best practices, consult the Qlik product documentation available at http://help.qlik.com.If you are new to AWS, see the ‘Getting Started’ section of the AWS documentation (which provides high levelreferences and links to specifics areas of documentation) at: wsgsgintro/intro.htmlInstallationIn order to install Qlik Sense on AWS, you should download Qlik Sense Enterprise fromhttp://www.qlik.com/download (a Qlik account will be needed in order to access the download site). Before youlaunch an AWS EC2 instance (to learn more visit: http://aws.amazon.com/ec2/) where you can install Qlik SenseEnterprise, follow the next steps:1.Create a security group for Qlik Sense (the name could be something generic such as QlikSense) with the rules in the following screenshot. Using the left navigation panel within the AWSManagement Console, go to EC2 Security Groups. Click “Create Security Group”.NameInbound RulesQlik SenseHTTP, HTTPS, RDP, Custom TCP Rule (4244), Custom TCP Rule (4248)If you are using VPC, the VPC ID can be entered here at this time, or it updated later once it isavailable. In the diagram, both HTTP and HTTPS are selected, but in most installations, youwould choose one of them (typically only HTTPS).Qlik Sense installation Guide for AWS 6
Make sure to modify the Source Inbound Rule (Source) drop down menu to provide the correctsource IPs. This can be a single IP or a range of addresses in CIDR notation. In essence, thesecurity group requires necessary ports to be opened depending on the setup. For example, if thedata source is on premise and requires the connection to, for example ODBC ports, those portswill need to be opened too.2.Create a key pair called “Qlik Sense”. In the case that an existing key pair is available, it canbe used instead. Using the left navigation panel within the AWS Management Console, go to EC2 Key Pairs and click Create Key Pair. Save this file (Qlik Sense.pem) somewhere safe andmake sure not to lose it. You will use this file to access the instance.3.Launch a Windows Server 2012 R2 Base instance. Using the AWS Management Console, goto EC2 Instance and click “Launch Instance”. This will start a seven step process to create theinstance.a.Choose AMI: Using the Quick Start option use Windows Server 2012 R2 Base. Click ‘Next’ to continue.If for whatever reason you have already an AMI available that will be used for this purpose, typically, thefollowing characteristics are desired as a minimum: Recommended 4 cores / 16 GB minimum (this depends on the data volumes) Clean O/S .Net 4.5.2 installed and updates applied (default is 4.5 for Windows 2012 ) IPv4 or IPv6 (dual stack) Internet access from server (for license registration)Qlik Sense installation Guide for AWS 7
b. Administrative rights and Remote Desktop access to the instance Service account with administrative rights on the server is availableChoose the Instance Type. Qlik runs best on memory optimized instances, such as the R3 series.Choose one of the following 3 instances which are a recommended starting point: R3.xlarge is a good choice with 4 cores / 60 GB RAM R3.2xlarge is a good choice with 8 cores / 60 GB RAM R3.4xlarge is a good choice with 16 cores / 122 GB RAMNotice that other larger instances are available and could be used. Some of the largerservers may require an email to AWS support first, depending on your organization’scurrent usage and limits. If a larger server may be needed, please refer to the nextsection: “Sizing and Scalability”.Note: while it is better to go through all the steps, if the ‘Review and Launch button isclicked at this point, a warning about security groups will come up. Change from thedefault security group to “Qlik Sense” security group that was previously created.c.Configure Instance: this section will show the default configuration options. On this screen, the newinstance could be assigned to the default VPC network or to a new one if needed. Set your VPIC, or ifthere are no changes, click ‘Next’ to continue.d.Add Storage: this section will show the default storage assigned to the new instance. Typically, thedefault disk type assigned (general purpose SSD) will be sufficient. For any scenario other than asimple proof of concept, we recommend you increase the size of the root volume to at least 100GB . IfQlik Sense installation Guide for AWS 8
the instance is going to be used for a Production environment where large volumes of data will behandled with frequent reload of Qlik Sense applications, the speed of the disk may become abottleneck. In this case you may need to increase the size that was allocated to the Qlik Sense system.Notice that the root volume can be increase accordingly and in some cases, there will bean attached Z: volume (usually referred as 'instance store' or 'instance storage') to yourinstance where the extra space has been allocated. The Z: volume is a temporary volumethat will be cleaned every time the instance is stopped. Without modification, it is then notrecommended to install anything on this drive as the installation would be lost upon thefirst restart. The data in an instance store persists only during the lifetime of itsassociated instance. Hence, if an instance reboots (intentionally or unintentionally), theinstance store (Z: drive) persists. However, the data that was st ored on the Z: drive willbe gone. It is a good practice to increase the C: drive as needed to store the data thereinstead.While Qlik's engine is primarily memory intensive, larger configurations requireappropriate disk I/O resources as well. Should you need to increase disk performance,the following link provides some guidance for selecting the appropriate disk configuration(including attaching additional EBS volumes for extra storage with the tails/#piopsThe following link describes the process of adding an EBS volume to an 2/latest/UserGuide/ebs-attachingvolume.html?icmpid docs ec2 consolee.Tag instance: this section will allow to specify a tag name for the instance. Tags enable you tocategorize your AWS resources in different ways, for example, by purpose, owner, or environment. Thefollowing link provides more information on how to set up tags. A blank tag is the default value andwhile a tag name doesn’t have to necessarily be provided, it is est/UserGuide/Using Tags.html?icmpid docs ec2 consoleQlik Sense installation Guide for AWS 9
f.Configure Security Group: this section will allow to specify the Security Group that should be used forthe new instance. Select the option to use an existing Security Group and select the ‘Qlik Sense’security group that was previously created. Click ‘Review and Launch’ when ready.After selecting the ‘Qlik Sense’ security group, the Inbound rules configured will bedisplayed:g.4.Review: in this last section, the configuration of the instance can be reviewed before being launched.Any modification needed can be made by either using the Back button or the links on the top . Click‘Launch’ when ready.When launched, a warning about key pairs will show up. Choose either an existing key pair or the“Qlik Sense” key pair that was previously created and saved.Qlik Sense installation Guide for AWS 10
5.(optional) Create an Elastic IP so that the server IP address doesn’t change even when theinstance is stopped and started again. Please notice that there is a cost associated with this step,hence it is marked as optional. Create an elastic IP via EC2 Elastic IPs Allocate New Address Yes, Allocate Select the new Elastic IP address, and click Associate Address Choose the running instance and click Associate6.Go back to the EC2 console and wait for the new instance to say “running” and “2/2 checkspassed”7.RDP to the new instance Using the left hand side navigation panel, Click EC2 Instances Choose yourinstance Connect Click Download Remote Desktop File Saves a RDP link you can use to connectQlik Sense installation Guide for AWS 11
Click Get Password Choose your QlikSense.pem file Click Decrypt password If applicable, securely store this string for future referenceClicking on the Remote Desktop file that was downloaded will open an RDP session on theserver. The decrypted password is needed in order to access the instance. After the initial login,the local Administrator password on the instance can be changed, if desired.8.Given that the new instance is not part of an existing domain, some local users need to be set upin order to use Qlik Sense as well as a Service Account to run Qlik Sense Server. In order tocreate a Service Account, follow the next steps. Note that this account will be used during theinstallation of Qlik Sensea.On the new instance that is going to be used as the Qlik Sense server, open the ComputerManagement windowb.Find the Users folder, then click Action New Userc.Enter a user name (i.e. 'QSAdmin') and a password (i.e. 'QlikSense!')Qlik Sense installation Guide for AWS 12
d.Uncheck 'User must change password at next logon'e.Check 'User cannot change password' and 'Password never expires'f.Click Createg.Double click on 'QSAdmin'h.Click 'Member Of'i.Click Addj.Type 'Administrators'k.Click 'Check Names'l.Click 'OK'm. Click 'Users'n.Click 'Remove'o.Click 'OK'In
Qlik Sense on AWS Deployment Guide 9.19.2016 Contents 1. Abstract 2. Introduction a. Qlik Sense: A Primer 3. Qlik on AWS - high level Architecture 4. Installation 5. Scalability and Sizing 6. Connectivity a. How to ge
