WIXLIB

Basics of Cloud Computing – Lecture 3Scaling Applications on the CloudSatish Srirama

Outline Scaling Information Systems Scaling Enterprise Applications in the Cloud Auto Scaling25/02/2014Satish Srirama2/37

Scaling Information Systems Fault tolerance, high availability & scalability areessential prerequisites for any enterpriseapplication deployment Scalability– Generally nodes in information systems supportspecific load– When load increases beyond certain level, systemsshould be scaled up– Similarly when load decreases they should be scaleddown25/02/2014Satish Srirama3/37

Scaling Information Systems continued Two basic models of scaling– Vertical scaling Also known as Scale-up– Horizontal scaling Aka Scale-out25/02/2014Satish Srirama4/37

Typical Web-based EnterpriseApplicationSource: http://en.wikipedia.org/wiki/File:LAMPP Architecture.png25/02/2014Satish Srirama5/37

Num. of serversTypical Load on an Application Server22201816141210864Optimal policyAuto ScaleAlways OnArrival rate 03002001000024681012Time [hours]Load on a server during 24-hour period.(ClarkNet traces) [Vasar et al,Nordicloud 2012]25/02/2014Satish Srirama6/37

Vertical scaling Achieving better performance by replacing anexisting node with a much powerful machine Risk of loosing currently running jobs– Can frustrate customers as the service istemporarily down25/02/2014Satish Srirama7/37

Horizontal scalingArrival rate [req/sec] Num. of servers Achieving better performance by adding morenodes to the system New servers are introduced to the system torun along with the existing servers22201816141210864Optimal policyAuto ScaleAlways 0002468101214Time [hours]Server allocation policies for different loads25/02/2014Satish Srirama8/37

Scaling Enterprise Applications in theCloudClient SiteClient SiteClient SiteLoad Balancer sh SriramaAppServerAppServerMemcache9/37

Load Balancer Load balancing has been a key mechanism inmaking efficient web server farms Load balancer automatically distributes incomingapplication traffic across multiple servers Hides the complexity for content providers 1 1 2– Allows server farms work as a single virtual powerfulmachine 1 1 2– Beyond load distribution, improves response time25/02/2014Satish Srirama10/37

Introduction- types of load balancers Network-Based load balancing : This type of load balancing is providedby IP routers and DNS (domain name servers) that service a pool of hostmachines. For example, when a client resolves a hostname, the DNS canassign a different IP address to each request dynamically based on currentload conditions. Network-Layer based load balancing : The load balancer may balancethe traffic based on the source IP address and/or port of the incoming IPpacket. This type of load balancing does not take into account the contentsof the packet, so is not very flexible. Transport-Layer based load balancing : The load balancer may chooseto route the entire connection to a particular server. This type of loadbalancing is very useful if the connections are short-lived and areestablished frequently. Application-Layer/Middleware based load balancing :This type of loadbalancing is performed in the application-layer, often on a per-session orper-request basis.25/02/2014Satish Srirama11/37

Introduction- classes of load balancers Non-adaptive load balancer: A load balancer can usenon-adaptive policies, such as simple round-robinalgorithm, hash-based or randomization algorithm. Adaptive load balancer: A load balancer can useadaptive policies that utilize run-time information, suchas amount of CPU load on the node to determine theserver to route the request to. Load Balancers and Load Distributors are not the samething– Strictly speaking non-adaptive load balancers are loaddistributors.25/02/2014Satish Srirama12/37

Load balancing algorithms Random: This load balancing method randomly distributesload across the servers available, picking one via randomnumber generation and sending the current connection toit Round Robin: Round Robin passes each new connectionrequest to the next server in line, eventually distributingconnections evenly across the array of machines being loadbalanced Least connection (Join-Shortest-Queue): With this method,the system passes a new connection to the server that hasthe least number of current connections. wSBpc4a5SM25/02/2014Satish Srirama13/37

Examples of Load Balancers Nginx - http://nginx.org/HAProxy - http://haproxy.1wt.eu/Pen - http://siag.nu/pen/Etc.25/02/2014Satish Srirama14/37

Testing the System by Simulating Load Benchmarking tools– Tsung, JMeter, etc Simulating concurrency is also possible Multiple protocols– HTTP, XMPP, etc.– SSL support25/02/2014Satish Srirama15/37

Scaling in the Cloud - bottleneckClient SiteClient SiteClient SiteLoad Balancer y BottleneckCannot leverage elasticitySatish Srirama16/37

Scaling in the Cloud - bottleneckClient SiteClient SiteClient SiteLoad Balancer erScalable and Elastic,MemcacheDB StoresKeyValuebut limited consistency andoperational flexibility NoSQL25/02/2014Satish SriramaLarge-scale Data Processing on theCloud - MTAT.08.036 (Fall17/372014)

Horizontal scaling – further examples MapReduce & Hadoop– We will look as part of Lectures 4 and 525/02/2014Satish Srirama18/37

AutoScaleArrival rate [req/sec]– High availability– Cost saving– Energy savingNum. of servers AutoScale allows systems to dynamically reactto a set of defined metrics and to scaleresources accordingly Providing:22201816141210864Optimal policyAuto ScaleAlways 0002468101214Time [hours]Server allocation policies for different loads25/02/2014Satish Srirama19/37

Typical Usecases Application that see elasticity in their demand Launching a new website with unknownvisitor numbers Viral marketing campaigns A scientific application might also have toscale out– Using 50 machines for 1 hour rather than 1machine for 50 hours25/02/2014Satish Srirama20/37

AutoScaling on the cloud Amazon Autoscale & Elastic Load Balance Vendor neutral autoscaling on cloud– Static Load Balancer Resources estimation onthe fly (e.g optimal heuristics)25/02/2014Satish Srirama21/37

Amazon Auto Scaling Amazon Auto Scaling allows you to scale your computeresources dynamically and predictably (scaling plan):– Dynamically based on conditions specified by you (for example,increasing CPU utilization of your Amazon EC2 instance)– Predictably according to a schedule defined by you (for example,every Friday at 13:00:00). EC2 instances are categorized into Auto Scaling groups forthe purposes of instance scaling and management You create Auto Scaling groups by defining the minimum &maximum no of instances A launch configuration template is used by the Auto Scalinggroup to launch Amazon EC2 instances25/02/2014Satish Srirama22/37

Amazon Auto Scaling - continued Auto Scaling– Monitor the load on EC2instances usingCloudWatch– Define Conditions andraise alarms E.g. Average CPU usageof the Amazon EC2instances, or incomingnetwork traffic frommany different AmazonEC2 instances– Spawn new instanceswhen there is too muchload or remove instanceswhen not enough load25/02/2014Satish Srirama23/37

Amazon CloudWatch Monitor AWS resources automatically– Monitoring for Amazon EC2 instances: seven pre-selected metrics at fiveminute frequency– Amazon EBS volumes: eight pre-selected metrics at five-minute frequency– Elastic Load Balancers: four pre-selected metrics at one-minute frequency– Amazon RDS DB instances: thirteen pre-selected metrics at one-minutefrequency– Amazon SQS queues: seven pre-selected metrics at five-minute frequency– Amazon SNS topics: four pre-selected metrics at five-minute frequency Custom Metrics generation and monitoring Set alarms on any of the metrics to receive notifications or take otherautomated actions Use Auto Scaling to add or remove EC2 instances dynamically based onCloudWatch metrics25/02/2014Satish Srirama24/37

Elastic Load Balance Elastic Load Balance– Automatically distributes incoming applicationtraffic across multiple EC2 instances– Detects EC2 instance health and diverts trafficfrom bad ones– Support different protocols HTTP, HTTPS, TCP, SSL, or Custom Amazon Auto Scaling & Elastic Load Balancecan work together25/02/2014Satish Srirama25/37

Will not be part of the assessmentCLOUDML BASED AUTOSCALE25/02/2014Satish Srirama26

REMICShttp://www.remics.eu/ Reuse and migration of legacy applications tothe cloudTextRequirementsUML/RSLRecover TrackMigrate BlueAge Modelio TALESoA MLClould MLUML/CodeSoA MLPIM4CloudValidateControl and Supervision Metrino FokusMBT RSL Testing25/02/2014 Modelio RedSeeds D2CM Models @ Runtime Model Driven Interoperability Performance monitoringSatish Srirama27/37

CloudML Developed to tame cloud heterogeneity Domain-specific language (DSL) formodelling the provisioning anddeployment at design-time– Nodes, artefacts and bindings can bedefined Different means to manipulate CloudMLmodels– Programmatically via Java API– Declaratively, via serialized model (JSON) Models@Runtime– Dynamic deployment of CloudML basedmodels25/02/2014Satish Srirama[Ferry et al, Cloud 2013]28/37

Cloud-based Performance Multiple approaches are possible Shell– Linux utilities Default– free –m– cat /proc/cpuinfo /prof/meminfo– df –h Sysstat package (iostat, sar)25/02/2014Satish Srirama29/37

Cloud-based Performance - continued Tools (distributed)– Collectd RRDtool– Generating visual performance graphs Multicast communication Does not impact system performance– Cacti RRD GUI Performance decreases by 20%25/02/2014Satish Srirama30/37

Cloud-based Performance - continued Cacti– Spikes denote gathering performance metrics25/02/2014Satish Srirama31/37

Control and Supervision with AutoScaling Autoscaler Java daemon– CollectD – collecting distributed performance metrics– Decides when it is ideal to add new nodes or terminate nodes CPU utilization of all servers on average is 75% in last 5 min CloudML engine and API– Modifying CloudML models– Re-deploying the modified model CloudML Bridge– Generating autoscaling configuration from the CloudMLmodel RRDtool - generating visual performance graphs Apache web server – displaying performance results25/02/2014Satish Srirama32/37

Performance monitoring Autoscaling25/02/2014Satish Srirama33/37

Visualizing CloudML Model25/02/2014Satish Srirama34/37

This week in lab You work with load balancing25/02/2014Satish Srirama35/37

Next Lecture Introduction to MapReduce25/02/2014Satish Srirama36/37

References Amazon Web (Cloud) Services – documentationhttp://aws.amazon.com/documentation/ Elastic Load g/ Load balancing - hms#.UwSBpc4a5SM Auto Scaling - Amazon Web Serviceshttp://aws.amazon.com/autoscaling/ Ferry, N., Chauvel, F., Rossini, A., Morin, B., & Solberg, A. (2013).Managing multi-cloud systems with CloudMF. In Proceedings ofthe Second Nordic Symposium on Cloud Computing & InternetTechnologies (pp. 38-45). ACM.25/02/2014Satish Srirama37/37

Adaptive load balancer: A load balancer can use adaptive policies that utilize run -time information, such as amount of CPU load on the node to determine the servertoroutetherequestto. Load Balancers and Load Distributors are not the same thing -Strictly speaking non-adaptive load balancers are load distributors. 25/02/2014 Satish .