Transcription
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]TCSS 562:SOFTWARE ENGINEERINGFOR CLOUD COMPUTINGAWS Demo II,Cloud Enabling TechnologyWes J. LloydSchool of Engineering and TechnologyUniversity of Washington – TacomaTR 5:00-7:00 PM1OFFICE HOURS – FALL 2021 Tuesdays: 4:00 to 4:30 pm - CP 229 7:15 to 7:45 pm – ONLINE via Zoom Thursdays 4:15 to 4:45 pm – ONLINE via Zoom 7:15 to 7:45 pm – ONLINE via Zoom Or email for appointment Zoom Link sent as Canvas Announcement O f f i c e H ou r s set b ased on Stu dent Demogr ap hi c s su r vey f eedb ac kNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.22Slides by Wes J. LloydL11.1
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]OBJECTIVES – 11/4 Questions from 11/2 Tutorial 3 - Best Practices with EC2 Tutorial 4 – Intro to FaaS – AWS Lambda Tutorial 5 – Intro to FaaS II – Files in S3, CloudWatch Term Project Proposals – AWS overview and demonstration 2 nd hour: AWS overview and demonstration Ch. 5: Cloud Enabling Technology Team planningNovember 4, 2021TCSS562:Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.33ONLINE DAILY FEEDBACK SURVEY Daily Feedback Quiz in Canvas – Take After Each Class Extra Creditfor completingNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.44Slides by Wes J. LloydL11.2
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-TacomaNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - Tacoma[Fall 2021]L11.55MATERIAL / PACE Please classify your perspective on material covered in today’sclass (25 respondents): 1-mostly review, 5-equal new/review, 10-mostly new Average – 6.04 ( - previous 6.52) Please rate the pace of today’s class: 1-slow, 5-just right, 10-fast Average – 5.20 ( - previous 5.36)November 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.66Slides by Wes J. LloydL11.3
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]FEEDBACK FROM 11/2 What is burning an image?and why does IS (Instance Storage) need to burn an image? “burning an image” is the notion of capturing all of the datacontained on a virtual disk and compressing it into an . imgfile A disk image is the full collection of bytes that contains thefile system the data In Ubuntu we typically format disks using the ext4 file system To recreate a disk in the cloud, we need the image whichcontains the file system the data IS (Instance Store) are local disks that do not persist If you want to save and restore data from an IS disk, imagefiles can be usedNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.77MAKE AN IMAGE FROM A DISK# ********** ******ON T HE LO C A L C O M P U T E R * * * * * * * * * * * * * * * * * * *# create 120 0 MB vir tual disk 1 , 2 5 8 , 2 9 1 , 2 0 0 b y t e ssu do dd if / dev/zero of vhd.i m g bs 1 M count 1200# format the disk us ing the e x t 4 f i l e s y s t e msu do mkfs.ex t4 vhd.i mg# mount the disk at "/ mn t "su do mount - t auto - o lo op v h d . i m g / m n t# check that the dis k is moun t e ddf - h# create a h ello fil e (o r cop y d a t a ) t o t h e n e w v i r t u a l d i s kcd /mntsu do echo "h ello wor ld ! " hello.tx tls - lcd# unmount th e virtua l di sksu do umount /mntNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.88Slides by Wes J. LloydL11.4
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]COMPRESS IMAGE, PUSH TO S3# compress the diskbzip2 vhd.img# push the disk image to S3aws s3 cp vhd.img.bz2 s3://tcss562-f21-imagesNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.99RESTORE ON THE CLOUD# * ** ** ** ** *** ** ** **O N T HE A WS E C2 VM * ** ** * *** ** ** ** ** **# w it h th e aws cl i in st a lle d an d co nfi gu re d# d ow nl oa d the i ma ge f r om S3aws s 3 cp s 3:/ /t cs s5 62 - f21 -i ma ge s/ vhd .i mg .b z2 vhd .i mg .b z2# u nc om pr es s t he i ma gebzi p2 - d vh d.i mg .b z2# w e ne ed t o c al cu la tepar ti ti on# d is k se ct ors a re 5 12# d iv id e th e d is k si ze# s ec to rs 12 58 29 12 00the n um be r of se ct or s f or th ebyt es e ac hby 51 2 to d ete rm in e se c tor s/ 5 12 2 45 964 8# c re at e a dis k pa rt it i on fo r th is di sk t ha t i s# 2 45 96 48 s ect or s in s i ze us in g th e e ph em er al dri ve o r# a n ew ly m oun te d EB S v olu me t ha t is un fo rm at t edsud o fd is k /de v/ nv me 1n 1November 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.1010Slides by Wes J. LloydL11.5
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]PARTITION THE DISKWelcome to fdis k (util-lin ux 2.34).Command (m for help): nPar titio n typep primar y (0 primar y, 0 extended, 4 free)e extended (contai ner for log ical par titions )Select (default p): pPar titio n number (1 -4, default 1): 1Fir s t s ector (2048 -9765 624 9, default 2048): 2 0 4 8Las t s ector, / -s ector s or / -s ize{K,M , G, T,P } (2048 -976 56 24 9, default97656249) : 24 5 9 6 4 8Created a new par tition 1 of type 'Linux' and of s ize 1 .2 GiB.Command (m for help): tSelected par tition 1Hex code (type L to lis t all codes ): 83Chang ed type of par titio n 'Linux' to 'Linux'.Command (m for help): w ( to w r i te a n d ex i t )November 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.1111COPY DATA TO NEW DISK PARTITION# now check if the partition has been created.# it should be listed as /dev/nvme1n1p1:ls /dev/nvme1n1*# now copy the data to the partitionsudo dd if vhd.img of /dev/nvme1n1p1# mount the disksudo mount /dev/nvme1n1p1 / mnt# and check if the hello file is therecat /mnt/hello.txt####we were able to copy the disk image to the cloudand we never had to format the cloud diskthis examples copies a filesystem from a local diskto the cloud diskNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.1212Slides by Wes J. LloydL11.6
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]FOR MORE INFORMATION Example script: 562/examples/copy -disk-to-cloud.sh URLs: https://help.ubuntu.com/community/DriveImaging https://www.tecmint.com/create -vir tual-harddisk-volume-inlinux/November 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.1313OBJECTIVES – 11/4 Questions from 11/2 Tutorial 3 - Best Practices with EC2 Tutorial 4 – Intro to FaaS – AWS Lambda Tutorial 5 – Intro to FaaS II – Files in S3, CloudWatch Term Project Proposals – AWS overview and demonstration 2 nd hour: AWS overview and demonstration Ch. 5: Cloud Enabling Technology Team planningNovember 4, 2021TCSS562:Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.1414Slides by Wes J. LloydL11.7
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]OBJECTIVES – 11/4 Questions from 11/2 Tutorial 3 - Best Practices with EC2 Tutorial 4 – Intro to FaaS – AWS Lambda Tutorial 5 – Intro to FaaS II – Files in S3, CloudWatch Term Project Proposals – AWS overview and demonstration 2 nd hour: AWS overview and demonstration Ch. 5: Cloud Enabling Technology Team planningNovember 4, 2021TCSS562:Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.1515OBJECTIVES – 11/4 Questions from 11/2 Tutorial 3 - Best Practices with EC2 Tutorial 4 – Intro to FaaS – AWS Lambda Tutorial 5 – Intro to FaaS II – Files in S3, CloudWatch Term Project Proposals – AWS overview and demonstration 2 nd hour: AWS overview and demonstration Ch. 5: Cloud Enabling Technology Team planningNovember 4, 2021TCSS562:Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.1616Slides by Wes J. LloydL11.8
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]OBJECTIVES – 11/4 Questions from 11/2 Tutorial 3 - Best Practices with EC2 Tutorial 4 – Intro to FaaS – AWS Lambda Tutorial 5 – Intro to FaaS II – Files in S3, CloudWatch Term Project Proposals – AWS overview and demonstration 2 nd hour: AWS overview and demonstration Ch. 5: Cloud Enabling Technology Team planningNovember 4, 2021TCSS562:Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.1717TERM PROJECTS Team 1: Cloud Data Streaming Case StudySatchit Dahal, Amir Almemar, Alekhya Palle Team 2: Deep Learning based Face Recognition for SmartHome SystemZichao Zhang, Zhifei Cheng, Sijin Huang Team 3: Natural Language Processing pipeline(data preparation training inferencing)Bob Schmitz, Viktoriya Grishkina, Danielle Lambion Team 4: – there is no team 4 Team 5: Serverless Imaging Processing Pipeline with OpenCVShishir Reddy Team 6: TLQ Pipeline, Programming Language ComparisonGuanchen Zhao, Minzhi Qu, Yanliu WangNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.1818Slides by Wes J. LloydL11.9
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]TERM PROJECTS - 2 Team 7: TLQ Pipeline, Service CompositionAndrew Lim, Di Mo, Solmaz Seyed Monir Team 8: TLQ Pipeline, Service CompositionDuy Tran, Pragati Chidanand Patil, Ranjana Bongale Ganesh Team 9: TLQ Pipeline, Platform Comparison (Azure vs AWS)Sri Vibhu Paruchuri, Nischal Khadka, Dev Gandhi Team 10: TLQ Pipeline, Database Comparison: RDS vs AuroraAshwin Meena Meiyappan, Ayushi Ameta, Ananya Ra Team 11: Secure Multi-party Computation (MPC)Davis Railsback, Trina Pal, Parshva Kotak Team 12: TLQ Pipeline, Programming Language ComparisonShuo Peng, Anmin Huang Team 13: IoT/Edge Computing Literature Survey PaperAnindya DeyNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.1919OBJECTIVES – 11/4 Questions from 11/2 Tutorial 3 - Best Practices with EC2 Tutorial 4 – Intro to FaaS – AWS Lambda Tutorial 5 – Intro to FaaS II – Files in S3, CloudWatch Term Project Proposals – AWS overview and demonstration 2 nd hour: AWS overview and demonstration Ch. 5: Cloud Enabling Technology Team planningNovember 4, 2021TCSS562:Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.2020Slides by Wes J. LloydL11.10
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-TacomaFUNCTION-AS-A-SERVICE[Fall 2021]AWSLambdaDemo2121AWS OVERVIEWAND DEMO22Slides by Wes J. LloydL11.11
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]LIST OF TOPICS AWS Management Console Elastic Compute Cloud(EC2) Instance Storage:Virtual Disks on VMs Elastic Block Store:Virtual Disks on VMs Elastic File System (EFS) Amazon Machine Images(AMIs) EC2 Paravirtualization EC2 Full Virtualization(hvm) EC2 Virtualization EvolutionNovember 4, 2021 (VM) Instance Actions EC2 Networking EC2 Instance MetadataService Simple Storage Service (S3) AWS Command LineInterface (CLI) Legacy / Service SpecificCLIs AMI Tools Signing Certificates Backing up live disks Cost Savings MeasuresTCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.2323AMAZON MACHINE IMAGES AMIs Unique for the operating system (root device image) Two types Instance store Elastic block store (EBS) Deleting requires multiple steps Deregister AMI Delete associated data (delete snapshot) - (files actually in S3) Forgetting both steps leads to costly “orphaned” data No way to instantiate a VM from deregistered AMIs Data still in S3 resulting in chargesNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.2424Slides by Wes J. LloydL11.12
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]EC2 VIRTUALIZATION - PARAVIRTUAL 1 st , 2 nd , 3 rd , 4 th generation XEN-based 5 th generation instances AWS Nitro vir tualization XEN - two virtualization modes XEN Paravirtualization “paravirtual” 10GB Amazon Machine Image – base image size limit Addressed poor performance of old XEN HVM mode I/O performed using special XEN kernel with XEN paravirtual modeoptimizations for better performance Requires OS to have an available paravirtual kernel PV VMs: will use common AKI files on AWS – Amazon kernelimage(s) Look for common identifiersNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.2525EC2 VIRTUALIZATION - HVM XEN HVM mode Full virtualization – no special OS kernel required Computer entirely simulated MS Windows runs in “hvm” mode Allows work around: 10GB instance store root volume limit Kernel is on the root volume (under /boot) No AKIs (kernel images) Commonly used today (EBS-backed instances)November 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.2626Slides by Wes J. LloydL11.13
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]EC2 VIRTUALIZATION - NITRO Nitro based on Kernel-based-virtual-machines Stripped down version of Linux KVM hypervisor Uses KVM core kernel module I/O access has a direct path to the device Goal: provide indistinguishable performance from baremetalNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.2727EVOLUTION OF AWS VIRTUALIZATION Fro m: h t t p : / / w w w. b r e n d a n g r e g g . c o m / b l o g / 2 0 17 - 1 1 - 2 9 / a w s - e c 2 - v i r t u a l i z a t i o n - 2 017. h t m lNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.2828Slides by Wes J. LloydL11.14
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]INSTANCE ACTIONS Stop Costs of “pausing” an instance Terminate Reboot Image management Creating an image EBS (snapshot) Bundle image Instance-storeNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.2929EC2 INSTANCE: NETWORK ACCESS Public IP address Elastic IPs Costs: in-use FREE, not in-use 12 /day Not in-use (e.g. “paused” EBS-backed instances) Security groups E.g. firewall Identity access management (IAM) AWS accounts, groups VPC / Subnet / Internet Gateway / Router NAT-GatewayNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.3030Slides by Wes J. LloydL11.15
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]SIMPLE VPC Recommended when using Amazon EC2November 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.3131VPC SPANNING AVAILABILITY ZONES32Slides by Wes J. LloydL11.16
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]INSPECTING INSTANCE INFORMATION EC2 VMs run a local metadata service Can query instance metadata to self discover cloudconfiguration attributes Find your instance ID:curl http://169.254.169.254/curl http://169.254.169.254/latest/curl http://169.254.169.254/latest/meta -data/curl http://169.254.169.254/latest/meta -data/instance -id; echo ec2-get-info command Python API that provides easy/formatted access to metadataNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.3333SIMPLE STORAGE SERVICE (S3) Key-value blob storage What is the difference vs. key -value stores (NoSQL DB)? Can mount an S3 bucket as a volume in Linux Supports common file-system operations Provides eventual consistency Can store Lambda function state for life of container.November 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.3434Slides by Wes J. LloydL11.17
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]AWS CLI Launch Ubuntu 16.04 VM Instances Launch Instance Install the general AWS CLI sudo apt install awscli Create config file[default]aws access key id access key id aws secret access key secret access key region us-east-1November 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.3535AWS CLI - 2 Creating access keys: IAM Users Security Credentials Access Keys Create Access KeysNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.3636Slides by Wes J. LloydL11.18
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]AWS CLI - 3 Export the config file Add to /home/ubuntu/.bashrcexport AWS CONFIG FILE HOME/.aws/config Try some commands: awsawsawsawsawsawsawshelpcommand helpec2 helpec2 describes-instances --output textec2 describe-instances --output jsons3 lss3 ls vmscaleruwNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.3737LEGACY / SERVICE SPECIFIC CLI(S) sudo apt install ec2-api-tools Provides more concise output Additional functionality Define variables in . bashrc or another sourced script: export AWS ACCESS KEY {your access key} export AWS SECRET KEY {your secret key} ec2-describe-instances ec2-run-instances ec2-request-spot-instances EC2 management from Java: oc/index.html Some AWS ser vices have separate CLI installable by packageNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.3838Slides by Wes J. LloydL11.19
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]AMI TOOLS Amazon Machine Images tools For working with disk volumes Can create live copies of any disk volume Your local laptop, ec2 root volume (EBS), ec2 ephemeral disk test/UserGuide/ami-tools-commands.html AMI tools reference: e/ami-tools-commands.html Some functions may require private key & certificate filesNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.3939PRIVATE KEY AND CERTIFICATE FILE Install openssl package on VM# generate private key file openssl genrsa 2048 mykey.pk# generate signing certificate file openssl req -new -x509 -nodes -sha256 -days 36500 -keymykey.pk -outform PEM -out signing.cert Add signing.cert to IAM Users Security Credentials - - new signing certificate - From: /set up-ami-tools.html?icmpid docs iam console#ami -tools-createcertificateNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.4040Slides by Wes J. LloydL11.20
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]PRIVATE KEY, CERTIFICATE FILE These files, combined with your AWS ACCESS KEY andAWS SECRET KEY and AWS ACCOUNT ID enable you topublish new images from the CLI Objective:1. Configure VM with software stack2. Burn new image for VM replication (horizontal scaling) An alternative to bundling volumes and storing in S3 isto use a containerization tool such as Docker. . . Create image script . . .November 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.4141SCRIPT: CREATE A NEW INSTANCE STOREIMAGE FROM LIVE DISK VOLUMEimage 1echo "Burn image image"echo " image" image.idmkdir /mnt/tmpAWS KEY DIR /home/ ubuntu/.awsexport EC2 URL http://ec2.amazonaws.comexport S3 URL https://s3.amazonaws.comexport EC2 PRIVATE KEY {AWS KEY DIR}/mykey.pkexport EC2 CERT {AWS KEY DIR}/signing.certexport AWS USER ID {your account id}export AWS ACCESS KEY {your aws access key}export AWS SECRET KEY {your aws secret key}ec2-bundle-vol -s 5000 -u {AWS USER ID} -c {EC2 CERT} -k {EC2 PRIVATE KEY}--ec2cert /etc/ec2/amitools/cert-ec2.pem --no-inherit -r x86 64 -p image -i/etc/ec2/amitools/cert-ec2.pemcd /tmpec2-upload-bundle -b tcss562 -m image.manifest.xml -a {AWS ACCESS KEY} -s {AWS SECRET KEY}--url http://s3.amazonaws.com --location USec2-register tcss562/ image.manifest.xml --region us-east-1 --kernel aki88aa75e1November 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.4242Slides by Wes J. LloydL11.21
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]COST SAVINGS MEASURES From Tutorial 3: #1: ALWAYS USE SPOT INSTANCES FOR COURSE/RESEARCHRELATED PROJECTS #2: NEVER LEAVE AN EBS VOLUME IN YOUR ACCOUNT THAT ISNOT ATTACHED TO A RUNNING VM #3: BE CAREFUL USING PERSISTENT REQUESTS FOR SPOTINSTANCES #4: TO SAVE/PERSIST DATA, USE EBS SNAPSHOTS AND THEN #5: DELETE EBS VOLUMES FOR TERMINATED EC2 INSTANCES. #6: UNUSED SNAPSHOTS AND UNUSED EBS VOLUMES SHOULDBE PROMPTLY DELETED !! #7: USE PERSISTENT SPOT REQUESTS AND THE “STOP”FEATURE TO PAUSE VMS DURING SHORT BREAKSNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.4343WE WILL RETURN AT 6:15 PM44Slides by Wes J. LloydL11.22
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]OBJECTIVES – 11/4 Questions from 11/2 Tutorial 3 - Best Practices with EC2 Tutorial 4 – Intro to FaaS – AWS Lambda Tutorial 5 – Intro to FaaS II – Files in S3, CloudWatch Term Project Proposals – AWS overview and demonstration 2 nd hour: AWS overview and demonstration Ch. 5: Cloud Enabling Technology Team planningNovember 4, 2021TCSS562:Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.4545OBJECTIVES – 11/4 Questions from 11/2 Tutorial 3 - Best Practices with EC2 Tutorial 4 – Intro to FaaS – AWS Lambda Tutorial 5 – Intro to FaaS II – Files in S3, CloudWatch Term Project Proposals – AWS overview and demonstration 2 nd hour: AWS overview and demonstration Ch. 5: Cloud Enabling Technology Team planningNovember 4, 2021TCSS562:Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.4646Slides by Wes J. LloydL11.23
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]CLOUD ENABLINGTECHNOLOGYNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.4747CLOUD ENABLING TECHNOLOGY Adapted from Ch. 5 from Cloud ComputingConcepts, Technology & Architecture Broadband networks and internet architecture Data center technology Virtualization technology Multitenant technology Web/web services technologyNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.4848Slides by Wes J. LloydL11.24
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]1. BROADBAND NETWORKSAND INTERNET ARCHITECTURE Clouds must be connected to a network Inter-networking: Users’ network must connect to cloud’snetwork Public cloud computing relies heavily on the internetNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.4949PRIVATE CLOUD NETWORKING For institutions with in-house private cloudsremote usersin-office usersNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.5050Slides by Wes J. LloydL11.25
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]PUBLIC CLOUD NETWORKING Resources can beextended by addingpublic cloud Places furtherdependency on theinternet to provideconnectivityNovember 4, 2021private cloudnetworkremote usersin-office usersTCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.5151INTERNETWORKING KEY POINTS Cloud consumers and providers typically communicate via theinternet Decentralized provisioning and management model is notcontrolled by the cloud consumers or providers Inter-networking (internet) relies on connectionless packetswitching and route-based interconnectivity Routers and switches support communication Network bandwidth and latency influence QoS, which isheavily impacted by network congestionNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.5252Slides by Wes J. LloydL11.26
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]CLOUD ENABLING TECHNOLOGY Adapted from Ch. 5 from Cloud ComputingConcepts, Technology & Architecture Broadband networks and internet architecture Data center technology Virtualization technology Multitenant technology Web/web services technologyNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.53532. DATA CENTER TECHNOLOGY Grouping servers together (clusters): Enables power sharing Higher efficiency in shared IT resource usage(less duplication of effort) Improved accessibility and organization Key components: Virtualized and physical server resources Standardized, modular hardware Automation support: enable server provisioning,configuration, patching, monitoring withoutsupervision tool/API support is desirableNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.5454Slides by Wes J. LloydL11.27
TCSS 562: Software Engineering for Cloud ComputingSchool of Engineering and Technology, UW-Tacoma[Fall 2021]CLUSTER MANAGEMENT TOOLSExample:Hyak ClusterUW-SeattleNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of Washington - TacomaL11.5555DATA CENTER TECHNOLOGY –KEY COMPONENTS Remote operation / management High availability suppor t : **redundant everything**Includes: power supplies, cabling, environmental controlsystems, communication links, duplicate warm replica HW Secure design: physical and logical access control Ser vers: rackmount, etc. Storage: hard disk arrays (RAID) storage area network (SAN): disk array w/ multiple servers(individual nodes w/ disks) and a dedicated network network attached storage (NAS): inexpensive single node withcollection of disks, provides shared filesystems, for NFS, etc. Network hardware: backbone routers (WAN to LANconnectivity), firewalls, VPN gateways, managedswitches/routersNovember 4, 2021TCSS562: Software Engineering for Cloud Computing [Fall 2021]School of Engineering and Technology, University of
TCSS 562: Software Engineering for Cloud Computing School of Engineering and Technology, UW-Tacoma [Fall 2021] Slides by Wes J. Lloyd L11.5 # compress the disk bzip2 vhd.img # push the disk image to S3 aws s3 cp vhd.img.bz2 s3://tcss562-f21-images November 4, 2021 TCSS562: Software Eng
