Build Your Cluster With Rocks PDF Free Download

2y ago

53 Views

1 Downloads

3.32 MB

52 Pages

Report/dmca

Download PDF

Transcription

Build Your Cluster with RocksYu FuUniversity of Florida2011 OSG Summer WorkshopLubbock, TX

You need a cluster Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

and an efficient tool to build it Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Clusters are cost/performance orientedcomputational engines, but are hard tomanage. Cluster management gets linearly harderas it scales out and as more and morefrequent updates come for modern OS. Heterogeneous nodes are a bummer.Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

What is Rocks? Rocks Cluster Distribution is a Linux distribution intended for high-performancecomputing clusters.Started by National Partnership for AdvancedComputational Infrastructure and the SCSDin 2000.Used by 1800 registered clusters so far andmany more unregistered clusters.The biggest registered academic Rockscluster as of now: 8632 CPUs.Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Rocks Overview Based on CentOS Linux. Uses standard RPM as package tool. Uses automatically generated custom RedHat kickstart to control the whole installation.Is NOT “system imager” based.All nodes are “installed, not “imaged”.Managed with MySQL and XML.Highly customizable.Supports heterogeneous nodes, even crossinstall a hybrid of i386 and x86 64.Excellent scalability with “Avalanche” installmode based on BitTorrent p2p transfer tool.Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Topology of RocksBuild Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

What You Need To Get Started A bunch of x86/x86 64 machines – you can even build a Rocks cluster on virtual machines.1GB memory on each machine for Rocks 5.4.30GB hard drive for Rocks 5.4 default install.Two Ethernet ports on one of them – thefrontend / headnode.PXE booting feature on worker nodes – can useCD or floppy emulators.A network switch to connect them up.Download Rocks ISO images and burn toCDs/DVDs – a single “Jumbo” DVD would meetneeds in most startup cases, you can add morerolls later.Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Build a Rocks Cluster Rocks provides for installation of a clusterwith minimum user interaction. Steps involved– Boot from CD or DVD on the frontend.– Answer a few questions.– Get favorite beverage – You’re on your way to a full fledged cluster in2 hours or less: 1 hour for the frontendserver install, 30 min for worker node, 20extra sec for each additional worker node.Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Let’s Start!Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Minimum Rolls for a frontend: base, kernel, os, web-serverBuild Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Default Rocks 5.4 frontend server disk partitions:/16 GB/var4 GBswap1 GB/export (symbolic link to /state/partition1)remainder of the primary diskBuild Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Manual partitioning:Minimum of 16GB for / partition.Must have a /export partition.Software RAIDs are supported, but LVM is not supported as in 5.4.Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Install Worker Nodes Run “insert-ethers” on frontend:Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Boot worker node from PXE.insert-ether will automatically detect it, and start install on it.Worker nodes will be named as compute- rack # - slot # by default.Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

From this point, “rocks-console” can display node’s screen. Based on VNC, no KVM switch is needed!Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Once a worker nodes is recognized by insert-ethers, you can go to the next node.Repeat the above procedure, until a whole rack isfinished.Then do the next rack:– Stop insert-ethers by F8.– Re-launch insert-ether with the rack number:insert-ethers --rack 1– Boot up worker nodes in this rack one by one from PXEuntil the last node. Repeat until the last rack. Your cluster is built!Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Customize Your Rocks Cluster Rocks kickstart is generated on-the-fly, controlled bythe graph XML s Customization can be done in user-defined XMLs.–––––PartitionsPackagesExtra filesPre-install scriptsPost-install scripts Check kickstart before install:/export/rocks/install/sbin/kickstart.cgi –c node nameBuild Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

An Example of Custom XML: ?xml version "1.0" standalone "no"? kickstart description Enable SSH /description main !-- kickstart 'main' commands go here -- /main pre !-- partitioning commands go here -- /pre package ssh /package package ssh-clients /package package ssh-server /package package ssh-askpass /package post cat > /etc/ssh/ssh config << 'EOF'Host rotocol1,2EOF /post /kickstart Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Customize Partitions A single drive example: pre echo “clearpart --all --initlabel --drives sdapart / --size 8000 --ondisk sdapart swap --size 1000 --ondisk sdapart /data --size 0 --grow --ondisk sda” >/tmp/user partition info /pre Software RAID pre echo “clearpart --all --initlabel --drives sda,sdbpart / --size 8000 --ondisk sdapart swap --size 1000 --ondisk sdapart raid.00 --size 20000 --ondisk sdapart raid.01 --size 20000 --ondisk sdbraid /data --level 1 --device md0 raid.00 raid.01” >/tmp/user partition info /pre Manual partitioningecho “rocks manual” > /tmp/user partition infoBuild Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Install Extra Software Packages All Rocks packages are installed and managedthrough RPM.– Prepare the RPM– Put it in /export/rocks/install/contrib/5.4/arch/RPMS– Add it in the XML: package your package /package – Re-create the Rocks distro:# cd /export/rocks/install# rocks create distroBuild Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Update a Software The same method can also be used to update asoftware in the Rocks. Rocks will automaticallypickup and install the newer version on workernodes. Get the RPM of the new version, e.g.:wget x86 64/SL/kernel-2.6.18-238.9.1.el5.x86 64.rpm Put it in /export/rocks/install/contrib/5.4/arch/RPMSor S Re-create the Rocks distro:# cd /export/rocks/install# rocks create distroBuild Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Non-RPM SoftwareTo share files/software not available in RPM,you can put them in Rocks system NFS:– On the frontend, go to the directory/share/apps.– Add the files you'd like to share within thisdirectory.– All files there will be available on the computenodes under: /share/appsBuild Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Custom Config Scripts Post-install custom config scripts can be put in the post /post section of the XMLs.Just like regular Shell scripts, but be careful with XMLreserved characters, e.g. replace “ ” with “>” etc. post cat > /etc/ssh/ssh config << 'EOF'Host rotocol1,2EOF /post Other scripts such as Python are also possible.Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Rocks DatabaseAutomated nodediscoveryNode 0MySQL DBinsert-ethersNode dhcpd.confBuild Your Cluster with RocksYu FuNode Nkickstart2011 OSG Summer Workshop, Lubbock, TX

Older versions of Rocks had a convenient web-based tool to manage its MySQLdatabase. Rocks 5 provides handy commands to deal with database-relatedtasks and no longer encourages user to directly access its MySQL.Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Rocks ‘n Rolls Rolls are Rocks Modules. It is the mechanism for delivery of packages and configuration in Rocks.What is inside a roll?––––Packaged binaries: RPMsAutomatic configuration dataInstallation mapA roll is basically a collection of RPMs plus Kickstart graph XMLspacked in the ISO form. There are tons of rolls readily available: ganglia, hpc, bio, sge, torque, condor, xen You can build your own roll!As OSG releases are moving to RPM, we may have an OSGroll in the future!Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Rocks 411 Service The 411 Secure Information Serviceprovides NIS-likefunctionality for Rocksclusters with bettersecurity and scalability.It is basically a file syncmechanism with listenerand poll agents.Unlike NIS, changes mustbe made on frontend.Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Cross-architecture Install Rocks is able to heterogeneous nodeseven with different architectures. Rocks can install i386 nodes from anx86 64 frontend. Rocks can install x86 64 nodes from ani386 frontend. It is quite simple to do so in Rocks 5.4,see details in User’s n/base/5.4/cross.htmlBuild Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Rocks Upgrade – Keep Configs Very simple!1. Create a backup roll of currentcustom version:# cd /export/siteroll/rocks/src/roll/restore# make roll2. Install a clean new version.3. Include the backup roll at thebeginning of the install.4. Done! It is also a handy way toback up the frontend.Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Further Information Rocks User’s on/base/5.4/ Rocks Wiki: https://wiki.rocksclusters.org Rocks User’s Mailing -rocks-discussionBuild Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

DemoBuild Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Questions?Build Your Cluster with RocksYu Fu2011 OSG Summer Workshop, Lubbock, TX

Build Your Cluster with Rocks Yu Fu 2011 OSG Summer Workshop, Lubbock, TX What is Rocks? Rocks Cluster Distribution is a Linux distribution intended for high-performance computing clusters. Started by National Partnership for Advanced Computational Infrastructure and the SCS