Transcription
GEEK GUIDE CONTAINERS 101Table of ContentsWhat Is a Container and How are Containers Used? 5What Are the Values of Containers? 6Who Are the Container Providers? 8Do Companies Need to Leave the VM StructureEntirely, or Can There Be Hybrid Approaches? 10Why Are Some Firms Waiting to Use Containers? 11What’s Involved in Managing Containers? 14Who Are Some of the Major Players in theContainer Runtime Space? 16Benefits Gained by Switching to Containers—Case Studies 18How Does Configuration Management Applyto Containers, and How Does Puppet Acceleratethe Adoption of Container Technologies? 19Conclusion 25SOL LEDERMAN is a technical people-oriented professional with more than thirtyyears of broad experience in system administration, software design and development,technical support, training, documentation, troubleshooting and customer management.Sol currently divides his time between running IT for a software firm and providing avariety of tech services to the federal government.2
GEEK GUIDE CONTAINERS 101GEEK GUIDES:Mission-critical information for the most technical people on the planet.Copyright Statement 2016 Linux Journal. All rights reserved.This site/publication contains materials that have been created, developedor commissioned by, and published with the permission of, Linux Journal(the “Materials”), and this site and any such Materials are protected byinternational copyright and trademark laws.THE MATERIALS ARE PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND,EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIEDWARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE,TITLE AND NON-INFRINGEMENT. The Materials are subject to change without noticeand do not represent a commitment on the part of Linux Journal or its Web sitesponsors. In no event shall Linux Journal or its sponsors be held liable for technicalor editorial errors or omissions contained in the Materials, including without limitation,for any direct, indirect, incidental, special, exemplary or consequential damageswhatsoever resulting from the use of any information contained in the Materials.No part of the Materials (including but not limited to the text, images, audioand/or video) may be copied, reproduced, republished, uploaded, posted,transmitted or distributed in any way, in whole or in part, except as permitted underSections 107 & 108 of the 1976 United States Copyright Act, without the expresswritten consent of the publisher. One copy may be downloaded for your personal,noncommercial use on a single computer. In connection with such use, you may notmodify or obscure any copyright or other proprietary notice.The Materials may contain trademarks, services marks and logos that are theproperty of third parties. You are not permitted to use these trademarks, servicesmarks or logos without prior written consent of such third parties.Linux Journal and the Linux Journal logo are registered in the US Patent &Trademark Office. All other product or service names are the property of theirrespective owners. If you have any questions about these terms, or if you wouldlike information about licensing materials from Linux Journal, please contact usvia e-mail at info@linuxjournal.com.3
GEEK GUIDE CONTAINERS 101About the SponsorPuppetPuppet is driving the movement to a world of unconstrainedsoftware change. Its revolutionary platform is the industrystandard for automating the delivery and operation of thesoftware that powers everything around us. More than33,000 companies—including more than 75 percent of theFortune 100—use Puppet’s open source and commercialsolutions to adopt DevOps practices, achieve situationalawareness and drive software change with confidence.Based in Portland, Oregon, Puppet is a privately heldcompany with more than 470 employees around the world.Learn more at http://puppet.com.4
GEEK GUIDE CONTAINERS 101Containers101SOL LEDERMANWhat Is a Container and How AreContainers Used?A starting point for an exploration of containers and howthey’re used is this simple definition: a container is apackaging format for a unit of software that ships together.A container is a format that encapsulates a set ofsoftware and its dependencies, the minimal set of runtimeresources the software needs to do its function. A containeris a form of virtualization that is similar to a virtual machine(VM) in some ways and different in others. VMs encapsulatefunctionality in the form of the application platform andits dependencies. The key difference between VMs andcontainers is that each VM has its own full-sized OS,while containers typically have a more minimal OS.5
GEEK GUIDE CONTAINERS 101Containers, because they don’t encapsulatean entire OS and its services, are an order ortwo of magnitude smaller than VMs.CIO’s article “What are containers and why do you needthem?” explains the motivation driving container hem.html):Containers are a solution to the problem of how to getsoftware to run reliably when moved from one computingenvironment to another. This could be from a developer’slaptop to a test environment, from a staging environmentinto production and perhaps from a physical machine in adata center to a virtual machine in a private or public cloud.Containers, because they don’t encapsulate an entire OSand its services, are an order or two of magnitude smallerthan VMs. Because they’re lightweight and have a minimalOS component, containers have some major advantages.They start up quickly and move easily from one platform toanother compatible one, and a number of containers canfit into the disk footprint of a single VM.What Are the Values of Containers?Containers are particularly useful in rapid developmentenvironments. The develop/deploy process cycles6
GEEK GUIDE CONTAINERS 101through these six steps:1. Develop or update an application.2. Deploy the application to the testbed.3. QA new code.4. Move code to the staging site.5. Verify operation in the staging environment.6. Move the new code to production.These cycles occur frequently and involve multipleparties (developers, QA staff, system administratorsand perhaps DevOps staff), and any bottlenecks inprovisioning the resources an application needs willdelay releases. Beyond the concerns of moving softwarethrough the life cycle quickly are the risks introducedby differences in the development, testbed, stagingand production environments. Of particular concern arebugs introduced into production caused by differencesin environments. The later in the life cycle bugs arecaught, the more expensive and time consuming theyare to correct.Containers are particularly valuable in rapiddevelopment cycles for three reasons. First, containersare much quicker to spin-up than servers or VMs.Second, smaller footprints make better utilization7
GEEK GUIDE CONTAINERS 101of server hardware than VMs. And third, becausecontainers enclose their running environment includingall of their dependencies, containers greatly minimizethe risks of inconsistencies in environments. Later in thisebook, I discuss the importance of carefully managingthe complexities of a container environment to speeddeployments while minimizing risks.Who Are the Container Providers?Table 1 shows the results of ClusterHQ’s “2016 ContainerMarket Adoption Survey” ner-usage-june-2016.pdf), which listsutilization percentages of the seven most popular containertechnologies according to survey respondents (note thatsome enterprises use more than one container technology,so the percent total is greater than 100%).Table 1. Utilization Percentages of the Seven Most PopularContainer TechnologiesDocker94%LXC15%rkt10%FreeBSD Jails5%Solaris Zones5%Other5%LXD4%8
GEEK GUIDE CONTAINERS 101Docker is by far the most widely adopted containertechnology. The ConvoxBlog explains why in itspost “Why Docker? The Image API is The reason to use Docker is for its modern packagingand runtime APIs. The Docker Image and Container APIshave become a de facto standard. Every major computingplatform—from OS X to AWS—now has native support forworking with Docker Images and Containers.Docker is also popular because it is tightly integratedwith the Linux kernel and runs on all major Linuxdistributions. Less adopted FreeBSD Jails and SolarisZones, in contrast, are built on less popular OSes. And,Docker has major community involvement through itscloud-based registry, testing and collaboration service,Docker Hub (https://hub.docker.com).Because the container space continues to see muchexperimentation and growth, there are a large numberof container technology providers, with a large matrix ofoverlapping features. No single document could providea fair comparison of the many offerings. However,W ikipedia has an article, “Operating-system-levelvirtualization”, with a table of 17 implementations(as of September 2016) showing the OS, license andfeature information for each vel virtualization). This table isan excellent jumping off point for comparing offeringsat a very high level.9
GEEK GUIDE CONTAINERS 101A bigger question than whether containers andVMs can coexist is how to network them together.Do Companies Need to Leavethe VM Structure Entirely, orCan There Be Hybrid Approaches?Using containers is not an all-or-nothing proposition forenterprises with a heavy investment in VMs. Plus, it doesn’tmake sense to update or refactor some applications to runin containers. The migration from VM-centric applications tocontainerized applications can be a planned and gradual one.A bigger question than whether containers and VMscan coexist is how to network them together. Enterprisesare faced with a mix of technologies, some running incontainers, some in VMs and some on bare metal. Toadd to the complexity, some applications are runningin public clouds, and others are running in privateclouds. The New Stack introduces the hybrid cloudand container virtual networking as the new normalin its article “How Overlay Networks Pave the Wayfor Seamless Hybrid Clouds” -cloud-easier-adopt):The question then becomes how to weave together all thecontainers running on and off premises and connect themto other (uncontained) services, without this turning intoa configuration hairball and security nightmare.10
GEEK GUIDE CONTAINERS 101Container overlay networks are essentially the de-factostandard now because they avoid the configuration hairball.The container virtual network rides on top of the underlyingIP networks, so it looks the same to the application regardlessof differences in the underlying network technology.A data center network doesn’t work the same way as,say, AWS Virtual Private Cloud. Essentially the problem ofmanaging the differences is pushed down into the containernetworking layer. That means there is no configuration orcode required in the application itself: it can just use regularnetworking constructs like TCP/IP and DNS.Planning a migration path will require major effort,but fortunately, the tools and practices to enable suchan effort are evolving rapidly.Why Are Some Firms Waitingto Use Containers?Enterprises commonly cite one or more of the followingsix reasons for not rushing to containers:1. Not all applications fit into the container model.Containers are particularly well suited to tasks thatare stateless or can be carved into small units offunctionality (that is, microservices) and that benefitfrom scaling specific tasks rather than the entireapplication. On the other hand, you will need toconsider how to deal with data persistence andstorage. There may be no reason to move monolithicapplications that run fine in VMs to containers.11
GEEK GUIDE CONTAINERS 1012. Security is a major concern because containers share acommon OS kernel. A virus that infects one containerplaces other containers that share that kernel atrisk. Although SELinux can help mitigate risk, manyenterprises are reluctant to deploy containers inproduction environments. VMs are less susceptibleto spreading vulnerabilities, because they are moreisolated from the hypervisor, and each runs its own OS.Additionally, the landscape of hardware virtualizationis much more stable than that of containers.3. The container ecosystem is evolving rapidly and maynot be ready for production in your organization.Developer-driven container usage can be at odds withoperational, security and compliance concerns. Whileit can be easier to build a container than a VM, youwill lose visibility into and control over what’s runningin your infrastructure. This is problematic if you needto remediate a wide-ranging vulnerability. Somecompanies are nervous about investing in managingthe complexity of containers if they believe they’llneed to train their staff in new tools and processesin the next year or two. Related concerns are how tohandle changing specifications, backward compatibilityand interoperability.4. Container management, configuration andorchestration is complex. Companies may find itdifficult to justify the expense and effort of trainingtheir IT personnel (or of hiring additional staff) to12
GEEK GUIDE CONTAINERS 101The challenges of provisioning large numbersof containers running different services, ofscaling sets of containers up or down, and ofmanaging the interaction among services canbe daunting without really solid and proventechnologies and workflows.migrate to containers. The concern is heightened ifmanagement has invested heavily in VM environments,especially if they have done so recently.5. Related to the complexity of managing containers isthe concern that management tools have not matured.In a particular deployment there may be hundreds ofcontainers, each running a single service. The challengesof provisioning large numbers of containers runningdifferent services, of scaling sets of containers up ordown, and of managing the interaction among servicescan be daunting without really solid and proventechnologies and workflows.6. Windows is the workstation platform of many developers,and Docker for Windows was just recently released.On June 8, 2016, Microsoft announced, “You cannow use Docker natively on Windows 10 with Hyper-VContainers, to build, ship and run containers utilizing13
GEEK GUIDE CONTAINERS 101the Windows Server 2016 Technical Preview 5 NanoServer container OS image” 4361). It will take time forthis offering to prove itself. And, requiring a VM torun on the Windows Server may hinder its adoption. Arelated nascent area is running Windows in a ization/windowscontainers/about/about overview).What’s Involved in Managing Containers?Managing containers requires leading-edge skillsand tools, beyond an understanding of the containerparadigm. Digital Ocean published an excellent fivepart tutorial series that introduces containers, servicediscovery, distributed configuration, networking,communication, scheduling and to-commoncomponents). Although the tutorial is focused onDocker, it provides a good overview of containertechnology for users of any container platform.Here is a brief summary of the major componentsthe tutorial introduces:1. Service discovery and distributed configurationstores: service discovery helps containers to scalewithout human intervention. Discovery assists ininteraction with other containers by finding availableservices that your container provides. Distributed14
GEEK GUIDE CONTAINERS 101configuration storage makes possible the dynamicscaling and configuration of containers withoutrequiring the containers themselves to be dependenton some static configuration.2. Networking: containers need to communicatewith one another and with their host servers orVMs. With the number of containers scaling up anddown, and with the potential for a large numberof containers in an application, robust networkingservice is paramount. Secure communicationbetween application components is another concern.Additionally, the networking service must handlesubnetting, gateways, MAC addresses and other tasks.And, of course, the networking tools need to provideall of those services in a dynamic environment.3. Scheduling: the scheduler needs to be able todetermine an appropriate host for an applicationcomponent and start a container on it. The schedulingservice relies on information in the distributedconfiguration stores in making its decisions. Theservice needs to handle potential constraints aboutwhether to run multiple containers on a particularhost, whether to run more than one container on agiven host, whether to start the container on the leastbusy host and any other constraints the administratorplaces on the application.4. Cluster management: cluster management is closely15
GEEK GUIDE CONTAINERS 101related to scheduling. A particular unit of workmay consist of containers, hosts, services and theirinteractions. It may be desirable to abstract awaythe management of that workload. Clusters are theabstraction for the resources and the interactionsthat are required to perform that work. Clustermanagement tools can operate on those abstractions.5. Orchestration: orchestration is a broad term that isoften used interchangeably with the terms schedulingand cluster management. Orchestration also involvesprovisioning, which is the process of creating andconfiguring a container and starting it so that it mayperform work.An additional major aspect of working with containersis configuration management. This involves making surethat the right versions of OS level as well as applicationsoftware and libraries are installed and managed (forexample, for upgrades). I dedicate a later section of thisguide to introducing container configuration managementand some of the complexities that a good set of tools canhelp manage.Who Are Some of the Major Players inthe Container Runtime Space?If you want to run containers, you have to run them ona single machine, VM or computer cluster. Note that youcan run containers with just Docker on a single machine.Swarm and the rest are clustered solutions.16
GEEK GUIDE CONTAINERS 101The three offerings shown in Table 2 are a goodstarting place for research.Table 2. Major Players in the Container Runtime SpaceOfferingWebsiteDescriptionfrom WebsiteDocker er Swarm isnative clustering forDocker. It turns apool of Docker hostsinto a single, virtualDocker s buildsupon 15 years ofexperience of runningproduction workloadsat Google, combinedwith best-of-breedideas and practicesfrom the community.”Mesoshttp://mesos.apache.org“Native support forlaunching containerswith Docker andAppC images.”17
GEEK GUIDE CONTAINERS 101Benefits Gained by Switching toContainers—Case StudiesThe Docker website includes several dozen case studiesof benefits enterprises gained when they adoptedcontainers (https://www.docker.com/customers).Beyond Docker’s own press, the following is asampling of the positive experiences of enterprisesmigrating to containers:nUber: “While the transition was painful, the endresult was what they had hoped for, getting rid oftheir three greatest pain points that stifled continuousdeployment. With Docker, they no longer had to waitfor the infrastructure team to write service scaffolding,wait for IT to locate services or wait for infrastructureteam to provision services” bers-deployments).n eBay: “The adoption of Kubernetes at eBay is not justabout moving to containers to deploy applications,but changing the application lifecycle at the company,which is centered around the infrastructure cloudlayer, with provisioning, deploying, monitoring,and remediating issues being the key functions fordevelopers and system administrators to p-openstack).n Yelp: “[Docker] provides the developers with more of18
GEEK GUIDE CONTAINERS 101the ability to do more of the management of the systemsthemselves”, Sam Eaton, Yelp Director of Operationssays. “It makes it easier for them to manage and beresponsible for all their own services, without having toask for operational help” olith-behind).n ADP: “[ADP] was an early tester of the Docker Datacenterstack and now has it running across 762 server nodeson top of its OpenStack cloud and will also be using thestack to provide a compatibility layer running on theAWS cloud” ckles-down-from-hyperscale-to-enterprise).n Goldman Sachs: “When our engineers discovered andstarted using Docker’s open source platform, they wereimmediately impressed by the portability it providesapplications”, Duet (a 27-year veteran of the firm) said. “Itinspired us to move towards a standardized infrastructurefor packaging, shipping and running our applications basedon Docker’s technology” sts-95-million-in-docker.html).How Does Configuration ManagementApply to Containers, and How DoesPuppet Accelerate the Adoption ofContainer Technologies?Configuration management tools provide automationto handle the complexities of deploying, managingand upgrading infrastructure software. Sophisticated19
GEEK GUIDE CONTAINERS 101By managing an enterprise-wide configurationfrom a central place, the administrator canquickly update applications, system patches,libraries and other resources.configuration managers allow IT staff to define thedesired state of infrastructure and applications while themanager enforces that state. By managing an enterprisewide configuration from a central place, the administratorcan quickly update applications, system patches, librariesand other resources. The configuration manager handlesdetermining which hosts get which updates, relievingthe administrator of the onerous and error-prone task ofmanaging the details. Configuration managers also enforceconsistency across environments, paving the way for rapidand continuous software deployment.Although some dismiss the importance of configurationmanagement tools in a container environment, claiming thatcontainer management systems often provide configurationmanagement capabilities and making separate toolsredundant, they miss another view. Luke Kanies, Founder andCEO of Puppet, one of the leading providers of configurationmanagement tools, articulates a different eek-foothold-in-containers):[While] virtualization made each individual machine20
GEEK GUIDE CONTAINERS 101less necessary and eliminated many of the difficultproblems involving managing physical machines, it alsoincreased the number of machines under managementabout tenfold, Kanies said. Meanwhile, Docker is goingto make everybody’s infrastructure at least another 10times bigger. Some people argue IT will have as much as100 times as many containers as it has VMs to manage—and potentially even more.So, every application you have just got more complex,more critical, more confusing and more complicated. Youneed way more management, not way less management.This excerpt from the Puppet web page introducing itsDocker integration lists a number of the many moving partsto manage in a container environment ker):Puppet Application Orchestration allows you to modelthe relationships between application services—forexample, databases, API servers, and message queues.Relationships can be modeled between any mix ofcontainers, microservices, persistent infrastructure,monoliths, devices, or whatever else makes up yourapplication’s architecture. With a model to reference,it’s easier to understand what to re-architect andwhere you need to re-architect to incorporatecontainers and microservices.Gareth Rushgrove, senior software engineer at Puppet,21
GEEK GUIDE CONTAINERS 101makes the case for an even greater need for configurationmanagement of containers even when there is supportin powerful management tools, such as Google’sKubernetes container manager guration-puppet):Some people look at configuration management (andtools like Puppet) as a way of managing host-boundresources like files, services, packages, users or groups.Kubernetes introduces higher-level primitives, likePods and Replication Controllers, aimed at makingthe management of distributed and scalable systemsdrastically easier. The story goes that you no longer needconfiguration management with those new primitives.The problems associated with those capabilities arepresent with systems like Kubernetes too, and onlypartially addressed by current native tooling—problemslike managing configuration drift, having a singlewell-audited change control mechanism, having amodel of your infrastructure outside Kubernetes, etc.This becomes even more important as deployments hitproduction, as well as in heterogeneous (read realworld) environments, where multiple generations oftechnology run side by side.Beyond the complexity of managing the relationshipbetween application services is the complexity ofworking with leading-edge technologies like Docker,22
GEEK GUIDE CONTAINERS 101Additionally, Puppet provides tools, in theform of Docker images, that allow Puppet torun on hosts that run Linux containers and ontop of container managers.Kubernetes and Mesos. Puppet has launched ProjectBlueshift eshift) as the vehicle to facilitate engagement withPuppet’s user and technology provider communities.Docker support includes Puppet’s Docker module to install,configure and manage Docker plus its host and the servicesrunning on that host (https://forge.puppet.com/puppetlabs/docker platform/readme). Additionally,Puppet provides tools, in the form of Docker images, thatallow Puppet to run on hosts that run Linux containersand on top of container managers. These tools alsofacilitate the creation of a local Puppet developmentenvironment et-container-centric-infrastructure). And,when Docker announced general availability of DockerUniversal Control Plane, a tool to deploy and managedockerized applications, Puppet immediately announcedits corresponding docker ucp module control-plane-puppet).Puppet can create resources in Google’s Kubernetescontainer manager via its Kubernetes /readme).23
GEEK GUIDE CONTAINERS 101Per Puppet’s announcement, the module:.allows you to use the Puppet domain specific languageto manage resources in Kubernetes—for instanceReplication Controllers, Services and Pods. This means: (1)It is easier to manage the state of Kubernetes resourcesover time, using source code that can be versionedalongside your application code. (2) You can be sure ofthe state of your Kubernetes infrastructure by takingadvantage of Puppet’s built-in reporting and toolslike PuppetDB. (3) If you’re already using Puppet, theKubernetes Puppet module provides a convenient way ofmanaging Kubernetes alongside your other infrastructure.And, Puppet users can create higher levelabstractions for Kubernetes tions-for-kubernetes-puppet).The Puppet community has developed modules forinstalling and managing Apache Mesos, open-sourcesoftware that allows enterprises to abstract away systemresources in order to build application-centric elasticdistributed systems. Installing Mesos on a compute cluster,installing several of the most popular Mesos frameworks,and using Mesos with Puppet are introduced in the “UsingPuppet with Mesos” article (https://puppet.com/blog/using-puppet-mesos). This article also includes a numberof links to modules and examples.Project Blueshift also includes modules to work with theopen-source CoreOS Linux distribution, which includesthe rkt container engine, and the project also distributes24
GEEK GUIDE CONTAINERS 101modules to work with Consul, the open-source toolfor discovering services on networks. As new containertechnologies are developed and prove themselves,Project Blueshift will develop modules, relationships withproviders and community support to accelerate theirintegration into the enterprise.ConclusionI’d like to close this guide with a second quote fromGareth Rushgrove, senior software engineer at Puppet.This statement focuses, at the surface, on Mesos andPuppet. But, it speaks to the heart of Puppet’s priorityof fostering community. One could argue that Docker iswildly successful because it combines great technologywith a strong emphasis in community. Puppet is followingthis same recipe for success:This collection of modules for managing Mesos withPuppet is another great example of the ingenuity ofthe Puppet community. Thank you to everyone who hascontributed to these modules and to making Puppeta useful tool for managing Mesos. All of this is alsoa g
Puppet Puppet is driving the movement to a world of unconstrained software change. Its revolutionary platform is the industry standard for automating the delivery and operation of the software that powers everything around us. More than 33,000 companies—including more than 75 percent of the Fortune 100
