WIXLIB

Microservices / eMag Issue 16 - August 2014The application consists of several components,including the StoreFront UI that implements theuser interface and services for managing the productcatalog, processing orders, and managing thecustomer’s account. These services share a domainmodel consisting of entities such as Product, Order,and Customer.three-dimensional scalability model: the scale cube,which is shown in Figure 2.Despite having a logically modular design, theapplication is deployed as a monolith. For example,if you were using Java then the application wouldconsist of a single WAR file running on a Webcontainer such as Tomcat. The Rails version ofthe application would consist of a single directoryhierarchy deployed using, for example, PhusionPassenger on Apache/Nginx or JRuby on Tomcat.Figure 2 - The scale cube.This so-called monolithic architecture has a numberof benefits. Monolithic applications are simple todevelop since IDEs and other development tools areoriented around developing a single application. Theyare easy to test since you just need to launch the oneapplication. Monolithic applications are also simpleto deploy since you just have to copy the deploymentunit – a file or directory – to a machine running theappropriate kind of server.This approach works well for relatively smallapplications. However, the monolithic architecturebecomes unwieldy for complex applications. A largemonolithic application can be difficult for developersto understand and maintain. It is also an obstacleto frequent deployments. To deploy changes to oneapplication component, you have to build and deploythe entire monolith, which can be complex, requirethe coordination of many developers, and result inlong test cycles.A monolithic architecture also makes it difficult totest and adopt new technologies. It’s difficult, forexample, to try out a new infrastructure frameworkwithout rewriting the entire application, which isrisky and impractical. Consequently, you are oftenstuck with the technology choices that you madeat the start of the project. In other words, themonolithic architecture doesn’t scale to supportlarge, long-lived applications.When using Z-axis scaling, each server runs anidentical copy of the code. In this respect, it’s similarto X-axis scaling. The big difference is that eachserver is responsible for only a subset of the data.Some component of the system is responsible forrouting each request to the appropriate server. Onecommonly used routing criterion is an attribute ofthe request such as the primary key of the entitybeing accessed, i.e. sharding. Another commonrouting criterion is the customer type. For example,an application might provide paying customers witha higher SLA than free customers by routing theirrequests to a different set of servers with morecapacity.Z-axis scaling, like X-axis scaling, improves theapplication’s capacity and availability. However,neither approach solves the problems of increasingdevelopment and application complexity. To solvethose problems, we need to apply Y-axis scaling.Decomposing applications intoservicesFortunately, there are other architectural styles thatdo scale. The Art of Scalability describes a useful,CONTENTSIn this model, the commonly used approach of scalingan application by running multiple identical copiesof the application behind a load balancer is knownas X-axis scaling. That’s a great way to improve thecapacity and the availability of an application.The third dimension to scaling is Y-axis scaling orfunctional decomposition. Whereas Z-axis scalingsplits things that are similar, Y-axis scaling splitsthings that are different. At the application tier,Y-axis scaling splits a monolithic application into a setof services. Each service implements a set of relatedfunctionality such as order management, customermanagement, etc.Page 5

Microservices / eMag Issue 16 - August 2014Deciding how to partition a system into a set ofservices is very much an art but there are number ofstrategies that can help. One approach is to partitionservices by verb or use case. For example, later onwe will see that the partitioned online store has aCheckout UI service, which implements the UI for thecheckout use case.Another partitioning approach is to partition thesystem by nouns or resources. This kind of serviceis responsible for all operations that operate onentities/resources of a given type. Later, we willsee how it makes sense for the online store to havea Catalog service, which manages the catalog ofproducts.Ideally, each service should have only a small set ofresponsibilities. (Uncle) Bob Martin has a PDF aboutdesigning classes using the single-responsibilityprinciple (SRP). The SRP defines a responsibility ofclass as a reason to change, and that a class shouldonly have one reason to change. It make sense toapply the SRP to service design as well.Another analogy that helps with service design isthe design of Unix utilities. Unix provides a largenumber of utilities such as grep, cat, and find. Eachutility does exactly one thing, often exceptionallywell, and can be combined with other utilities using ashell script to perform complex tasks. It makes senseto model services on Unix utilities and create singlefunction services.It’s important to note that the goal of decompositionis not to have tiny (e.g. 10 to 100 LOC as some argue)services simply for the sake of it. Instead, the goalis to address the problems and limitations of themonolithic architecture. Some services could verywell be tiny but others will be substantially larger.Figure 3 - The microservice architecture.the application’s main logical components into astandalone service. Let’s look at the consequences ofdoing that.Benefits and drawbacks of amicroservice architectureThis architecture has a number of benefits. First,each microservice is relatively small. The code iseasier for a developer to understand. The small codebase doesn’t slow down the IDE, making developersmore productive. Also, each service typically startsa lot faster than a large monolith, which againmakes developers more productive and speeds updeployments.If we apply Y-axis decomposition to the exampleapplication, we get the architecture shown in Figure3.Second, each service can be deployed independentlyof other services. If the developers responsible for aservice need to deploy a change that’s local to thatservice, they do not need to coordinate with otherdevelopers. They can simply deploy their changes.A microservice architecture makes continuousdeployment feasible.The decomposed application consists of variousfront-end services that implement different parts ofthe user interface and multiple back-end services.The front-end services include the Catalog UI, whichimplements product search and browsing, andCheckout UI, which implements the shopping cartand the checkout process. The back-end servicesinclude the same logical services that were describedat the start of this article. We have turned each ofThird, each service can be scaled independentlyof other services using X-axis cloning and Z-axispartitioning. Moreover, each service can be deployedon hardware that is best suited to its resourcerequirements. This is quite different than when usinga monolithic architecture where components withwildly different resource requirements – e.g. CPUintensive vs. memory-intensive – must be deployedtogether.CONTENTSPage 6

Microservices / eMag Issue 16 - August 2014The microservice architecture makes it easierto scale development. You can organize thedevelopment effort around multiple, small (e.g. twopizza) teams. Each team is solely responsible for thedevelopment and deployment of a single service or acollection of related services. Each team can develop,deploy, and scale their service independently of allother teams.The microservice architecture also improves faultisolation. For example, a memory leak in one serviceonly affects that service. Other services will continueto handle requests normally. In comparison, onemisbehaving component of a monolithic architecturewill bring down the entire system.Last but not least, the microservice architectureeliminates any long-term commitment to atechnology stack. In principle, when developing anew service, the developers are free to pick whateverlanguage and frameworks are best suited for thatservice. Of course, in many organizations it makessense to restrict the choices but the key point is thatyou aren’t constrained by past decisions.Moreover, because the services are small, it becomespractical to rewrite them using better languagesand technologies. It also means that if the trial ofa new technology fails, you can throw away thatwork without risking the entire project. This is quitedifferent than when using a monolithic architecture,where your initial technology choices severelyconstrain your ability to use different languages andframeworks in the future.DrawbacksOf course, no technology is a silver bullet, andthe microservice architecture has a number ofsignificant drawbacks and issues. First, developersmust deal with the additional complexity of creatinga distributed system. Developers must implementan inter-process communication mechanism.Implementing use cases that span multiple serviceswithout using distributed transactions is difficult.IDEs and other development tools focus on buildingmonolithic applications and don’t provide explicitsupport for developing distributed applications.Writing automated tests that involve multipleservices is challenging. These are all issues that youdon’t have to deal with in a monolithic architecture.many more moving parts – multiple instances ofdifferent types of service – that must be managed inproduction. To do this successful you need a highlevel of automation, either homegrown code, a PaaSlike technology such as Netflix Asgard and relatedcomponents, or an off-the-shelf PaaS such as PivotalCloud Foundry.Also, deploying features that span multiple servicesrequires careful coordination between the variousdevelopment teams. You have to create a rolloutplan that orders service deployments based onthe dependencies between services. That’s quitedifferent than when using a monolithic architecturewhere you can easily deploy updates to multiplecomponents atomically.Another challenge with using the microservicearchitecture is deciding at what point during thelifecycle of the application you should use thisarchitecture. When developing the first version ofan application, you often do not have the problemsthat this architecture solves. Moreover, using anelaborate, distributed architecture will slow downdevelopment.This can be a major dilemma for startups whosebiggest challenge is often how to rapidly evolve thebusiness model and accompanying application. UsingY-axis splits might make it much more difficult toiterate rapidly. Later on, however, when the challengeis how to scale and you need to use functionaldecomposition, tangled dependencies might make itdifficult to decompose your monolithic applicationinto a set of services.Because of these issues, the adoption of amicroservice architecture should not be undertakenlightly. However, for applications that need toscale, such as consumer-facing Web applicationsor SaaS applications, it is usually the right choice.Well-known sites such as eBay (PDF), Amazon.com, Groupon, and Gilt have all evolved froma monolithic architecture to a microservicearchitecture.Now that we have looked at the benefits anddrawbacks, let’s look at a couple of key design issueswithin a microservice architecture, beginning withcommunication mechanisms within the applicationand between the application and its clients.The microservice architecture also introducessignificant operational complexity. There areCONTENTSPage 7

Microservices / eMag Issue 16 - August 2014Communication mechanisms in amicroservice architecturelatency mobile network, would be inefficient andresult in a poor user experience.In a microservice architecture, the patterns ofcommunication between clients and the application,as well as between application components, aredifferent than in a monolithic application. Let’s firstlook at the issue of how the application’s clientsinteract with the microservices. After that we willlook at communication mechanisms within theapplication.A much better approach is for clients to make a smallnumber of requests per page, perhaps as few as one,over the Internet to a front-end server known as anAPI gateway, which is shown in Figure 5.API-gateway patternIn a monolithic architecture, clients of theapplication, such as Web browsers and nativeapplications, make HTTP requests via a load balancerto one of N identical instances of the application. Butin a microservice architecture, the monolith has beenreplaced by a collection of services. Consequently,a key question we need to answer is what do theclients interact with?Figure 5 - API gateway.An application client, such as a native mobileapplication, could make RESTful HTTP requests tothe individual services as shown in Figure 4.The API gateway sits between the application’sclients and the microservices. It provides APIsthat are tailored to the client. The API gatewayprovides a coarse-grained API to mobile clientsand a finer-grained API to desktop clients that usea high-performance network. In this example, thedesktop clients makes multiple requests to retrieveinformation about a product, whereas a mobile clientmakes a single request.Figure 4 - Calling services directly.The API gateway handles incoming requests bymaking requests to some number of microservicesover the high-performance LAN. Netflix, forexample, describes how each request fans out toon average six back-end services. In this example,fine-grained requests from a desktop client aresimply proxied to the corresponding service, whereaseach coarse-grained request from a mobile client ishandled by aggregating the results of calling multipleservices.On the surface, this might seem attractive. However,there is likely to be a significant mismatch ingranularity between the APIs of the individualservices and data required by the clients. Forexample, displaying one webpage could potentiallyrequire calls to large numbers of services. Amazon.com, for example, describes how some pagesrequire calls to more than 100 services. Making thatmany requests, even over a high-speed Internetconnection, let alone a lower-bandwidth, higher-Not only does the API gateway optimizecommunication between clients and the application,but it also encapsulates the details of themicroservices. This enables the microservices toevolve without impacting the clients. For example,two microservices might be merged. Anothermicroservice might be partitioned into two ormore services. Only the API gateway needs to beupdated to reflect these changes. The clients remainunaffected.CONTENTSPage 8

Microservices / eMag Issue 16 - August 2014Now that we have looked at how the API gatewaymediates between the application and its clients, let’slook at how to implement communication betweenmicroservices.Inter-service communicationmechanismsAnother major difference with the microservicearchitecture is how the different components ofthe application interact. In a monolithic application,components call one another via regular methodcalls. But in a microservice architecture, differentservices run in different processes. Consequently,services must use an inter-process communication(IPC) to communicate.There are pros and cons to both approaches.Applications are likely to use a mixture of the two.For example, in the next section, which discusses howto solve data management problems that arise in apartitioned architecture, you will see how both HTTPand messaging are used.Decentralized data managementSynchronous HTTPThere are two main approaches to inter-processcommunication in a microservice architecture. Oneoption is a synchronous HTTP-based mechanismsuch as REST or SOAP. This is a simple and familiarIPC mechanism. It’s firewall-friendly so it worksacross the Internet, and implementing the requestreply style of communication is easy. The downsideof HTTP is that it doesn’t support other patterns ofcommunication such as publish-subscribe.Another limitation is that both the client and theserver must be simultaneously available, which isnot always the case since distributed systems areprone to partial failures. Also, an HTTP client needsto know the host and the port of the server. Whilethis sounds simple, it’s not entirely straightforward,especially in a cloud deployment that uses autoscaling in which service instances are ephemeral.Applications need to use a service-discoverymechanism. Some applications use a service registrysuch as Apache ZooKeeper or Netflix Eureka. Inother applications, services must register with a loadbalancer, such as an internal ELB in an Amazon VPC.Asynchronous messagingAn alternative to synchronous HTTP is anasynchronous message-based mechanism such as anAMQP-based message broker. This approach has anumber of benefits. It decouples message producersfrom message consumers. The message broker willbuffer messages until the consumer is able to processthem. Producers are completely unaware of theconsumers. The producer simply talks to the messagebroker and does not need to use a service-discoverymechanism. Message-based communication alsosupports a variety of communication patternsCONTENTSincluding one-way requests and publish-subscribe.One downside of using messaging is a need fora message broker, which is yet another movingpart that adds to the complexity of the system.Another downside is that request-reply style ofcommunication is not a natural fit.A consequence of decomposing the application intoservices is that the database is also partitioned.To ensure loose coupling, each service has its owndatabase (schema). Moreover, different servicesmight use different types of database – a so-calledpolyglot-persistence architecture. For example,a service that needs ACID transactions mightuse a relational database, whereas a service thatmanipulates a social network might use a graphdatabase. Partitioning the database is essential, butwe now have a new problem to solve: how to handlethose requests that access data owned by multipleservices. Let’s first look at how to handle readrequests and then look at update requests.Handling readsConsider an online store where each customer has acredit limit. When a customer attempts to place anorder, the system must verify that the sum of all openorders would not exceed the customer’s credit limit.It would be trivial to implement this business rule ina monolithic application but it’s much more difficultto implement this check in a system that managescustomers by the CustomerService and orders bythe OrderService. Somehow, the OrderServicemust access the credit limit maintained by theCustomerService.One solution is for the OrderService to retrievethe credit limit by making an RPC call to theCustomerService. This approach is simple toimplement and ensures that the OrderServicealways has the most current credit limit. Thedownside is that it reduces availability because theCustomerService must be running in order to placean order. It also increases response time because ofthe extra RPC call.Page 9

Microservices / eMag Issue 16 - August 2014Another approach is for the OrderService to storea copy of the credit limit. This eliminates the needto make a request to the CustomerService and soimproves availability and reduces response time.It does mean, however, that we must implementa mechanism to update the OrderService’s copyof the credit limit whenever it changes in theCustomerService.A major benefit of this approach is that it decouplesproducers and consumers of the events. Not onlydoes this simplify development but compared todistributed transactions it improves availability. Ifa consumer isn’t available to process an event thenthe message broker will queue the event until it can.A major drawback of this approach is that it tradesconsistency for availability. The application has tobe written in a way that can tolerate eventuallyconsistent data. Developers might also need toimplement compensating transactions to performlogical rollbacks. Despite these drawbacks, however,this is the preferred approach for many applications.Handling update requestsThe problem of keeping the credit limit up to datein OrderService is an example of the more generalproblem of handling requests that update dataowned by multiple services.Refactoring a monolithDistributed transactionsOne solution, of course, is to use distributedtransactions. For example, when updating acustomer’s credit limit, the CustomerService coulduse a distributed transaction to update both its creditlimit and the corresponding credit limit maintainedby the OrderService. Using distributed transactionswould ensure that the data is always consistent. Thedownside of using them is that it reduces systemavailability since all participants must be availablein order for the transaction to commit. Moreover,distributed transactions have fallen out of favor andare g

microservices emag issue 16 - august 2014 facilitating the spread of knowledge and innovation in enterprise software development microservices and soa p. 12 adrian cockcroft on microservices and devops p. 15 microservices? what about nanoservices? p. 19 building products at soundcloud p. 21 the strengths and weaknesses of microservices p. 26