Migration Of Monolithic Applications To Microservices

Transcription

Migration of Monolithic Applications toMicroservicesPoint of View

What is Microservice?DefinitionA microservices architecture consists of a collection of small, autonomous services. Each service is self-containedand should implement a single business capability.CharacteristicsPrinciples Many smaller (fine grained), clearlyScalabilityscoped services Single Responsibility Principle Domain Driven Development Bounded Context Independently ManagedResiliency Clear ownership for each service Typically need/adopt the“DevOps” modelDecentralized GovernanceAuto - ProvisioningAvailabilityIndependent & AutonomousFailure IsolationContinuous Deliverythrough DevOps2

Pros & Cons of Monolithic AppsProsCons Simplicity Simple to build, test and deploy. Scale horizontally, in one direction by runningseveral copies of the application behind a loadbalancer Performance Monolithic applications have performanceadvantages due to shared-memory access whichis faster than inter-process communication (IPC) Cross-cuttingConcerns Apps typically have a large number of concerns,such as logging, rate limiting, and securityfeatures such audit trails and DOS protection In monolithic everything is running through thesame app, it’s easy to hook up components tothose cross-cutting concerns An error in any of the modules in theapplication can bring the entire applicationdownReliabilityMonolithic app services tend to get tightlycoupled and entangled as the applicationevolves, making it difficult to isolate servicesfor purposes such as independent scaling orcode maintainabilityTight CouplingArchitecture is harder to understand,because there may be dependencies whichare not obvious when we’re looking at aparticular service or controllerUnderstandabilityDue to a single large codebase and tightcoupling, the entire application would haveto deploy for each updateUpdates3

When to use Microservices?Maintainability Application CodeMaintainability Constant changes toCoordination Coordination acrossdifferent teams Applicability ofbusiness logic &frequent break-fixes indemandcoordination with Unmanageable & timeconsuming complexcode changesdifferent teams Bigger the application,Plugging & Unplugging Business need ofLife Cycle Management Longer lifecyclecouple or decouple amanagement oflogicmonolithic application Lot of overhead as the Every functional pieceentire application willof the application willhave to be unit testedhave to be tested ,& debuggedirrespective ofthe larger the risk ofwhether they havecoordinationbeen modified or notDeprecated Features &Attributes Feature & attributeswithin code becomedeprecated Code refactoringbecome tedious task ifthe size of theapplication isconsiderable bigger4

Challenges solved by MicroservicesMicroservices can quickly adapt to marketconditions like cloud transformation andcontainerization.08The distributed architecture ofmicroservices enables increasedperformance and scalability of application.Integration Facades: Enables integrationwith multiple tools like API, MessageQueue, Service Bus without modifyingentire application01Removes dependencies betweenteams and enable faster code toproduction0702060305Organizational alignment: Reduces developerslearning curve and encourages teams to buildmore complex products and features iteratively04Highly modular approach of microservicesensure better maintenance, troubleshootingby identifying issue directly in the modules.Improves Developers Efficiency by reducingdevelopment time with small code bases andprovides a reusability benefits over time.Independent deployable components whichallows new features into production withoutany dependencies.5

Tradeoffs by MicroservicesPerformanceEventual ConsistencyMicroservices decreases the performance ofapplication as the message flow & communicationincreases when number of Microservices increasesAsynchronously2Asynchronous calls when it comes tomicroservices are a big performance gain butcomes at a price. Asynchronous programming ishard and even harder to debugResiliency413Microservices require writing Circuit breaker andRetry communication patterns to achieveresiliency. Thus increases overall complexity andmaintenance of code to avoid applicationdisruptionEventual consistency typically yields better performance,easier operation, and better scalability while requiringprogrammers to understand a more complicated data model.However, eventual consistency pattern has issues. Let’s say aREST API data is updated in your application which uses EntityFramework. Further the get operation in MVC’s view issupposed to reflect the changes. Microservices requiremultiple resources to update and relies on distributedtransactions.5Operational CostThere is higher cost while running each of the microservices ineach container as it will utilize more CPU and memory. Alsoreliable connectivity is required for communication betweenservices interacting with each other. Cost of monitoringmicroservices in production environment is higher6

Monolithic to Microservices - PlanningEstablish an application architecturebaseline Understanding of monolithic applicationswith business functionalityUnderstanding the importance of a moduleand what business purpose it solves.To have a wholistic approach and not a finegrained one.Domain Driven Design Self Contained Identify self contained modules as part ofthe overall planning.Modules are loosely integrated inmonolithic applications and should beeasier to transform.Self-contained systems contain bothbackend and UI.Assessing the size of the namespaceIf any single component in the architecturehas more 10% of functionality, it’s suggestedto break it down into smaller chunks.Monolithic application should be translated toservices with namespaces organized bydomains and sub-domains.Avoid Disruption Split front-end and back-end Best Practices A typical flaw with monolithic services isabout splitting the front-end with the backend and the business layer.Plan Microservices architecture by keepingthese three layers into consideration.Follow the MVC model for breaking up theservice.To shield users from disruption and ensurethat no important business application goesoffline during the processDecouple the presentation logic into a remoteUI with API access.Install proxies between API consumers andservices and Set up facades.Release new code with toggles and darklaunches rather than cutovers. Identify the most independent module andmigrate that first.Never allow the migrated Microservicesmodule code to continue in monolithicapplication.Do not hesitate to break the existingMicroservice to further smaller MicroservicesCreate a parallel design if apps are businesscritical7

Cost Consideration Personnel: Noteveryone is familiarwith Microservices.What is the cost ofbringing thosepersonnel up to datewith Microservices?OrganizationalExpenses: Benefits arebest realized whenadministered by smallcross functional teams.Tools: Containerizationand other supportingtechnologies which canbring down the cost A microservicesarchitecture, with fewerapplicationdependencies andsimple APIs, willimmediately reduce thetime and money spenton applicationmaintenance. Microservices canincrease the speed ofdevelopment, ensuringthat the organizationremains competitive. Thishowever depends on howwell the application ismodularized and thoughtthrough. Savings on applicationmaintenance haveproven to be more costeffective within a fewyears of adoption. A well architectedmicroservices architecturewill have fasterturnaround time for anychanges that businesswants depending of themarket requirementsSoftware that createsmicroservice directlyfrom the legacy or onpremise system, we canbypass complexESB/SOA layers thatwere previously used forlegacy integrations.When decommissionESB/SOA and replace itwith independent,directly connectedmicroservice-based APIs,we will always achieveexponential savings bycomparison DevOps, Agile, andother moderndevelopment practicesrely heavily onautomated testing. Microservices make fora much cleaner testingprocess. They’re builtsimpler, so it’s easier toreview their code.8

High Level Architecture ConsiderationsSimple DecoupleCapability Start with capabilities thatare decoupled from themonolith, that don’trequire changes to manyclient facing applicationsthat are currently usingthe monolith and possiblydon’t need a data store. Decoupling edge servicesgives us a good idea aboutoperational prerequisitesthat is needed for theother monolithic servicesthat need decoupling.Minimize Dependencyback to Monolithic A major benefit ofmicroservices is to havea fast and independentrelease cycle. There will be scenarioswhere dependencieswith monolithicapplications cannot beavoided. It's suggested thatcreate an interface withwhich Microserviceinteracts and has theabstraction from themonolith to reduce thedependency. Split StickyCapabilities EarlyDecouple Vertically andRelease the Data EarlyDecouple Capability andnot CodeOne of the preferredbest practices is toidentify the stickinessin your monolithicapplication and extractthose sticky businesslogic at the very earlystages. The general concept ofa monolithic applicationis with having tightlycoupled layers. There are alsodependencies on otherexternal systems. If thedata layer schemachanges releasing thosedata dependencies sothat those entities canlive on their ownbecomes a significantfactor of how well yourmicroservices arearchitected and moreimportantlyUnderstanding aparticular modulewithin the monolithicservice anddeconstructing itbased on capabilityand not trying to reengineer the code is apreferred andacceptable mode. Translating thecapability intomodern API by notlooking at how thecode was built helpsto stay within the bestpractice's realm.Ensuring that thosesticky capabilities areconverted into domaindriven design andconverting them intoproducingmicroservices is a goodpractice9

Architecture Patterns for TransitionPattern TypePattern NamePattern OutlineDomain-driven DesignDecomposition of business domain into sub-domains and identification of bounded contexts (groupsof functional units capable of autonomous existence)StranglerCoexistence of legacy / monolith components along with decomposed microservices. This iscommonly used for incremental modernization / transition needsAPI GatewayAn API Gateway serves as a single entry-point to numerous microservices offering abstraction,aggregation, protocol independence and securityAggregationAggregation of results from multiple microservices eliminating need for interaction with multiplegranular servicesBack-end for Front-endLayered microservices implementation with a layer dedicated for specific front-end needs (consumersinteract only with this layer). Example – microservices for mobile , desktop etc.Database per ServiceDedicated database for each service not accessed by another serviceShared DatabaseDatabase shared by 2-3 microservices; used mostly in modernization implementationsCommand Query ResponsibilitySegregation (CQRS)CQRS splits applications into Command (Create, Update, and Delete requests) side and Query sideSagaUse of Event Driven architectures to coordinate multiple microservices (each having its own database)Isolation /encapsulationSidecarDeploy helper components (monitoring, logging, configuration etc.) of an application into a separateprocess or container to provide isolation and encapsulationOthersCircuit BreakerInvocation of service calls by dependent services via a proxy which forces a timeout after a regulatednumber of attempts in case of an error in the providing serviceDecompositionIntegrationDatabase10

Microservices – Azure Service FabricNeed for Containerized Microservices DeploymentContainerized microservices deployment offers numerous benefits in terms of agility, scalability, rapid capabilities addition etc. Aneffective container orchestrator is needed for maximizing containerization benefits in Azure (and other cloud environments)How Azure Service Fabric fits ture/service-fabric/modernizeAzure ServiceFabric is a Container Orchestration platform offering auto-scaling, auto-fault-healing, rolling upgrades etc. It allowsfor developers to focus on businessapp-azure-service-fabriccapability creation and takes care of infrastructure and resource management needs itself.Key Offerings Support for multiple container typesApplication Health MonitoringAuto scale-in / out of microservicesRolling upgradesServices DiscoverySecurity11

Microservices – The “Serverless”Serverless microservices implementation is gaining traction due to increasing maturity of Azure Serverless offeringsA summary of key Azure serverless microservices offerings:OfferingAzure FunctionsOutlineAzure Functions can be purposed as serverlessmicroservices. They can be exposed as HTTPbased API end-points and allow for event-driveninvocationServerless good for:1.Greenfield Development2.Relatively small logic chunks (services)3.Operations that do not require longduration executionAzure Logic AppsAzure Container Instances (ACI)Logic Apps can be leveraged to effectivelyorchestrate Azure Function implementedmicroservices. They also automate routine taskslike notifications.ACI helps to orchestrate containerizedmicroservices in a serverless fashion (thoughlimited in capabilities relative to RegularContainer Orchestrators like AKS / Service Fabric)Serverless may not be good for:1.Inter-cloud portability2.Modernization of applications withsignificant legacy dependencies3.Long-running operations4.Performance-intensive applications12

Splitting ApproachIterative12Identify monolithdecompositionpatternsStrategize ondependencies3Design a decompositionincrement and determinegranularity relative toteams’ maturity45Plan implementation relative totime available / maturity6Implement incrementRegulate/ManageComplexity (microservices)Iterative Identify decomposition pattern - A combination of decomposition patterns including capability-driven, domain-driven, UI-driven etc. areleveraged for monolith decomposition into services Firm up dependencies strategy - A decomposition strategy is firmed up to make sure dependencies shift from the monolith to microservicesbeing implemented, progressively Decompose in increments - Decomposition is performed in increments. Capabilities / components would be identified based on the current stateof decoupling, frequency of alterations etc. Peripheral capabilities that are frequently altered are gain priority. Granularity of decomposition isdriven by factors like current level of decomposition maturity , time available for an increment etc. Plan increment implementation - Implementation is planned in iterations in accordance with time available and the current maturity of theimplementation team Execute increment and move to the next decomposition increment - Increment Implementation is executed as planned Manage / Contain Microservices Complexity - Microservices complexity is continually assessed and complexity containment is strategized on13

Application Migration ApproachUse glue codeApplyDomain Driven Design Stop addingfunctionality to themonolith. Split the front endfrom the back end. Decompose anddecouple the monolithinto a series ofmicroservices. Using domain analysisto modelmicroservicesCreatePresentation layer(Anti-corruption layer) "Glue code" allowsthe monolithicapplication to proxycalls to the newservice to obtain newfunctionality. Ensure that the newservice is not pollutedby data modelsrequired by themonolithic application Separate thepresentation layerfrom the backendlayer. Develop this tier asSingle Page Applicationor MVC application. These applicationsinteract with themicroservices via thegateway, usingstandard HTTP calls.Retire the monolithManage the data Embrace eventualconsistency. Use patterns like“Scheduler AgentSupervisor” and“CompensatingTransaction” for dataconsistency. Keep your servicesloosely coupled. Use Event drivenarchitecture style topropagate changes to itsmodels or entities. Avoid tight couplingbetween publishers andsubscribers. Peel away the monolithby applying Stranglerpattern. Over time, existingfunctionality is movedinto microservices, themonolith will shrink insize and complexity,until it no longer exists. At this point, the anticorruption layer cansafely be removed.14

Data Migration ApproachSource Monolithic Data StoresMigrated Data from MonolithicSystem to Micro Service ParadigmPattern StoreSaga PatternStrangler PatternAPI Composition PatternSupported Script based DataLoading in Target SchemaShared Data Store PatternData Source Scan by respectiveSupported Script based CrawlerScan ReportXML Log tobe used forautomatedDesign InputCommand Query ResponsibilitySegregation PatternAutomatedPattern FitmentRecommendationAnalysis for DesignPDF ScanReport withSalientInformationMachineLearningguided DesignInputsTarget SchemaManualAnalysis ofInformationCreation of Target Schema inChosen Azure Platform(s) bySupported ScriptsDisposition Plan based onSource Morphology andEnterprise ParametersSupplying Enterprise Parameters for DispositionUsage FrequencyRelevanceBusiness ValueData CriticalityTarget DesignStore(s)IdentificationMonolith SplitDesignSecurity forSplit DesignTarget DataStructure15

Detailed Steps for Data MigrationDiscovery Identification of ALLSource Systems Scanning each and everySource System for DataStructure, Relationships,Constraints, Proceduresand Functions, Jobs,Usage Information andother Data Elements Generating XML Log forfurther MachineConsumption towardsAnalysis Generate PDF and ExcelReport for ManualReference towardsManual Inputs duringDesignDisposition Collect Information forDisposition Decisionfrom Discovery Output Shortlist DispositionParameters as applicablefor the target EnterpriseEcosystem [StandardParameters includeUsage Frequency,Relevance, BusinessValue, Data Criticality] Decision on Boundary forShortlisted Parameters Rule basedCategorization [To beMigrated, To beArchived, To be Retiredetc.] of Source Databased on DispositionParametersAnalysis Pre-defined guideline(based on Micro ServiceBest Practices) drivenAnalysis of DiscoveredInformation in light ofDisposition Plan andCategorization Appropriate PatternFitment based on PatternGuidelines Additional AnalysisInformation identifiedmanually moderatingautomaticallyrecommended AnalysisDesignExecutionLoadV&V1/2Base Identification of TargetData Platforms Alignment withArchitecture Principles ofChosen Pattern Design of SPLIT ofMonolith Data Source(s)into one or multipleTarget Data Container(s)in an optimal waytowards zero or minimalredundancyNext Slide Design of Security forSPLIT Data Targets atsame or higher level ofSecurity Design of Data Structurein Target System as XMLinput to Execution Stage16

Detailed Steps for Data on Sanity check for one or multipleTarget EnvironmentsPrevious SlideLoad Sampling Pilot Datafrom SourceSystem(s) Automated Creation of the Scriptto create the Designed TargetData Structure in TargetSystem(s) Script based DataLoading for PilotSampling Run the generated Scripts on toTarget Data Environment(s) tocreate the designed DataStructure Export of wholeSource Data in amutually agreedformat Execution Log in Produce forfinding Failed Cases Data Transformationas per Target DataStructure Failed Scripts to be amendedmanually (if needed) Re-run Rectified Scripts onScheduleV&V Tool based DataStructure Validation Validation Report Tool based DataVerification forrandomly chosenData Sets2/2Base Estimation of theOverall Processkeeping alDependencies inmind Roadmap Creationfor MigrationJourney Verification Report Loading ofTransformed Data inTarget Ecosystems17

Database Design ChallengesChallenges ofMonolithic DatabaseDesignCreates a tight coupling andinability to deploy your servicechanges independentlyIt is difficult to scale individualservices. You only have theoption to scale out the entiremonolithic database.With a single shared database,over a period, you end uphaving huge tables. This makesdata retrieval difficult.Need a solid strategy to split database into multiple small databases aligned with applications. In short we need to decouple applications/servicesfrom using a single monolith shared database.How to Handle Yourdata in a MicroserviceArchitectureMicroservices shouldfollow Domain DrivenDesign and have boundedcontexts.Each microservice should haveits own database and shouldcontain data relevant to thatmicroservice itself.Follow the Code First approachover Data First approach - henceyou design your models firstDatabases should be treated asprivate to each microserviceKeep your mind open towards aNoSQL DB as well if it fits yourcriteriaInstead of an ACID transaction,offload the event messages to aqueue (follow Pub-Sub model)18

Microservices Best PracticesClarity of objectivesGoals of microservices adoption should be very clear. These could be quick technology modernization, faster application release cycles orscalability on cloud, among others. Microservices adoption could be counter-productive in the absence of clear objectives.AgilityMicroservices implementation needs to be quick for realization of benefits. Each microservice should have an optimal granularity anddevelopment cycle. Monolith decomposition should be planned in optimal increments for quick microservices implementation.Monitoring & VisibilityMicroservices volumes tend to grow rapidly. Effective monitoring and clear visibility across the multitude of microservices is key tomicroservices manageabilityAutomationDue to relatively large volumes of small components, automation is key to effective value realization of microservices-composed applications.Consequently, release (deployment, rollback and others), monitoring etc. need to be automated.Microservices BoundariesUse of Domain Driven Design (DDD) Bounded Context principles of boundary-delimitation helps define logical microservices boundaries.However, domain evolution must be addressed with boundaries redefinitionDynamic Scale-up / downCloud compatibility of microservices would help realize the benefits of dynamic / on-demand scale up and scale down in cloud environments.Microservices should be designed with an eye on making them cloud native / cloud-compatible.Fault ToleranceDue to inter-dependencies across a large number of microservices, effective fault tolerance mechanisms would need incorporation. Patterns likecircuit-breaker, and asynchronous event-driven architectures help fault tolerance implementation.Centralized Logging & DashboardDebugging across the multitude of microservices would require putting in place centralized logging and dashboards across services19

Thank You

Microservices require writing Circuit breaker and Retry communication patterns to achieve resiliency. Thus increases overall complexity and maintenance of code to avoid application disruption Resiliency 3 Eventual Consistency 4 There is higher cost while running each of the microservices in