WIXLIB

Microservices with Apache Karaf andApache CXF: practical experienceAndrei Shakirin, Talend

Agenda Microservices and OSGiCore ideas of OSGiApache KarafDesign and develop in OSGi: the history of one projectRemote communication in OSGi with Apache CXFConclusions and lessons learned

About Me Software architect in Talend Team PMC in Apache CXF Contributions in Apache Syncope, ApacheAries and Apache Karaf

Microservices(James Lewis and Martin Fowler) Application as suite of small servicesOrganization around business capabilitiesEach service runs in own processSmart endpoints and dumb pipesDecentralized data management andtechnologies Infrastructure automation

Microservices: Pros and ConsPros: Services themselves are simple, focusing on doing onething well Systems are loosely coupled Services and can be (relatively) independentlydeveloped and deployed by different teams Services can be scaled differently Services can (but not must) use different technologiesand languages

Microservices: Pros and ConsCons: Remote calls are expensive and unreliable Change syntax or semantic of remote contracts introducesadditional risks Mistakes in services boundaries definition are costly Testing, debugging and monitoring in distributed systembecame more difficult Infrastructure becomes more complex Eventual consistency

OSGi Modular ApplicationsWhat is the module?

OSGi: Modules and Modularity

OSGi: Modules and Modularity

OSGi: software modules Implements a specific functionCan be used alone or combined with othersProvides functionality to be reused or replacedHas well defined nameHas a versionjars modules

OSGi: software modulesBut: It is hard to control which version of the functionalitywill be used at the runtime You cannot encapsulate functionality in the module Self-describing module contract is missing

Keep the name and version of JAR fileAdd explicit package dependencies (requirements)Add explicit package exports (capabilities)Provide API as external contract (OSGi services)OSGibundle

OSGi Services Service Contract is one or more java interfacesBundle can register the service in registryOther bundle can get and listen for the serviceMultiple registered services can be distinguished using propertiesNo any coupling between bundles except Service Contract: neither incode, no on the classpath (different to java ServiceLoader)

Declare OSGi Services: Option 1 Declarative ServicesChristian Schneider Blog: “Apache Karaf Tutorial part 10 - Declarative services”

Declare OSGi Services: Option 2 Blueprint

OSGi DecouplingActiveMQ BundleMyAsyncCommunication BundleExports:Imports:org.apache.activemq.pool 5.14.0,org.apache.activemq.command 5.14.0org.apache.activemq.pool [5.14,6),org.apache.activemq.command [5.14,6)DecouplingMyBusinessDomain BundleImports: my.connector.async[1.0,2)JMS CommunicationImplementationExports: my.connector.async1.0.0Import Service:Export Services nnector.async.Listener

OSGi DecouplingArticle Logic BundleArticle API BundleExports: my.domain.article1.1.0Imports: my.domain.article(my.domain.article.Availability interface)[1.0,2)AvailabilityDAOBundleAvailabilty LogicImplementationMyCartService BundleImports: ngImport Service:Export Service main.article.Availability

Classic Microservices vs OSGiAspectMicroservicesOSGiApplication structureSuite of small servicesSuite of bundles / modulesBoundariesAround businesscapabilitiesModularization aroundbusiness and technicalaspectsCommunicationLightweight remoteFlexible: local or remoteContractRemote APILocal java interfaces orremote APIDecentralized DataManagementDesiredDepends on requirementsfor single process, desiredfor multiple processesInfrastructure AutomationDesiredDesired

Apache Karaf OSGi based Container using Apache Felix or Eclipse Equinoximplementations Runs as Container, Docker Image, embedding (karaf-boot) Provisioning (maven repository, file, http, ) Configuration Console Logging, Management, Security

Karaf 0/xmlMaven Repo / ositories , turesBoot , order-serviceconsolefeature:addurl ture:install order-service

Migration to OSGi in eCommerce Project Business Domain: WebShop, eCommerce Team: 20 – 30 persons Initial technologies: Java, Spring, Hibernate, Apache CXF,Apache Camel, ActiveMQ, Tomcat Current technologies: Java, Hibernate, Apache CXF, ApacheCamel, ActiveMQ, OSGi Apache Karaf

Online Shop ArchitectureWeb BrowserExternal consumersFrontendMobile AppRESTOnline eck SystemSAPOracle DB

Online Shop DesignRESTRESTRESTRESTUser ServiceCart ServiceArticle ServiceOrder ServiceCustomer DomainArticle DomainOrder DomainCore DomainDAOsSAPDBDBMessagingDB

Online Shop DesignRESTRESTRESTRESTUser ServiceCart ServiceArticle ServiceOrder ServiceCustomer DomainArticle DomainOrder DomainCore DomainDAOsSAPDBDBMessagingDB

Step 1: Packages address Classes for business function are grouped together - high cohesion Less dependencies between packages - low coupling Private and public packages are easily recognizable (model, api, impl)Christian Schneider ApacheCon Europe 2014 “Reflection of Design of Business Applications”

Step 2: Connectors APIRESTRESTRESTRESTUser ServiceCart ServiceArticle ServiceOrder ServiceCustomer DomainArticle DomainOrder DomainCore DomainAPI: OSGI SvcAPI: OSGI SvcAPI: OSGI SvcAPI: OSGI ServicesDB ConnectorDB ConnectorDB ConnectorSAP ConnectorAPI: OSGI ServicesMessaging ConnectorSAPDBDBDBActiveMQ

Step 3: Parallel Web And OSGi DeploymentModule CoderesourcesspringOSGI-INF/blueprint packaging bundle /packaging plugin groupId org.apache.felix /groupId artifactId maven-bundle-plugin /artifactId /plugin OSGiFeaturesWeb ApplicationWAR

Step 4: Refactor Complex Domain Logic (Camel essing().timeout(100).to(„direct:getPrice“).to bilityValidator).bean(priceAvailResponseMapper) What type of data is transmitted? Debug me Would it be harder in plain Java?1PriceAndAvailResult getPriceAndAvail (Cart cart,AvailabilityOptions options);2ATPOptions atpOptions mapAvailabilityOptions(options); final Future AvailabilityReturner availabilityFuture executorService.submit(availabilityTask);final Future PriceReturner priceFuture executorService.submit(priceTask); sult result new PriceAndAvailabilityResult(availability,price); Type safe interfacesClearly shows what data is proceedNot essentially verbose as Camel routeEasy to debug and understand for teamChristian Schneider ApacheCon Europe 2014 “Reflection of Design of Business Applications”

Step 4: Domains APIs And RefactoringRESTRESTRESTRESTUser ServiceCart ServiceArticle ServiceOrder ServiceAPI: OSGI ServicesCustomer DomainAPI: OSGI ServicesArticle DomainAPI: OSGI ServicesOrder DomainAPI: OSGI ServicesCore DomainAPI: OSGI SvcAPI: OSGI SvcAPI: OSGI SvcAPI: OSGI ServicesDB ConnectorDB ConnectorDB ConnectorSAP ConnectorAPI: OSGI ServicesMessaging ConnectorSAPDBDBDBActibeMQ

Step 5: Separate containers for some servicesRESTRESTContainer MiddlewareRESTRESTUser ServiceCart ServiceRESTRESTCheckout ServiceOrder ServiceContainer EventRESTRESTEvent ServiceRESTContainer ArticleMessagingRESTNewsletterServiceRESTDBArticle ServiceRESTAddress ServiceSAPDBRESTMessagingDBMessaging

REST Communication in OSGi Aries Remote Service Admin (RSA)Christian Schneider “Lean Microservices on OSGi“, ApacheCon EU 2016 Explicit via CXF

Design For Failure With Hystrix(Netflix)

Resilience With Hystrix

Resilience With Hystrix

Conclusions and Lessons Learned Design your application modular (either in OSGi or not) Care about decoupling between modules, high cohesion insidethe module and modules dependencies Continuously refactor your modules to achive optimalboundaries Stay on single process at the beginning, split application intodifferent processes only if it is required and brings benefits Define your remote and async APIs carefully, design remotecalls for failure

OSGi Critic and MythsOSGi is complex: in understanding, in build, in deployment and indebugging and has poor tooling supportThe most understandable specification in the world(inclusive HTTP, ConfigAdmin, DS, RSA, JTA, JMX, JPA) packaging bundle /packaging plugin groupId org.apache.felix /groupId artifactId maven-bundle-plugin /artifactId /plugin Features, configuration, security, console

OSGi Critic and Myths

OSGi Critic and MythsOSGi is not supported by frameworks and libraries

OSGi Critic and MythsOSGi is not supported by frameworks and libraries

OSGi Critic and MythsThe most important OSGi feature is hot updates: install, delete orreplace the bundle on the flyYes, OSGi is designed for updates without restarting the wholeapplication, but:1. Normally it is safer to restart the whole Container to havereproducible state in production2. Hot deployment is not a free lunch: application have to be designedand tested for that3. The main OSGi gain is not a hot deployment, but clean modularapplication design, isolation and decoupling of modules. Hotdeployment is more derivative feature4. Can be useful in developer environment, special use cases, partlyrestarts

REST Communication in OSGi Consider REST Architectural Style principles (resources design,verbs contracts, response codes, statelessness) Reuse CXF providers, features and interceptors (logging,security) Customize (if necessary) through own JAX-RS Filters andInterceptors, MessageBodyReaders and Writers,ParamConverters, CXF Interceptors Consider to use Swagger to document and test your API Make your external calls resilient

OSGi DecouplingHibersap BundleMySapFacade BundleExports: org.hibersap,Exports: ibersap.mapping.modelDecouplingMyBusinessDomain BundleSAP JCO CommunicationImplementationImports: my.connector.sapExports: my.connector.sapImport Service:Export Service ector.sap.OrderExport

Karaf DeploymentConfigured as Jenkins JOBs with folwoing steps:1. Stop Karaf Instance2. Replace org.apache.karaf.features.cfg3. Start Karaf Instance4. Waiting for AvailabilityService

System veloper Tests QA Performance tests Load testsLBProduction

Swagger in JAXRS API

Swagger in JAXRS API: Java First

Swagger in JAXRS API: WADL First

Microservices: Pros and Cons Cons: Remote calls are expensive and unreliable Change syntax or semantic of remote contracts introduces additional risks Mistakes in services boundaries definition are costly Testing, debugging and monitoring in distributed system became more difficult Infrastructure becomes more complex