Transcription
Data Governance & the Road to Compliancewith Apache KafkaHow to create secure, controlled data access for all
Table of contents1Data compliance2Why is it so important?3Apache Kafka4Challenges with Kafka5Kafka self-service6Governing data7Kafka governance with Axual8Use case - Rabobank9Free trial
DATA COMPLIANCEData is everybody’s businessDigital transformation has made data “the new oil” of the21st century economy. And the data that fuels us needsto be secure, real-time and compliant. That means everyone of us — from smartphone owners to the Pentagon —must make data governance and compliance our business.Protecting data means complying with legislation toprotect sensitive digital assets by storing and managingthem to safeguard against loss, theft, corruption andransom. It makes sure that user access is tightly controlledto detect and thwart unauthorised attempts at entry.Compliance starts with governance. It’s your baseline.If you don’t have the means to organise and control yourdata, you can never move on to the next step: complyingwith all applicable regulations.
WHY IS IT SO IMPORTANT?Compliance is mandatoryInadequate data protection leaves you vulnerable on threefronts: client litigation in the event of a breach, publicprosecution for violating regulations, and ransomwareFINANCEIn the financial sector, simple compliance is rarely enough.attacks.Watchdogs keep a close eye on every move and consumersIf the integrity of your data is at risk, it can also damageproviders can’t afford to see their data protectionyour reputation and even close your business. But gooddata management and proactive security measures aren’tjust about defence. They are also essential to secure trustdemand 100% security and reliability. Financial servicecompromised. Data integrity and data security strategiesgenerally need to exceed the minimum requirements ofcompulsory compliance.between the security provider and the data’s owner.ENERGYCompanies in the energy sector are also legally compelledto comply with data protection laws, like GDPR and itsdizzying array of rules for data storage. The transition torenewable energy demands insights into market andcustomer behaviour — insights that feed on data. Smartsolutions allow you to safely handle and accessever-increasing data flows.
APACHE KAFKAThe gold standard in data streamingApache Kafka is an open-source distributed eventstreaming platform. It provides high-performance datapipelines, streaming analytics, data integration, andMore than80% ofFortune 100 companiestrust Kafkamission-critical applications.With more and more companies using Kafka, particularly70% in Banking & Financein the Finance and Energy sectors, it has established itselfas the de facto standard for data streaming today.60% in Energy & Utilities100% in IT & Services
CHALLENGESChallenges of using KafkaWhen it comes to compliance, there are pain points anyWhich applications? Which permissions? Which topics?organization working with Kafka needs to overcome.By default, authorizing applications to access specifictopics is not configured on Kafka. It means anyone canWho owns it?produce and consume to any topic. Depending on yourKafka Broker configurations all have their defaults, butneeds, you might opt for Mutual SSL to secure your topics.some of those defaults don’t really work in your favor. AutoBut then you need to administer access by applications andtopic creation is one of those examples. Nonexisting topicsprincipals.are automatically created without any check oncompleteness or correctness. A simple typo in the producerThis can quickly become an administrative nightmare forconfiguration can create a real mess. How can you clean ityour platform team, who are forced to rely onup? Does the topic hold sensitive data? Who can theadministration scripts or poorly supported, unreliable openconsumer contact with questions?source tools — if there are any.Where did my topic go?As easy it is to create a topic, it is just as simple to deleteone. That is not such great news if you are relying on thetopic data for your consumer application.
KAFKA SELF-SERVICEWhy self-service makes senseYour DevOps teams need to focus on deliveringfunctionality. They can’t get their job done if they arecontinually occupied by compliance issues.Kafka Self-Service provides a secure and effective solutionfor data governance. At the same time, it resolves manyother issues by limiting the possibility of non-compliance.Using a self-service interface is an efficient way to enforcea governance structure. It allows DevOps teams to freelyinteract with Kafka, while conforming to your ownpredetermined rules and practice.By allowing developers to create and manage topicsthemselves, they can do their best work without relying ona central team — ensuring autonomy, efficiency and quickfeedback.
GOVERNING DATAGovernance to the rescueWhat organizations need is a structure of data governancethat allows for secure, controlled access for Kafka topicadministration — without losing essential business agility.BUSINESSAGILITYDATAGOVERNANCECOMPLIANCE
OUR SOLUTIONHow we do itStream data and application ownership are key.Self-service offers agility.No Kafka topic is created before an owning team andOur Stream Team guarantees the streaming platform SLAmetadata has been defined. Each team has thefor you. That means your DevOps teams can focus onresponsibility to selectively allow produce/consume accessconfiguring and controlling their topics and applications.to the topic. A one-stop, easy-to-use, self-service interfaceNo need for a central gatekeeper as the responsibility liesmaintains the big picture within your organization.with the people who know most about the data.Security is at the heart of our platform.Environment-specific rules allow precise customization.Every single application must use SSL certificates in theOur streaming platform enables more fine-grainedconnection to Apache Kafka. To prevent cross-environmentgovernance through environment-specific rules. Ondata mangling, exactly one certificate is used pernon-production environments, where no real customerenvironment, which is validated to make sure only trusteddata exists, the rules can be a bit more relaxed as comparedconnections are allowed.to environments with sensitive data.
Challenges:USE CASE Anyone can create and delete (oops!) a topicHow Rabobank achieves with KafkaSafely scale Kafka with Axual: As Kafka topics are being created by teams, theyDEVOPS TEAMSREGISTEREDAPPLICATIONS180 DATA STREAMS INPRODUCTION0.5 – 1BMESSAGES SENTPER DAY Extremely hard to continuously store secure dataagainst threats and therefore to comply with e.g. GDPR.120 400 Anyone can produce and consume from any topicautomatically become the owner Teams are in charge to allow produce/consumerequests from any application to/from their topics The Netherlands Financial services 48,000 employees Serving 9.5 million Every single application uses secure andauthenticated connections to Apache Kafka Teams are able to safely test different topicconfigurations or schema versions in a non-productionenvironment, before going to production.customers worldwide 681 billion in assetsBusiness impact & ROI: Teams move faster due to self-service capabilities –no need to rely on a central data team Better sleep knowing your teams are compliant toGDPR and other regulations Saved 400,000 in maintenance costs – themaintenance team shrunk from 6 FTE to 1 FTE
Experience it yourselfDiscover the all-in-one Kafka platform. Get started todaywith a free SaaS trial or request a personal demo.START FREE TRIALREQUEST A DEMO
Apache Kafka is an open-source distributed event streaming platform. It provides high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. With more and more companies using Kafka, particularly in
