Transcription
BUILDINGA FUTUREREADY DATAARCHITECTUREOVERVIEWAccenture helpeda leading globalcollaborationtechnology companyprepare for its IPOby reengineering itsdata platform and datagovernance processeswith a future-readyarchitecture thatenabled scalable andtrusted reporting ofbusiness metrics.1 BUILDING A FUTURE-READY DATA ARCHITECTUREApproaching its IPO, a leading technologyunicorn was challenged with limited trust indata accuracy, eroding the ability of leaders toconfidently characterize the performance of thebusiness. As a consequence of rapid growth toglobal scale, internal data systems grew up insiloes — aligned with the different functionalareas of the business — with no centralized datagovernance process or data architecture. Thissituation manifested in multiple problematic ways: Analysts spend 100s of hours onManagement Reporting Unpredictable and inefficient queryperformance Interactive/Ad-hoc analyticsis slow SOX controls & compliance difficultto implement Rigid architecture not extensible asBusiness Evolves
The lack of governance in the company’sdata model allowed anyone in the business toinstantiate or redefine key metrics, but this meantit lacked a clear audit trail to track adherenceto standard business definitions, evolution ofmetrics over time, and computational efficiencyand accuracy. They needed to be able to manageand communicate their business performancewith clear confidence and limited manual effort —this required data clean-up, metric certification,and the establishment of repeatable processesand scalable architecture.Accenture initiated an effort, starting withan evaluation of the Top 20 priority metrics,each of which had limited data veracity.Early work included development of datadefinitions, agreement on key data sources.Alignment across multiple stakeholders wasrequired to consider alternatives (Oracle,Vertica, RedShift, Snowflake), and thensocialize the results and achieve sign off. Thisled to providing assistance with the new dataarchitecture design and implementation.With Accenture’s help, this company hasrearchitected its data pipelines using afully relational, as-a-service Snowflake datawarehouse — with a robust data governanceorganization on top. Snowflake provided theability to scale up and down, automatically andon the fly, providing the exact performanceneeded, at the time needed.This new Data & BI systems enabled by Snowflake& Tableau Dashboards made insights available asclose to real time as possible, ensuring businessusers aren’t just more confident in the company’smetrics, they also had an easy and intuitive way todrill down into the data and develop new insightsat a speeds previously impossible. Together, thetwo organizations have created a flexible, futureready data architecture — one fit to support thebusiness as it scales to new heights.2 BUILDING A FUTURE-READY DATA ARCHITECTURETHE STORYCLIENT PROFILEThis recently IPO’d high growth SaaS client hasenjoyed explosive growth over the past decade,and continues to expand and serve consumersand business users around the world.OPPORTUNITYHaving enjoyed stratospheric growth since itsfounding a decade ago, a leading technologyplatform found itself with a data challenge as itprepared for its IPO. Its business had grown sofast, and on such a scale, that its internal datagovernance and architecture were in need ofan upgrade.With anyone in the business able to redefinekey metrics at any time without version control,the company lacked a clear way to track howits performance data had evolved over time.What’s more, some of the SQL queries used tocreate the metrics were becoming convolutedand complex, making them less computationallyefficient than they could have been.Left unchecked, this data model risked thebusiness losing confidence in the metrics beingproduced – everything from monthly recurringrevenue, to the different platforms supported,right up to the total number of registered users.With an upcoming IPO making robust companyperformance data all the more important,everyone at this leading unicorn knew a rethinkof internal data management was called for.Recognizing the time and effort involved inputting its own team together, the companyasked Accenture to help it take data managementto the next level. The vision? To build a flexible,fast, future-ready data architecture — andcompliment it with a far more mature approachto data governance.
SOLUTIONAccenture got to work straight away, puttinga small team of focused experts on the groundwith an initial goal of helping the company mapthe lineage of all its existing data and metrics.By conducting detailed interviews with dataowners and gathering intelligence from otherstakeholders, the team quickly establishedthe true “as is” state of the company’s datamanagement.The next step was to evaluate options toimprove speed and usability. Reengineeringthe company’s data lake itself would have beena highly complex and lengthy process, so theteam settled on a smarter and more efficientsolution. By pulling all the data from the datalake and putting it into a new data store, theyrealized they’d be able to radically simplify andstreamline the reporting and making the ad hocanalysis far faster.Using Snowflake as the new data store meantthe company would benefit from exceptionalcomputational performance delivering close toreal-time metrics. The technology’s scalabilitywould also ensure the business could handle notonly today’s terabytes of data volumes, but themultiple petabytes of data likely to be requiredfor even a single performance metric in just afew years’ time. Choosing Snowflake meantthe company could perform simultaneous dataloading and computation — a unique feature notfound in any other data warehouse solution and asignificant boost for productivity and efficiency.So, taking each metric in turn, the teamoptimized ETLs to minimize errors, assisted indevelopment of the data pipeline to pull datainto AWS and ingested data loads into SnowflakeData Store using Snowpipes, Snowflake’scontinuous data ingestion service aftersimplifying the underlying queries and pushthe resulting insights to a new centralized anduser-friendly dashboard powered by Tableau.But creating faster, scalable, more usablemetrics solved only part of this company’sdata challenge. It also needed better data3 BUILDING A FUTURE-READY DATA ARCHITECTUREmanagement to prevent inefficienciesand establish process evolved from bestpractices. So the team developed a setof recommendations for a mature datamanagement model, including data hygienefactors and a data governance board.RESULTSWithin a year, the team had upgraded all thecompany’s metrics and created a robust datamanagement structure to continually supportthem. The result was a dramatic boost inthe company’s confidence in its own data —especially important as it approached its IPO.With help from Accenture, the organizationnow benefits from: Sleek and user-friendly data dashboards.No longer required to write SQL queries,extract CSV files, and plot metrics inindividual spreadsheets — everyone acrossthe business can now access intuitiveand simple-to-use visualizations of keycompany data — instantly. Deeper insights from drill-down capabilities.Business users no longer have to laboriouslyrun separate queries for each variationof a metric — the new solution allowsthem to immediately slice and dice anycombination of top-line figures andcompare the results by geography, timeperiod, or other parameters. Daily business insights. The speed andsimplicity of the solution is saving hundredsof analyst hours every year. And becausethe dashboards are updated dynamicallyin close to real time, the company is ableto track changes in performance andcustomer behavior as they happen. Ability to scale metrics at the point of need.Snowpipes give business users access tosimple “drag and drop” functionality forcreating new metrics. Whatever analysisof a business need — the success of the
latest marketing campaign, the results oflatest talent acquisition programs or what’spopular in the cafeteria — if they have thedata, the business can create the metricsat a pace that was previously unthinkable. Mature data management. With changesto data and metrics now approved,documented, and tracked by a datagovernance board, the company canbe sure no arbitrary or undocumentedmodifications are made to complex dataqueries — building confidence in the dataveracity and security.Foundation for intelligent operations. Withthe data warehouse and data pipelinesup and running, the business has anindustrialized data platform for exploringmachine learning and automation, openingup new possibilities for company insightsand performance gains.“It all soundsreally simple,but it is not thatsimple when itcomes to us We basicallywent and lookedat what are the“main” KPIs andhow are theycalculated?That’s whereAccenture wasreally helpful!”Lead Data Engineer4 BUILDING A FUTURE-READY DATA ARCHITECTURE
FOR MORE INFORMATION,PLEASE CONTACT:Craig VaughanManaging Director, Applied Intelligencecraig.vaughan@accenture.comAsh KrishnanSenior Manager, Communications,Media & Technologykrishnan.ashok@accenture.comJacqueline MorganSr. Principal, Software & Platformsjacqueline.morgan@accenture.comABOUT ACCENTUREAccenture is a leading global professionalservices company, providing a broad range ofservices and solutions in strategy, consulting,digital, technology and operations. Combiningunmatched experience and specialized skillsacross more than 40 industries and all businessfunctions — underpinned by the world’s largestdelivery network — Accenture works at theintersection of business and technology tohelp clients improve their performance andcreate sustainable value for their stakeholders.With more than 477,000 people serving clientsin more than 120 countries, Accenture drivesinnovation to improve the way the world worksand lives. Visit us at www.accenture.com.Content current April, 2019Copyright 2019 Accenture.All rights reserved.Accenture and its logoare trademarks of Accenture.
led to providing assistance with the new data architecture design and implementation. With Accenture's help, this company has rearchitected its data pipelines using a fully relational, as-a-service Snowflake data warehouse — with a robust data governance organization on top. Snowflake provided the ability to scale up and down, automatically and
