CASE STUDIES BUSINESS INTELLIGENCE - Onwelo

Transcription

CASE STUDIESBUSINESSINTELLIGENCE

01.Client:German company that connects two representatives of high-class globalbankingProduct / Service:Construction of generic ETL data flows supplying Data Hub and Raw INlayers as well as development analysis and testing assistanceIndustry:FinancialGoal:To implement a new generic way to supply the data warehouse from allsource systemsChallenges and problems¹ No test approach and strategy¹ Current solution generated a large amountof work with every change in the sourcesystem¹ Limited analytical documentation¹ Very limited documenting of developmentprojects¹ Lack of sufficient development team¹ Undefined range of requirementsTasks¹ Agreeing on strategies, architecture, directinvolvement of IT and business¹ Implementation of development strategy (documentation, requirements, codingstandards)

¹ Implementation of data flows from thesource system to the Data Hub layer (generic solution)¹ Implementation of data flows from the DataHub layer to Raw IN layer (generic solution)¹ Implementation of data flows from datawarehouses to the Data Hub OUT layer(generic solution)¹ Defining the development and testing process in consultation with client management¹ Providing detailed documentation of thesolution¹ Operational support as part of the production installation for the client’s teamClient:International bank operating all over the world02.Product / Service: Defining reporting processes Migration of processes between databases Implementation of new requirementsIndustry:BankingGoal: To migrate existing processes to SAS systems and implement newrequirements To optimize and modify working reports To create statements and support banking processes To re-certify and withdraw old processes To perform UAT tests on system toolsChallenges and problems¹ Lack of technical and business documentation of existing processes¹ Lack of technical skills of individuals who arethe recipients of reports¹ Lack of people responsible for the currentsolution¹ Communication problems on business-ITline and related difficulties

¹ No development procedure — every individual developer creates processes in a waythat other developers do not know¹ Various naming of the same identifiers anddata in the database¹ Application overload resulting from not optimally created processes by too many users¹ Bad sign-up process and interception of errors occurring in processes and applications¹ Errors in data retrieved from the database¹ Poor way of marking the required information in the databaseTasks¹ Migrating existing processes between theOracle and SAS databases¹ Implementing new requirements for existing processes¹ Creating new processes and tasks for reporting purposes¹ Creating and modifying a system for handling customer refunds regarding UOKiKdecisions¹ Recertification of SAS reporting processesafter the migration period¹ Conducting tests of solutions being developed¹ Optimizing and modifying existing processes¹ Handling and creating cron processesClient:One of the largest universal banks in Poland03.Product / Service:Migration of data warehouse supply from Oracle solutions (OracleWarehouse Builder, PL SQL) to IBM InfoSphere DataStageIndustry:BankingGoal:To transfer existing ETL processes to IBM InfoSphere DataStage

Challenges and problems¹ Migration scale (c. 500 stars, over a billionlines processed as part of daily data loading,over 100 GB of daily data increments)¹ Tests and cooperation of development teamwith data warehouse owners¹ Incomplete business documentation¹ Optimization of warehouse loading time using the strengths of DataStage environment¹ Preparation of business test cases for developer tests¹ Changing the approach in ETL solutions byeliminating the Operational Data Store layer¹ Cooperation with teams from another cityTasks¹ Developing agile ETL processes and minimizing the risk of errors¹ Developing a communication path with DataOfficers responsible for testing individualwarehouse areas¹ Providing support during tests — helpingto find errors and discrepancies betweenthe documentation and the actual statusof processes¹ Cooperating with other customer teams¹ Automating some aspects of data warehouse migration¹ Ensuring data lineage and strict adherenceto technical conventions established priorto migration¹ Refreshing the documentation and creatingnew documentation based on metadata,in a manual and automated manner, usingopen source tools¹ Self-organization of the team in the scrummethodology¹ Support for a dedicated customer framework (configuration, parameterization)¹ Support for test environments and implementation of test cases as part of UAT tests

Client:German chain of stores offering electronics and home appliances04.Product / Service:Development and maintenance of data warehouses and BIs embeddedin the Azure cloud environmentIndustry:FMCGGoal:To implement new business functions for the DWH/BI environment andto maintain this environmentChallenges and problems¹ Takeover of the solution after previous vendor disappeared suddenly¹ Undefined ecosystem — no processes responsible for deployment, testing, changemanagement, no environments or platformmaintenance processes¹ Limited trust in vendors (effect of cooperation with the previous vendor)¹ Lack of business and system analysts on theMMS side (difficulty in defining requirementsand acceptance of projects)Tasks¹ Requirement analysis¹ Solution design Use of Azure cloud components (PAAS,SAAS) Optimizing the solution¹ Defining processes supporting the environment¹ Process implementation¹ Implementation testing¹ Deployment

05.Client:The Polish branch of one of the largest insurance companies in theworldProduct / Service:Creation of an anti-fraud system for controlling internal malpracticesIndustry:InsuranceGoal:To build a platform supporting fraud detection process in the company— identifying policies sold by blacklisted agents, payment analyses,welcome areaChallenges and problems¹ Launching the system on a new global BigData platform¹ Necessary reverse engineering of the previous solution based on VB¹ Implementation of the agile scrum method¹ Improving the effectiveness of detectionprocesses¹ Building a system that will easily be scalableto the entire organization¹ The system must be easily expandable toother process areasTasks¹ Reverse engineering of the previous VBsolution (several dozen Excel files and MSaccess files)¹ Solution design¹ Launching dev/test and production environments¹ Process implementation¹ Launching scrum framework, stakeholdereducation¹ System testing¹ Deployment

Client:Austrian commercial bank06.Product / Service:Development, maintenance and optimization of data warehouses Data model prepared for the needs of the Polish and Austriancompany Optimization, development and maintenance of data warehouselayers Analyses of data marts and reportingIndustry:BankingGoal: To maintain and develop systems after the division of banks To create new data warehouse components for new systems To optimize data warehouse only for the necessary data in the area To exclude unnecessary processes after dividing the bankChallenges and problems¹ No access to production data¹ Incorrect and incomplete test data¹ Lack of people who are familiar with theadopted target systems¹ Limited analytical documentation¹ Lack of complete knowledge about the datawarehouse¹ Lack of people who are well-versed in sourcesystems¹ Resource restrictionsTasks¹ Developing a roadmap for system optimization¹ Optimization and development in the datawarehouse¹ Arranging the scope of works in the warehouse¹ Developing and testing introduced changes¹ Analyses of source systems, data warehouses and extracts for external systems

Client:One of the leading mobile network operators in Poland07.Product / Service:Creating control for a new platform for billing prepaid clientsIndustry:TelecommunicationsGoal:To design, develop and deploy a set of new control methods for thenewly created billing systemChallenges and problems¹ Analysis of the new billing platform, newservices, free units, relations between themand analysis of the billing method¹ Developing a migration plan for clients fromexisting control methods to new ones¹ Rewriting the existing code from scratch,including the optimization techniques¹ Finding where the new systems store theinformation needed for controls defined bythe business (e.g. regarding usage limits)Tasks¹ Designing and building 11 new usage controls — comparison of individual events atthe MSISDN level¹ Designing and building 9 new configurationcontrols — comparison of individual parameters at the given MSISDN level¹ Migration of existing 20 reports to the newplatform, rewriting the code in PL/SQL fromscratch

¹ Migration of existing workflows in Informatica as well as designing and adding new ones¹ Designing new OLAP cubes and views thatsupply them¹ Writing a new loader for loading CDRs fromthe new system¹ Conducting tests with a business teamClient:One of the leading mobile network operators in Poland08.Product / Service: Online cluster for monitoring applications / systems / containers andinfrastructure based on collected metrics and logs Integration with monitored systems Integration with the Umbrella Monitoring System Alarm generation Online dashboards Business analyses with owners of integrated systemsIndustry:TelecommunicationsGoal:To deploy a system for monitoring applications / systems / containersand infrastructure based on collected metrics and logsChallenges and problems¹ Lack of customer experience with selectedtechnology – inadequate technology forsome requirements¹ Open source technology — insufficientdocumentation, no authentication and authorization mechanism and no possibility topurchase a license for a given functionality¹ Lack of dedicated infrastructure — insufficient resources¹ Lack of defined requirements

Tasks¹ Upgrading the existing environment to anewer version and changing the configuration¹ Proposals for integration, alert rules, dashboards¹ Securing the environment, implementing theauthentication and authorization mechanismmodule¹ Integration with monitored systems¹ Implementing alerting module¹ Creating alert rules¹ Integration with the Umbrella MonitoringSystem¹ Administrating and maintaining the environment¹ Business meetings and analyses with ownersof integrated systems¹ Creating the architecture of the next versionof the environment09.¹ Creating dashboardsClient:A company from the automotive industry, a global leader in the field oftransmission and suspensionProduct / Service:Monitoring of infrastructure and logsIndustry:AutomotiveGoal:To deploy a framework for infrastructure monitoring for 80 hostsincluding interactive and in-depth analyticsChallenges and problems¹ Development of a unified model of application logs¹ Preparation of one repository containing datafrom multiple machines ( 80)¹ Detecting bottlenecks in processing¹ Stream data processing¹ Installation and configuration of the environment

Tasks¹ Developing framework for the review ofinfrastructure and application logs in realtime¹ Developing PoC (after installing the X-Pack)of machine-learning system to prevent errors¹ Using technologies: ELK Stack (Elasticsearch, Kibana, Logstash, Metricbeat, Filebeat, Ansible, Kafka, ZooKeeper)10.Client:A company from the automotive industry, a global leader in the fields oftransmission and suspensionProduct / Service:Development of an analytic layer for metadata describing videos fromcar camerasIndustry:AutomotiveGoal:To develop an efficient search system that combines defined text andnumerical criteriaChallenges and problems¹ A large amount of data (thousands of hoursof recordings described by millions of metadata files)¹ Analysis of problems resulting from previousimplementation attempts¹ Lack of a coherent and uniform metadatastructure¹ Combining data searching from varioussources into a single context (flat files, database, other)¹ Thousands of attributes describing individualsource videos¹ Developing a data model that is convenientto analyze

Tasks¹ Developing an index containing millionsof documents¹ Automation and deployment of ELK production environment¹ Using technologies: Python — itrk Mobileyedata transformation (automotive standard),ELK Stack (Elasticsearch, Kibana, Logstash),integration with core system of the RSZAWAKRAKÓWKIELCEul. Karolkowa 3001-207 Warszawawarszawa@onwelo.comul. Prądnicka 20a30-002 Krakówkrakow@onwelo.comul. Zagnańska 4925-528 Kielcekielce@onwelo.comKATOWICEGDAŃSKLUBLINul. Sokolska 6540-087 Katowicekatowice@onwelo.comul. Azymutalna 980-298 Gdańskgdansk@onwelo.comul. Nałęczowska 1420-701 Lublinlublin@onwelo.comNEW YORKLEIPZIGLUCERNE85 Broad StreetNY 10004nyc@onwelo.comPetersstraße 5004109 Leipzigleipzig@onwelo.comHirschengraben 336003 Lucerneluzern@onwelo.comONWELO SA, Karolkowa 30, 01-207 Warsaw. Registration court: THE DISTRICT COURT FOR THE CAPITAL CITY OFWARSAW IN WARSAW, THE 12TH COMMERCIAL DIVISION OF THE NATIONAL COURT REGISTER, NIP: 5272739561.The amount of share capital: 1 413 751,00 zł.

¹ Implementation of data flows from the source system to the Data Hub layer (ge-neric solution) ¹ Implementation of data flows from the Data Hub layer to Raw IN layer (generic solution) ¹ Implementation of data flows from data warehouses to the Data Hub OUT layer (gene