WIXLIB

Another research-based maturity model is the“Value-Based Big Data Maturity Model” proposed in[27]. It focuses on the data quality and argues that thisis critical in gaining a competitive advantage. The modelproposed by Dremel et al. follows a similar path andincorporates 34 generic capabilities necessary toleverage the potential of big data analytics [28]. Themodel was developed using input from consultantsworking for a single company. Furthermore, the resultshave not been empirically validated. Cosic et al. [29]proposes a business analytics capability modelconsisting of 16 capabilities grouped under 4 areas(governance, culture, technology, and people). Similarto the abovementioned models, an empirical evaluationof this model is lacking.In summary, several maturity models for advanceddata analytics and related domains have been proposedby researchers and practitioners. These models are oftendomain-specific or focus solely on specific topics (suchas the data quality). Many of these models do notincorporate necessary capabilities, nor are subjected torigorous validations. Some do not provide detailsregarding their application, making their use in practicedifficult. Moreover, many of such models focus solelyon technological capabilities, ignoring the factorsregarding the organization and context.2.2. ADA and Firm PerformanceADA can help firms understand the organization andits competitive environment better and be more agile instrategic decision-making [30]. When it is strategicallyand technically implemented in an organization, it canpromote innovation in product/service offerings andimproved business decisions [2], [10]. Thus, it can be asignificant differentiator between high and lowperforming firms [31]. However, improving ADAcapabilities and thereby achieving higher firmperformance is a formidable undertaking [32].The impact of ADA capabilities on value creationand firm performance has attracted attention of manyresearchers [33]. Elia et al. [34] explores ADA’spotential to contribute to value creation in variousorganizational dimensions, e.g., strategic value forcompetitive advantage, transactional value foroperational efficiency, and transformational value forbusiness innovation. However, the ability to create valuefrom ADA is considered to have a relation to thematurity level of ADA capabilities [1], [35]. Thesecapabilities are not only about the technical aspects ofADA solutions, but they also encompass strategic,process, people, and process aspects [3], [21].The study by Huang et al. [36] indicates a directinfluence of the value generated from ADA on firmperformance, measured in two dimensions: operationaland market performance. Firm performance is definedas the extent to which a firm generates superiorperformance with respect to its competitors [4].Operational performance is about productivity, profitrate, return on investment, and sales revenue of anorganization [36]. Market performance is on the abilityof the firm to enter new markets and improve its positionin existing ones [4].In brief, the literature suggests an influence of theADA capabilities on various aspects of organizationalperformance. Hence, a positive relation between thematurity level of ADA capabilities - as specified by aspecific ADA maturity model- and the firm performancewould provide evidence for the validity of that maturitymodel. We will adapt this view for the evaluation of theADA-CMM that we propose in this study.3. ADA-CMM DevelopmentIn this section, we briefly present the researchprocess we have followed for the development of theADA-CMM.3.1. Initial model designAs the extant literature suggests a substantialnumber and type of ADA capabilities for organizations,we base our initial model on the capabilities proposed inthe literature. More specifically, the capabilitiesproposed by Brinch et al. [8] provided the foundation ofour model. We chose these capabilities as they are basedon a thorough review of the literature on the capabilitiesin various domains, such as big data analytics, advanceddata analytics, IT, and business process management.These capability areas are frequently identified in otherdata analytics maturity models1 and their relevance hasbeen validated through a case study [8]. The listidentifies the following 6 capability areas: IT, process,performance, human, strategic, and organizationalcapabilities.Although the study by Brinch et al. [8] providesdefinitions for ADA capability areas and relatedpractices, it does not propose a maturity model for them;hence, does not incorporate any structure or mechanismto assess the maturity level of these capabilities inorganizations. Therefore, we explicitly adopted amaturity level structure and accordingly definedmaturity level characteristics for all ADA capabilities.We adopted the maturity model structure of theProcess and Enterprise Maturity Model (PEMM) [37] asit allows for self-assessment and considered easy toapply in practice [38]. PEMM is a descriptive maturitymodel, which could also be used for prescriptivepurposes [13]. Moreover, it is a continuous model; thatis, it is based on scoring different dimensionsPage 268

(capabilities) at different levels and weighing theindividual scores [39].For the initial version of the ADA-CMM, we alignedthe ADA capabilities, as defined by Brinch et al. [8],with the structure of the PEMM. Accordingly, weincorporated 24 sub-elements (capabilities) categorizedunder 6 main elements (capability areas). Adopted fromPEMM, each sub-element is characterized by 4 maturitylevels. The structure of the ADA-CMM is depicted inFigure 1.Maturity Level Characteristics for Level 1 (Low)Sub‐element 1(Capability 1)Maturity Level Characteristics for Level 2(Moderate)Maturity Level Characteristics for Level 3 (High)Main element 1(Capability Area 1)Maturity Level Characteristics for Level 4 (Top)Sub‐element 2(Capability 2) Maturity Level Characteristics for Level 1 (Low) Figure 1. The structure of the ADA-CMM3.2. Model refinement with the Delphi studyDelphi study is a method for structuring a groupcommunication process. It is a method where a panel ofexperts evaluates the content of the developed artifact.It is executed in multiple rounds of questionnairesmoderated by a facilitator [40]. The experts do notdirectly face each other to prevent bias. We chose thismethod because it provides the opportunity to access abroad range of domain experts and combine their viewswhen there is a need to solve a practical problem [41].The panelists were selected based on theirexperience and knowledge of the ADA domain.Potential experts were contacted via email. Among 15experts that have been approached, 9 accepted theinvitation to take part in the panel. It is important to havea heterogeneous group of experts in terms of theirbackground, to reduce single culture bias and providediverse insights [42]. The panelist included 3academicians, 3 consultants, and 3 industry expertsworking in the ADA and related domains. Personalintroduction meetings were conducted to ensure theengagement of the participants, in which the objectivesof the study and responsibilities of the expert panel wereexplained.An ideal Delphi study involves 2 or 3 rounds, asmore rounds may result in a slower convergence amongexpert opinions [43]. We asked the experts to contributein 3 rounds through online questionnaires. In thequestionnaire, for each main element (capability area)and sub-element (capability), the experts were asked tochoose among 3 options: stay, change, or go. In caseparticipants chose the latter two options, they had toprovide reasoning about their decision. In the Delphirounds, we took percentage agreement as the measure ofthe level of consensus and set 80% as the threshold todecide whether the elements should stay [44]. If this wasnot the case, the element was changed or removed fromthe model and presented to the other panelists in thefollowing round.In Delphi round 1, we presented the participants thestructure of the model, main and sub-elements, and theirdefinitions. In this first round, the maturity levels foreach element were not presented. Two main points offeedback we received were on specifying the scope andfocusing the model on ADA more explicitly throughupdating various elements and sub-elements. Weexplicitly defined the scope of the model as big data andadvanced analytics projects related to businessprocesses, IT, and business analytics in organizations.Furthermore, various changes were introduced in thenames, definitions, or sub-elements of five of the sixelements.In Delphi round 2, the participants received a reportof changes in the initial model due to their feedback.They also received the revised model including this timethe definitions of the maturity level characteristics foreach sub-element (capability). The discussions led toseveral changes in all components of the model. Thisincluded name changes in the main elements, andvarious content related changes in sub-elements andcorresponding maturity level characteristics.In Delphi round 3, the goal was to evaluate thedescriptions of the maturity levels and confirm the mainelements, sub-elements and corresponding 4 maturitylevel characteristics for each maturity level. Accordingto the feedback, the model has been updated andfinalized. The final model as a result of round 3 wassent back to panelists to receive a final confirmation.4. ADA-CMM (Final Model)In this section, we briefly introduce the final versionof the ADA-CMM. Table 1 presents an overview of themain elements and sub-elements of ADA-CMM. Eachmain element encompasses a set of sub-elements thatrepresent a distinct capability.The element People & Culture considers theknowledge and commitment of employees regardingADA, the diversification of teams, and the adoption ofanalytical capabilities to improve business processes.The Performance & Value focuses on the performancemetrics that show how ADA capabilities can turn datainto value and innovation processes to develop best-inclass service operations. The Strategy includescapabilities related to the definition of ADA vision,mission, and objectives, and the linkage of ADA to ITand business process priorities. The Data & Governancerelates to the data architecture, elimination of repetitivemanual work, the linkage of IT systems and operationalprocesses, data governance, and the available dataanalytics tools. Finally, the Process Design &Page 269

Collaboration emphasizes on the capabilities regardingthe competence and skill development of employees, theway they are informed about new technologies, howADA projects are managed, and the extent ofinformation sharing and functional project involvement.Table 1. ADA-CMM main and sub-elementsMain Element(Capability Area)People & CulturePerformance & ValueStrategyData & GovernanceProcess Design itmentTeam DiversityUsagePerformance MetricsInnovation ProcessesADA StrategyStrategic AlignmentData ArchitectureAutomationData IntegrationData GovernanceData Analytics ToolsCompetence & Skills DevelopmentCommunicationPortfolio ManagementOrganizational CollaborationADA-CMM further includes the definition of stics in 4 levels. There are 17 sub-elementsrelated to a specific element, representing anorganizational capability necessary to create value fromADA. We provide an excerpt of ADA-CMM as anexample in Figure 2 and provide the complete model inthe online report2. We built the descriptions of subelements based on the main sources we used for ADAcapabilities ([8], [37]), and enriched them with othersources in the literature where possible.Each sub-element is characterized by four maturitylevels and related characteristics similar to PEMM [37](low, moderate, high, and top levels). For each subelement, an organization can be at a different level (asexemplified in Figure 2) and weigh the individual scoresinto an average maturity score per main element.Organizations can use ADA-CMM to assess thecurrent situation, develop and prioritize improvements,and control the progress of implementation [13]. It canbe used as a self-assessment tool of the ADAcapabilities. To gain a more reliable self-assessment, itis ideal that the assessment is conducted with multipleparticipants with different organizational roles,backgrounds and motivations [45]. The assessment cantake place in a focus group or workshop setting whereeveryone can express their opinion and have a groupdiscussion about each element until there is a consensus.Alternatively, it can be performed as an online survey,2where every participant individually expresses theiropinion, and the results are aggregated.The self-assessment represents the current situationand unveils the areas of ADA capabilities in which theorganization excels, and which areas have room forimprovement. The gap between the current situation andthe desired situation can help prioritize improvements inthe ADA capabilities. Conducting a regular selfassessment can facilitate monitoring the progress ofimprovements. The target group of ADA-CMM is theorganizations using or planning to implement ADAwithin their organization. Finally, the ADA-CMM canbe used to benchmark and identify the organizationalADA performance compared to the industry standard.Figure 2. An excerpt from ADA-CMM, containingan example sub-element and maturity levelcharacteristics for the Strategy and Data &Governance elements5. EvaluationAlthough ADA-CMM has been developed as a jointeffort of domain experts through a Delphi study, itshould be evaluated in real-life business settings. For itsevaluation, we focused on its validity [13], i.e., that itcan be used for its intended purpose of use. Accordingly,we proposed the research model depicted in Figure 3.We presumed that the maturity level of ADAcapabilities of an organization impacts the firmperformance through the value generation potential ofADA (ADA value) as a mediator. We expect amediation effect as we presume that the increased firmperformance can materialize only when ADAcapabilities are put in action and value is generated fromADA initiatives.The complete ADA-CMM is available at: https://bit.ly/35qQIW6Page 270

ADACapabilityMaturityMarketPerformanceADA ValueOperationalPerformanceFigure 3. Research model5.1. Evaluation MethodTo evaluate the validity of ADA-CMM, we designedan online survey to collect data about the maturity levelof ADA capabilities of organizations (as assessed usingADA-CMM), the value creation aptitude of ADA (ADAValue), and the firm performance (as characterized in 2forms: market and operational performance).The survey consisted of four sections3. The firstsection evaluated the maturity level of ADA capabilitiesof organizations as assessed by ADA-CMM. Thissection consisted of 17 questions, each referring to oneof the sub-elements of ADA-CMM. The definitions ofthe sub-elements were rephrased such that they becamea question. Each question had a 4-point Likert scale,ranging from 1 (to a small extent) to 4 (to a very greatextent) corresponding to the 4 maturity levels andaligned with our maturity model structure [37].The second part of the survey focused on the valuecreation aptitude of ADA in a firm. Based on theframework of Elia et al. [34], we defined fivedimensions to measure the value created by ADA in rmational, strategic, and infrastructural value.The statements had a 5-point Likert scale ranging from1 (strongly disagree) to 5 (strongly agree).The third part of the survey referred to the firmperformance, which consisted of four items related tomarket performance and four related to the operationalperformance. These questions are based on the validateditems as given in [4]. The statements had a 5-pointLikert scale ranging from 1 (strongly disagree) to 5(strongly agree) and included an option of “don't know/ not applicable”.The final part of the survey aimed at collectinggeneral information about the survey participant andorganization that is being assessed. The participantswere asked questions on the country of their residence,age, work position, general work experience, andexperience with ADA, and the sector and size of theorganization. The name of the organization andparticipant were not mandatory fields for enablinganonymity if preferred. The answer options of the sectorwere classified using the GCIS scheme.A pilot survey was sent out to several respondents toreview the questions and completion time. The survey3was distributed via social media and specificallytargeted emails. The target group was informationmanagers. The survey was online for three weeks, andthe target group was actively stimulated to fill in thesurvey by sending out invitations and reminders.5.2. DescriptivesIn total 56 participants responded to the survey. Weremoved one response due to straight-lining and tworesponses due to missing data. Our univariate andmultivariate outlier analysis did not result in the removalof any data. We aggregated multiple data points thatcorrespond to participants from the same organization toreach a firm-level result. After data cleaning andaggregation, our data set consisted of 48 observations.The descriptive information on the work and ADAexperience of the participants and their organizationalsize are shown in Table 2. Participants were mostly fromthe sectors of industrials (e.g., capital goods,transportation) (29%), IT (26%), financials (14%),consumer staples (10%), and healthcare (7%). Figure 4depicts the boxplots for the aggregated maturity scoresper element of the ADA-CMM (D&G: Data &Governance, P&V: Performance & Value, S: Strategy,P&C: People & Culture, PD&C: Process Design &Collaboration), the overall score for ADA value, and theoverall performance score aggregated for eachorganization.Table 2. An overview of descriptive statisticsPersonalexperience (years)WorkADAOrganization size(employee #)017% 107%0-22-55-10 1028%21%115029%35%24%5125019%24%29%251100021%13%9% 100024%Figure 4. Box plots of the survey results permaturity level of ADA-CMM eleme

The Analytics Maturity Model developed by The Data Warehousing Institute is a tool to assess a firm's big data analytics governance, organization, data management, analytics, and infrastructure in five maturity stages, with a chasm between stage 3 and 4 [23]. The Big Data Business Model Maturity Index assesses a BDA