Transcription
MANAGEMENTS T R AT E G YMEASUREMENTM A N AG E M E N T AC C O U N T I N G G U I D E L I N EEvaluatingPerformance inInformationTechnologyByMarc J. EpsteinandAdriana RejcPublished by The Society of Management Accountants of Canada, the AmericanInstitute of Certified Public Accountants and The Chartered Institute ofManagement Accountants.
N OT I C E TO R E A D E R SThe material contained in the Management Accounting Guideline Evaluating Performance in Information Technology is designedto provide illustrative information with respect to the subject matter covered. It does not establish standards or preferredpractices. This material has not been considered or acted upon by any senior or technical committees or the board of directorsof either the AICPA, CIMA or The Society of Management Accountants of Canada and does not represent an official opinionor position of either the AICPA, CIMA or The Society of Management Accountants of Canada.Copyright 2005 by The Society of Management Accountants of Canada (CMA Canada), the American Institute of CertifiedPublic Accountants, Inc. (AICPA) and The Chartered Institute of Management Accountants (CIMA). All Rights Reserved.No part of this publication may be reproduced, stored in a retrieval system or transmitted, in any form or by any means,without the prior written consent of the publisher or a licence from The Canadian Copyright Licensing Agency (AccessCopyright). For an Access Copyright Licence, visit www.accesscopyright.ca or call toll free to 1-800-893-5777.ISBN: 1-55302-173-8
S T R AT E G YE VA L U AT I N G P E R F O R M A N C EI N I N F O R M AT I O N T E C H N O L O G YINTRODUCTIONWith respect to the first, doubts aboutthe potential payoffs of IT investmentscan be traced to numerous IT projectsmade without the rigor of measurementof either the benefits or costs of suchinvestments. Decisions were made basedon compelling arguments and keeping upwith competitors resulting in billions ofdollars of wasted corporate assets.France Telecom, for example, announcedthat they spent 700 million euros onexternal IT services in 2003 which is afterslashing 138 million euros from the ITspending of the prior year (838 millioneuros). Almost every organization hasstories about unfulfilled promises aboutthe benefits of new ERP or IT systems.In the United States alone, annualexpenditure on IT now runs into trillionsof dollars, and approaches 50 percent ofnew capital investment for mostorganizations with little evidence toCONTENTSEXECUTIVE SUMMARYWhile some of the recent publications oninformation technology question thevalue creating role of IT in today’sbusiness environments (Carr, 2003 and2004), others assert that, with theeconomic recovery in many parts of theworld, innovation — especially ininformation technology — is becomingeven more critical to growth and highperformance.PageINTRODUCTIONOBJECTIVES AND TARGET AUDIENCEENSURING ACCOUNTABILITY ININFORMATION TECHNOLOGYIDENTIFYING THE OBJECTIVESAND DRIVERS OFSUCCESSFUL INFORMATIONTECHNOLOGY INVESTMENTSDEVELOPING THE APPROPRIATE METRICSFURTHER SPECIFICITY ON CALCULATINGBENEFITS (OUTPUTS) FROM ITINVESTMENTSMONITORING THE STRATEGICIT OBJECTIVES, DRIVERS ANDPERFORMANCE MEASURESMEASURING THE DISRUPTION COSTSOF IT INITIATIVESMANAGING IT INVESTMENT PROCESSESRECOGNIZING THE RISK ASSOCIATEDWITH IT INVESTMENTSALIGNMENT OF THE IT CONTRIBUTIONMODEL WITH THE BALANCEDSCORECARD, SHAREHOLDERANALYSIS AND ROITHE APPLICABILITY OF THE ITCONTRIBUTION MODEL ANDMEASURES TO OTHERBUSINESS FUNCTIONSGUIDANCE FOR MANAGERSBIBLIOGRAPHY355615202323242526303135Though there has been significantdiscussion concerning the importance ofevaluating the payoffs of IT investment,there has been little guidance as to howto design or implement an appropriateperformance evaluation system. Thus,investments are often made without therigor of measurement of either thebenefits or costs of such investments.Typically costs are much higher thananticipated and the benefits are far lowerand harder to achieve.Senior IT managers are also frustrated.They are convinced that, if measuredproperly, IT does create value.This Guideline will develop an ITperformance measurement framework,articulate specific measures, describe thecausal relationship between variousdrivers and measures, and throughexamples, illustrate how companies canidentify and measure the payoffs of ITinvestments.3
MANAGEMENTS T R AT E G YMEASUREMENTsuggest that this expenditure has generated asatisfactory return (Murphy, 2002, Davenportand Prusak, 1997). Typically, the costs oftechnology are much higher than anticipated,the cost of conversion is also higher, whereasthe benefits are far lower and harder toachieve than expected. In addition, this ignoresthe very significant costs related to employeetime wasted and the disruption to personnel,operations, and the revenue stream of theorganization. While the claim that someorganizations collapsed because of ineffectiveIT policies may be an exaggeration, many ChiefExecutive Officers (CEOs) and business unitleaders view IT as a value destroyer or a costrather than a value creator implying itscorroding impact on the organization’scompetitive advantage.On the other hand, the advocates of the boldand comprehensive new vision of howorganizations can use information technologyto create value believe IT matters more thanever, yet in a different way. By moving from anera of technology to an era of technologycapability, the focus has shifted from individualtechnologies to the benefits that can becreated with them. Because of this focus ontechnology capabilities, innovation is emergingnot just from technologists, but from the usersof the technology components whounderstand how to use IT to deliver higherlevels of organizational performance. By fillingthe gap between the rate of technologyinnovation and people’s understanding andability to use and implement the technology,organizations can use these innovations to leadto substantial improvements in theirperformance. Even for organizations that wereactually quietly making a big difference in theirmarkets by leveraging IT, it was still oftendifficult for these Chief Information Officers(CIOs) to quantify those results, and prove thebenefits. Then, when earnings declined, IT wasan easy target for cost cutting.A primary reason for doubts about thepotential value organizations can derive fromexisting and future investments in IT is relatedto the absence of a proper methodology toevaluate the payoffs of IT investments. So far,there has been little guidance of how to designor implement an appropriate IT performanceevaluation system, i.e. how to identify anddocument the contribution of informationtechnology to high-performance organizations.Historically, organizations were driven by4enthusiastic managers who were over relyingon technology and did not demanddevelopment of the needed skills and themeasures to complete these analyses. Today,the financial managers and other decisionmakers want the IT requests to be framed in aROI or shareholder value format so that theycan be effectively compared with alternativepotential company investments. According tothe Forrester Report, 90% of executives makethe IT funding decisions based on the financialimpact of IT initiatives; however, the topchallenge in selecting which IT project to fundis the lack of objective data (Cameron et al.,2000). Senior IT managers are convinced thatthey do create value and believe that, ifmeasured properly and with adequate support,they would be significant profit centers fortheir organizations. But without adequateperformance evaluation systems, they havedifficulties proving the value adding role of ITand find themselves continually fighting for andjustifying the resources that are needed. CEOsand CFOs lack information to make wellinformed decisions on the payoffs of theseinvestments and, as a consequence, corporategoals seem to focus on reduction of the costsof IT rather than maximizing IT value creationactivities.OBJECTIVES AND TARGETAUDIENCEAs IT managers must show the payoffs of ITinvestment to convince key executives thatthey should be strong supporters of IT efforts,a framework for evaluation of IT performanceis a significant need. Few things are moreconvincing to top executives than measurableresults. When a new project is proposed,additional funding is typically based solely onthe results anticipated from the project.Currently, IT executives do not have propertools to measure the payoffs of IT. Evenfinancial managers that have expertise inmanagement control and performancemeasurement, have not focused on thebenefits of IT and have not developed theappropriate measures. Consequently, thepayoffs of IT are not measured, ROI is notcalculated, and IT investments are notevaluated with the same rigor as othercorporate investments. The purpose of thisguideline is to provide a model and a selectionof measures for evaluating performance ininformation technology in both for-profit and
E VA L U AT I N G P E R F O R M A N C E I N I Tnot-for-profit organizations to help CIOs betterjustify and evaluate their initiatives and aid CEOsand CFOs in making better resource allocationdecisions.This Management Accounting Guideline’sobjectives are as follows:To develop a general model of key factorsfor organizational success in IT integration (ITContribution Model) that includes fourdimensions: the critical inputs and processesthat lead to success in IT outputs andultimately to overall organizational success(outcome). To articulate each of the key factors(antecedents and consequences of ITsuccess) as objectives to facilitate furtheroperationalization of the model. To outline the specific drivers of IT successbased on the objectives related to inputs,processes, outputs, and outcomes, andidentify the causal relationships between thedrivers. To provide the specific measures of ITperformance. Following the cause-and-effectrelationships between the drivers of ITsuccess, measures are developed to trackperformance of IT initiatives along the fourdimensions. The metrics can be used forboth IT project justification prior to its start(planning) as well as for evaluation aftercompletion (performance measurement). To provide examples of how to assignmonetary values to non-financial IT outputs(benefits). Although some benefits do notalways easily translate into short term profits,they should ultimately lead to either costsavings or increased revenues. To provide an example of how to calculatethe IT payoffs. Here, the guideline specificallyrecognizes the importance of measuring boththe total costs of an IT initiative — includinga range of different disruption costs — aswell as the benefits, and additionallyconsiders the risks associated with ITinvestments. To show how the IT Contribution Model isconsistent with other measurementframeworks such as the Balanced Scorecardand Shareholder Value Analysis.The target audience of this guideline is theaccounting and finance professionals that dealwith the challenges of performancemeasurement and control in IT. Presented in a systematic format, the guideline is also intendedto help CIOs, Chief Training Officers (CTOs)and senior IT managers better understand howinformation technology contributes to higherlevels of corporate performance, more easilyevaluate the profitability of IT investments, andmake better resource allocation decisions. Inaddition, it is also helpful for the CEOs, CFOs,and other decision makers that struggle toidentify, document, measure, and communicatethe short-term results and long-term impacts ofIT investments. This includes both cost savingsand value creation, and thus provides argumentsfor additional IT resources when appropriate.ENSURING ACCOUNTABILITY ININFORMATION TECHNOLOGYTypical large IT investments, E-commerceinvestments, and very large ERP systemimplementations are all faced with the samechallenges of demonstrating the value of theinvestment and historical difficulties in estimatingboth the revenues and total costs. This typicallyfalls on senior corporate and senior financialmanagers to evaluate the payoffs and makerecommendations on resource allocation —which are typically significant. But it is not onlygrowing budgets that make IT an importantexpenditure. The consequences of improper andineffective IT initiatives can be significant if weconsider the integration of the IT departmentinto the mainstream functions of theorganization. Also, IT infrastructure upgradesprovide vast potential to contain costs, enhanceproductivity, improve quality, spur innovation,and satisfy and retain customers.With CEOs and CFOs demanding accountabilityfor the tremendous investment in informationtechnology, IT managers are required to ensureaccountability, calculate the return oninvestment, develop a value-added approach,and make a bottom-line contribution. At theheart of accountability is measurement andevaluation. IT managers faced with the challengeto demonstrate the validity of IT initiatives tothe business must find ways to measure andcommunicate the contribution of IT so thatviable existing initiatives are managedappropriately, new projects are only approvedwhere there is satisfactory return, and marginalor ineffective projects are revised or eliminated.To properly assess the payoffs of investments inIT, organizations must implementcomprehensive systems to evaluate the impactof IT initiatives on financial performance and the5
MANAGEMENTS T R AT E G YMEASUREMENTtrade-offs that ultimately must be made whenthere are competing organizational constraintsand numerous barriers to implementation.However, with larger and more complex ITsolutions and with business process changestypically enabled by or driven by informationtechnology, IT projects should be viewed asbusiness change projects with IT componentsrather than stand-alone IT projects. That is,change management constitutes an importantpart of IT projects and the business value ofany IT investment must be clearly stated.Business value is created through acombination of IT tools and businessprocesses, thus, there must be a cleardistinction between IT and businessaccountabilities. IT managers are accountableprimarily for successfully delivering theappropriate technologies, infrastructure, andtechnical support, while accountability fordelivering business results rests jointly with theIT function, the relevant business managers,and performance measurement professionals.The benefit realization process usuallytranscends departmental boundaries, andwhen designing an appropriate IT evaluationmodel, this should be considered.IDENTIFYING THE OBJECTIVESAND DRIVERS OF SUCCESSFULINFORMATION TECHNOLOGYINVESTMENTSThe identification and measurement of theimpacts of IT investments are particularlydifficult as they are often linked to long timehorizons, a high level of uncertainty, andimpacts that cannot be easily quantified. But,measurement and evaluation systems can besimplified and implemented with manageablecost through early planning, and a change inphilosophy and attitude of both the IT staff andthose it serves.In recent years, organizations have placedincreasing importance on the development ofperformance metrics to better measure andmanage IT programs and projects; however,few specific metrics have been proposed. Withthe high costs that are often associated with ITinvestments and the seemingly smallpercentage of IT projects that pay off, theimpression is often that the projects areflawed whereas it may be that it is theperformance measures that are flawed.Worse, the lack of performance metrics may6have also led in part to the lack of both actualand perceived accountability for IT operationsto its various stakeholders.New IT performance measurement modelsThere have been some recent attempts tomeasure the value of IT. One authordeveloped a ratio called InformationProductivity (IP) which is simply the ratio ofthe Economic Value-Added (EVA) to the totalcost of information management (Strassmann,1999). With information technology being oneof the fastest growing components of the costof information management this metric isdesigned to reflect an organization’s success atconverting the cost of information managementinto profit.Another proposal is to expand conventionalfinancial measurement, like return oninvestment and payback period, to aneBusiness context which is a whole-viewmeasurement of business performance acrossboth internal and external constituents(Cameron et al., 2000). By setting weightedeBusiness objectives relating to end-customersuccess, hyper-partnering efficiency, and multiorganization financial performance andapplying quantitative and qualitative impactmetrics, organizations can track a project’simpact on a given eBusiness objective.In yet another approach, Intel has developed aBusiness Value Index (BVI) (Intel, 2003b,Curley, 2004). BVI is a component index offactors that impact the value of an ITinvestment. It evaluates IT investments alongthree vectors: IT business value, impact on theIT efficiency, and the financial attractiveness ofthe investment. All three factors use apredetermined set of defining criteria thatincludes customer need, business and technicalrisks, strategic fit, revenue potential, level ofrequired investment, and the amount ofinnovation and learning generated. Eachcriterion is weighted, and project managers orprogram owners score their projects againstthese criteria to produce total scores for eachof the three vectors. By graphically depicting thethree indices for each project, BVI methodologyprovides some decision support to managers tocompare and contrast investments, and thendetermine the investments that align best withtheir business priorities.Though all of these approaches are helpful,specific tools for identification and
E VA L U AT I N G P E R F O R M A N C E I N I Tmeasurement are necessary. (Additional relevantapproaches can be found in Tardugno et al. 2000,Remenyi et al. 2000, Murphy 2000, Devaraj andKohli 2002, Lutchen 2004, Weill and Ross 2004,Curley 2004 and Schubert 2004). This guidelineattempts to provide a format so thatorganizations can identify and measure the costsand benefits of IT. It provides a useful model andspecific metrics that will help organizationsmeasure the inputs, and processes, as well as thebenefits and value resulting from IT initiatives. Asmost contemporary approaches to performancemeasurement emphasize the alignment betweenstrategic objectives and measures — since it isdifficult to measure performance unless it is clearwhat an organization is trying to achieve — webegin with the necessary alignment betweeninformation technology and the corporatestrategy.Designing the IT Contribution Model —convergence of corporate strategy andtechnologyInformation technology increasingly serves as theengine of innovation and the foundation ofgrowth for organizations worldwide. Virtually noorganization can generate new products andservices, manage its investments, communicatewith suppliers, buyers, or employees, and extendits markets without IT. Business and IT areinseparable, with few decisions resting entirely inthe domain of either business or IT. Thedecision-making process surrounding ITinvestments, thus, must seek to expand ITcapabilities to satisfy future business needs aswell as current needs. Investment decisions mustconsider the strategic perspective if they are tobe effective. In fact, the alignment between thecorporate and IT strategy is crucial for tworeasons:A proper incorporation of IT strategy into thecorporate strategic plan allows for sufficientcapital and human resources to fund ITprojects and programs; Carefully planned and implemented ITinitiatives significantly contribute to successfulexecution of the corporate strategy.The strategic alignment thus refers to thealignment of IT investment strategy with theachievement of the organization’s strategic goalsand objectives. The question to be asked is ‘Willthis IT investment help us ac
information technology question the value creating role of IT in today’s business environments (Carr, 2003 and 2004), others assert that, with the economic recovery in many parts of the world, innovation — especially in information technology — is becoming even more critical to g
