WIXLIB

The Leader inConfiguration Audit & ControlVisible Ops and Foundational Controls:An Eight Year Study Of High PerformingIT OrganizationsGene Kim, CISACTO, Tripwire, Inc.10/18/2007

Introductory Questions2 What about your job causes you to feel uncomfortable? In your interactions with your business peers andmanagement, what situations don’t feel right to you?

Agenda Background on Visible Ops and high performing ITorganizations Review 2006 IT Controls Performance Study Two key surprises on performance and controls 21 foundational controls that had majority of impact on ITperformance Expanded benchmark to 350 IT organizations in 20073 Present findings from 2007 IT Controls PerformanceStudy What have we learned about metrics that actually matter? How to improve IT performance in 90 days

The Highest Performing IT Organizations Get ResultsOperations Metrics Benchmarks:Best in Class: Server/SysAdmin Ratios Highest ratio of staff10,000for pre-productionprocesses Lowest amount ofSize of Operation# Servers100010010Best in ClassOps and Securityunplanned work Highest changesuccess rate Best posture ofcompliance Lowest cost ofEfficiency of Operationcompliance10204Source: www.itpi.org406080100Server/SysAdmin Ratio120140

Common Traits of the Highest PerformersCulture of Change management Integration of IT operations/security via problem/change managementProcesses that serve both organizational needs and business objectivesHighest rate of effective changeCausality Highest service levels (MTTR, MTBF)Highest first fix rate (unneeded rework)Compliance and continual reduction ofoperational variance 5Production configurationsHighest level of pre-production staffingEffective pre-production controlsEffective pairing of preventive and detective controlsSource: IT Process Institute

Seven Habits of Highly Effective IT OrganizationsThey 1.Have a culture that embraces change management.2.Monitor, audit, and document all changes to the infrastructure.3.Have zero tolerance for unauthorized changes.4.Have specific, defined consequences for unauthorized changes.5.Test all changes in a preproduction environment before implementinginto production.6.Ensure preproduction environment matches production environment.7.Track and analyze change successes and failures to make futurechange decisions. 6All high performers have created Cultures of Change Management Causality Planned Work

Visible Ops: Playbook of High Performers The IT Process Institute has beenstudying high-performingorganizations since 1999 What is common to all the highperformers? What is different between them andaverage and low performers? How did they become great?7 Answers have been codified in theVisible Ops Methodology The “Visible Ops Handbook” is nowavailable from the ITPIwww.ITPI.org

Visible Ops: Four Steps To Build An Effective ChangeManagement ProcessService Design & ManagementSecurity ManagementService Level ManagementCapacity ManagementAvailability & ContingencyManagementService ReportingFinancial ManagementControl ProcessesAsset & Configuration ManagementChange ManagementPhase 3: EstablishRepeatable BuildLibraryRelease ProcessesRelease ManagementSupplier ProcessesResolution ProcessesIncident ManagementProblem ManagementCustomer RelationshipManagementSupplier ManagementAutomationTripwire captures known goodstate in preproduction.Tripwire captures productionchanges that need to bebaked into the build.Phase 4:Continually improveTripwire detects change,which all process areas hingeupon.8Phase 1: ElectrifyFence, Modify FirstResponseTripwire enforces the changeprocess.Tripwire rules out change asearly as possible in the repaircycle.Source: ITPI Visible OpsSource: IT Infrastructure Library (ITIL) / BS 15000Phase 2: Catchand Release, FindFragile ArtifactsTripwire protects fragileartifacts.Tripwire enforces changefreeze and preventsconfiguration drift.

ITPI Survey: Demographics9IT EmployeesIT BudgetAverage483 114 millionMin3 5 millionMax7,000 1,050 million

Surprise #1: How Good The High Performers Are High performers contribute more to the business 8 times more projects and IT services 6 times more applications When high performers implement changes 14 times more changes One-half the change failure rate One-quarter the first fix failure rate When high performers manage IT resources One-third the amount of unplanned work 5 times higher server/sysadmin ratios When high performers are audited Fewest number of repeat audit findings10Source: IT Process Institute, May 2006High performers also have3x higher budgets, asmeasured by IT operatingexpense as a function ofrevenue

And Security High Performance, Too When top performers have a security breach Loss events are 29% less likely than in medium performers, and84% less likely as low performers Failure to detect of the security breach by an automated control is60% less likely than medium performers, and 79% less likely thanlow performers Time to detect is minutes for top performers, hours for mediumperformers, and days for low performers 11Top performers also allocate 3x more budget to security,as a function as IT operational expense

Surprise #2: What The High Performers DoDifferentlyTop Two Differentiators between Good and Great1. Systems are monitored for unauthorized changes2. Consequences are defined for intentional unauthorized changesFoundational Controls:Foundational Controls:Medium vs LowHigh vs Medium12Source: IT Process Institute, May 2006

Design Survey: Pick IT ControlsService Design & ManagementSecurity ManagementService Level ManagementCapacity ManagementAvailability & ContingencyManagementService ReportingFinancial Management1 We selected the 6 leading BS15000areas within ITIL that are conjecturedto be “where to start.”These were Access, Change,Resolution, Configuration,Release, Service LevelsControl ProcessesAsset & Configuration ManagementChange ManagementRelease ProcessesRelease ManagementSupplier ProcessesResolution ProcessesIncident ManagementProblem ManagementCustomer RelationshipManagementSupplier ManagementAutomationSource: IT Infrastructure Library (ITIL) / BS150002 We then selected 63 COBIT controlobjectives within these areas.13Source: COBIT, IT Governance Institute/ISACA

The 63 IT ControlsAccessChangeConfiguration ReleaseService LevelResolutionDo you have aformal process forrequesting,establishing, andissuing useraccounts?Do you have aformal IT changemanagementprocess?Do you have aformal process forIT configurationmanagement?Do you have astandardizedprocess for buildingsoftware releases?Do you have adefined process formanagingincidents?Do you use tools toautomate therequest, approval,tracking, and reviewof changes?Do you have anautomated processfor configurationmanagement?Do you use tools toautomate the buildof new releases ofsoftwareapplications?Do you havesomeone (a servicelevel manager) whois responsible formonitoring andreporting on theachievement of thespecified serviceperformancecriteria?Do you have anautomated means ofmapping useraccounts to anauthorized user?For eachemployee/resource,do you record a listof system accessrights?Do you audit useraccounts to ensurethat they map to anauthorizedemployee?Do you haveprocedures to keepauthentication andaccess mechanismseffective?Do you have aformal process forsuspending andclosing useraccounts?Do you haveprocesses forgranting andrevoking emergencyaccess to relevantstaff?Do IT personnelhave well-definedroles andresponsibilities?Do you have anautomated processfor defining andenforcing useraccount roles?Do user accountsever allow actionsthat exceed theirspecified role?Do you monitoraccounts to detectwhen they exceedtheir specified role?Do you track yourchange successrate?Do you track thenumber ofauthorized changesimplemented in agiven period?Do you track howmany changes aredenied the first timethey are consideredby the changeauthority?Do you monitorsystems forunauthorizedchanges?Are their definedconsequences forintentionalunauthorizedchanges?Do you have achange advisoryboard orcommittee?Do you have achange emergencycommittee?Do you use changesuccess rateinformation to avertpotentially riskychanges?Do you distribute aforward schedule ofchanges to relevantpersonnel?Do you conductregular audits ofsuccessful,unsuccessful, andunauthorizedchanges?Do you rigorouslyenforce separation14of duties betweenAre changesthoroughly testedDo you have aconfigurationmanagementdatabase (CMDB)?Does the CMDBdescriberelationships anddependenciesbetween theconfiguration items(infrastructurecomponents)?Does yourconfigurationmanagementdatabase specify towhich businessservice eachconfiguration itemsupports?Are you able toprovide relevantpersonnel withcorrect and accurateinformation on thepresent ITinfrastructureconfigurations,including theirphysical andfunctionalspecifications?Do you monitor andrecord the time ittakes to correctconfigurationvariance?Do you useautomated softwaredistribution tools?Do you test allreleases beforerollout to a liveenvironment?For release testingpurposes, do youmaintain anidentical testingenvironment to yourproductionenvironment?Do you have adefinitive softwarelibrary (DSL)?Do you have aservice catalog?Do you regularlyreview your servicecatalog?Do you regularlyreview service levelagreements?Do you have aserviceimprovementprogramme?Do you everrenegotiate thedefinedconsequences inthe service levelagreement?Do you have aformal process todefine servicelevels?Does your servicelevel agreementcover ALL of thefollowing h capacity,levels of usersupport, continuityplanning, security,and minimum levelof systemfunctionality?Do you have anautomated processfor managingincidents?Do you track thepercentage ofincidents that arefixed on the firstattempt (first fixrate)?Do you use aknowledge databaseof known errors andproblems to resolveincidents?During an incident,do you ever rebuildrather than repair?Do you have adefined process formanagingproblems?Do you have anautomated processfor managingproblems?Do you follow astructured methodfor analyzing anddiagnosingproblems?Do you have adefined process formanaging knownerrors?Do you proactivelyidentify problemsand known errorsbefore incidentsoccur?Is there integrationbetween yourproblemmanagement andchangemanagementprocesses?Is there integrationbetween yourproblemmanagement andconfigurationmanagementThe resulting controls that weselected were in the followingcontrol categories: Access Controls: 17 controls Change Controls: 13 controls Configuration Controls: 7controls Release Controls: 6 controls Service Level Controls: 8controls Resolution Controls: 12controls