Transcription
Presentation to the City’s Audit Committeeon October 25, 2019, Agenda Item AU4.4Review of Toronto Transit Commission’s (TTC)Revenue Operations: Phase TwoPRESTO/TTC Fare Equipment and PRESTO RevenueBeverly Romeo-Beehler, FCPA, FCMA, B.B.A., JD, ICD.D, CFFAuditor GeneralTara Anderson, CPA, CA, CIADirectorTeam: Thusha Anandarajan, Saba Mohammed,Serena Noh, Laura Wright1
TTC Phase 2 Audit - AgendaIntroduction1. Complexity2. Issues Identified Equipment, AvailabilityCalculation & Control gaps3. Moving forward2
IntroductionIn addition to fare evasion, the following canalso result in passenger revenue loss to theTTC:1.Malfunctioning fare equipment 2.PRESTO card readers (bus, streetcar)TTC subway fare gates (stuck in open position)PRESTO vending machines on new streetcars(2 types)Revenue transactions not capturedproperly by PRESTO’s back-end systems3
IntroductionAudit Phase 1Focus was on: Fare evasion andthe estimatedloss ofpassengerrevenueEffectivenessand efficiency offare inspectionprogramAudit Phase 2Focus was on whether TTC isreceiving all the PRESTOrevenue it should, including areview of the following: Functionality of PRESTO andTTC fare equipment TTC’s contract with Metrolinx Capturing all PRESTOrevenue transactions on TTCthrough PRESTO’s back-endsystems4
TTC is a key clientPRESTO Boardings: June 201839,077K as of June 2019,according to TTC5
1. Complexity6
Flow of PRESTO Revenue Transactions from Device toPRESTO’s Systems to TTC’s Bank Account/PRESTO/central Sys(amMoney gets deposited i n TTC .srevenue b a n k account:::: ---to oa;1yS ett.l ernentM onthly, e c.onci 11at1oncon-1p 1etea byPRESTOaooTTCSubSy.stewns. toCentxal SystemM anualrecon ciliauon iSoone by PRESTObetw e,ensubsystems andcen tral systemTransactionsw nich 1000.unesuan.sacttonsmissing 1n thecen tral systemfrom subsystemsFareGo 3 2 Sobsy.stem(PRESTO hoSlS ana su pponson r.eha rr or TIC)FareGo3 5Precise PairklLinkSuD nd EventsControl Gap tofrom Pl-ecisee nsu re moneyPa.rkLinks ubs:ystenl donot ffl.2f.Ge n: 1rnoT ransactions andEvents&et I n the bank d e v ice lev e l datanot o btai n ed andis pureed after 7days. noPRESTOoemntlsysmm TTCge,sthe money at'themonlhencLrec:onc11,at ,onMOOi Point o1'S.les(M POS)andStation Point ol'sa tes(SPOS) IBYICEIS(l.lSea by retarterb e t'Ween dev i cesand subsystems 3416 rep0rt d e not fully addr ess. - 't"'lh is r 1srk,' .network f orPREST O sal es)SW/PRE-Va-Fa r-e G.atesM aCt'l tnes.'-.:i ·Devk'::eLevel DataParkeon V endingM a.Ch ines./PREiizE}! ·. ·· ci ' . ·.- !. - . 'Auditor General's Office - Integrity, ExcellenceBuses and Streetc arsand Innovation7
Example of complexity – one typeof fare equipment (fare gates)8
Example of Complexity“Fare gate activities are monitoredusing FareGo 3.2, which is an S&Bsoftware system, currently hostedby Telus, on behalf of Accenture, avendor of PRESTO”(page 71 of AGO report, TTC Phase 2 RevenueOperations Audit)9
Governance and Contractual RelationshipsCard reader a '&il8ib II)' calcul&oooVe ndor A---i & Jb-Vendor AC&r I re&CJer mon1t or1ns tootLavailability cak:uletian[Parl eon \lendlns mac54.Jb-V oodor B )r·····- ; - - ---·····::::: e METROLI NX lPRESTO/PRESTO RevenueSecban E[VendorDVendor EVendorF;:. . . . . 0 . . 1101 I I i 101 i I i 111 IFare G1111:es secuoo D A.1.4. B.1.2 A.'2liost data ooncentratar DetYrork forfare gac.es ancl 111eooor1 connectM tYfor all Pf?ESTO oe,,.,1ces- .[[A .1,B .1.1Determination af in-sarv,ce ,.-ehiclesPRESTO d a ."JCe manitaring))A.2. D.2t- orl c onnectiOII rrom fareD .'2gi;ite ta data c ancen11ratorColn &llll to 1 .e11 00'".lectJon rromvending m e dmnes an neviA.2. B 2streetcarsP RESTO sales c hannelJE .4l I 011 1 , , . :W'etroi,1\1( v endlns maon- 11eevailab Tl)' c e lcu5auonVendor G; . .B.2 . A.2Coou-act:ed maintenance for fa re gateSFare g;rte .avarlabchty calculationC.5Schecme GcNemanc:e Structure( lPRESTO/( ;.v;( CCTranspo( (I UP :ID) 8.] ti] ti1 tl l· · s{II2lim' . J[]JC.I,J/,11-TTC \lendorsITC & PRESTO Vendo.raPRESTO SUl - rsContractual Relatians.h!pVoting Mem ber -PRESlO/Metrdlnx and A.ccenturevendorssecoooAudit Re po.rt.Secbon Rafsrence10
2. Issues IdentifiedFARE EQUIPMENT & AVAILABILITY CALCULATIONCONTROL GAPS11
PRESTO Card ReadersRoles and Responsibilities: Devices are owned by PRESTO PRESTO bought them from Scheidt & Bachmann (S&B) Maintenance is done by PRESTO Monitoring of out-of-service instances are done byPRESTO's vendor’s offshore team Revenue transactions are recorded in PDS subsystem Out-of-service instances are recorded in the devicemonitoring software toolHow availability is calculated: Using device out-of-servicestatuses in the device monitoring software toolWho calculates the availability? Accenture12
PRESTO Card ReadersAvailability Calculation Issues Include: Availability calculation is only provided for Monday toFriday between 6 AM and 10 PM (A.1.1) Frozen card readers may be captured as "in-service" ratherthan "out-of-service" in the availability calculation (A.1.1) Vehicles that are in-service but not recorded in NextBusGPS application system (due to a number of differentreasons) are excluded from the availability calculation(A.1.2) Vehicles that are improperly included in TTC'smaintenance list and are actually in-service are excludedfrom the availability calculation (A.1.2) Not all out-of-service device statuses occurring betweenthe 15 minute pings are captured in the availabilitycalculation (A.1.3)13
PRESTO Card ReadersAvailability Calculation Issues Include (continued): Due to issues with the device monitoring software tool during ouraudit, some devices were not included in the availabilitycalculation and some devices were captured with an incorrectstatus such as "in-service" rather than "out-of-service" (A.1.4) PRESTO's vendor appears to be able to make adjustments to thedevice statuses for the availability calculation. The analysis andsupport for the weekly rate, including any adjustments made, isnot provided to TTC (A.1.4) TTC does not get a daily availability calculation spreadsheet forholidays in India/Canada and weekends in Canada (Note: Dailyavailability spreadsheet is now provided to TTC for holidays inIndia) (A.1.5) For daily spreadsheet not provided during holidays in India,PRESTO staff were not aware of the issue and back-up could notbe provided to confirm that these days were included in the weeklyrate (data purged after 60 days, when it is required to be kept for 7years per contract) (A.1.5)14
PRESTO Card Readers – Availability Rate Calculation. A.II A 1 2llne9uoeoi::J,. lo:&., C I D .-0 nc,:t a l ' v a,c:c,ur and SCHTle l"n&y ncn.b e 11nc::AI .a: a-11:. a.lJNoe.s.a.s . . a.re .nc '- :M'1I 'C.he road e c::c ng 1".a,.res are excllu eo r,-c::,n,. cnea ca, t:.on,eaoe.r &coses - e .eco,oeo lJO!Y' aa ·e vwc:-e monn.onng sc:,:f"n t.oc J .cas. .So8C'Vloe, CM" cx.n:-or--s.e ec::a.-cr-!-1.A. .1. T :air-clRi(wk:asce,eo ,eaoe,,s b e casxurecr as · o e . --ct,e& I or ou1.-or.sen, ,ceF.-ozenA14ts:saues .nh C e v 1 c e " ' 1 e TOOiTho.-e are rssue-& .,.u, ·u-.e 1. 1 "1:Setr 11' &1:: son eun-.es causesNocr,e,.,,oe:s co nCR oe capcureo 1n "tne3"81180 11 C&k:cUl&t.JonVosOnly t h e .15 m , n o sta'll.Us ss.A. 131Nc11.AD -- T cec 'l:.-eocne a 'l:Ycaacu1auon ano ex.cluoes all OU1: o Tserv1oe pew iO llS tn bet.Y.1een u-elnclludeONoc en matruncc.ons U .&'1: oocu U' e.L5 .,,. nvce 11 '"8 1ncervaas are reneeteo 1npingst.hO D V . R r , 1 8 rB-t:elNoVas.,.,-c,wnA.-1s. ,n an-aY lieJel' .et ll J:S oa,ry -8',a,1ao11('(),1n --e&o1euaauon ro,holidays.,. 1 e a . . - a a .-XS, ,"EIID'.k.llBrlCl::s Canada n o t -a-var Ib aTnea ·ca - -- lS r. Fnclay1: etv.1oon 1 1 anodl .10.Pf'l.'I TTCpcov,ded v,nn u-.o b a c k - u p f e w oa u"tY r.at.e nCM. rort.T e v#ee1 rya,"&1l&t lrcy" re'l:ecapll.Urecl fa.- Pair- PRESTO s . . - - s s t a f f , , . . , . . .-. 1 . . - , . . , . o · m t . sx:ludod .-. U-.e da,tv .va1lab 11 001. . e .-ene,c:ceo as pan. OT U oe v.1eekry ra1.e Hoo-w-.1e.-er. v .e CJiod e,oe,,.,e u-.e t &Cik-us O T.-eoo ang.,.,eekry ,ra-.e ro, Ol'ese aays 1::0 ve,nTyu.e'1" -a.ey "'"ere -&CJJ,O,e,O ll &Ck15
PRESTO Card ReadersMonitoring for out-of-service instances is done by PRESTO's vendor'soffshore monitoring team 24/7Incident Management Issues Include: Monitoring team did not always open a service ticket in the incidentmanagement system (called ServiceNow) for out-of-service devices(B.1.1) Device monitoring software tool used by PRESTO's vendor does nothave reporting/extracting capability available for the TTC and thedata in the device monitoring software tool is purged after 60 days(B.1.2) PRESTO does not maintain a running log of swapped devices,contrary to the Master Agreement (B.1.3) TTC staff need to improve the accuracy of the bus maintenance listprovided to PRESTO (B.1.4) TTC staff did not always report and raise service tickets in PRESTO'sincident management system for malfunctioning devices (B.1.5)16
PRESTO Vending Machines onNew StreetcarsRoles and Responsibilities: Machines are owned by PRESTO PRESTO bought them from S&B Maintenance is done by S&B Monitoring of out-of-service instances are done byS&B Revenue transactions and out-of-service instances arerecorded in FareGo subsystemHow availability is calculated: Based on repair time (calculated fromthe time the issue was raised in the incident management system tothe time it was fixed)Who calculates the availability? Scheidt & Bachmann (S&B)17
PRESTO Vending Machineson New StreetcarsAvailability Calculation Issues Include: Out-of-service machines as a result of "coin box full" areexcluded from the availability calculation because it istechnically not broken so it is not the vendor's responsibility(A.2) Out-of-service machines as a result of network connectivityissues are excluded from the availability calculation becauseconnectivity is the responsibility of another vendor (A.2) Prior to July 2019, PRESTO was responsible for the first linemaintenance but this repair time was not included in theavailability calculation (A.2) Not all out-of-service machines were included in theavailability calculation as being "not available" but shouldhave been, according to the definition between PRESTO andits vendor (A.2)18
PRESTO Vending Machineson New StreetcarsIncident Management Process: S&B staff monitor the out-of-service instances in their backend system (FareGo) TTC also raises out-of-service incidents to PRESTO's incidentmanagement system S&B is responsible to fix hardware and software issues,Garda is responsible for coin collection, and Telus isresponsible for network connectivity issues19
PRESTO Vending Machines on New Streetcars – e.g. coin box fullThe new TTC streetcars have two vendingmachines on board, which take both tokensand coins. The coin boxes inside thestreetcars are emptied nightly at the garageIf a coin box inside the vendingmachine is full, the vendingmachine will show as out ofserviceSystem will generate awarning message run byVendor APRESTO should ideally notify TTC tomake the streetcar available at nightfor coin collection by Vendor BVendor A ideally shouldnotify PRESTO that thecoin box is full20
TTC Subway Fare GatesRoles and Responsibilities: Fare Gates are owned by TTC TTC bought them from S&B Maintenance is done by TTC and S&B Manual identification of out-of-service instancesare done by TTC Revenue transactions are recorded in FareGosubsystemHow availability is calculated: Using fare gate out-of-servicestatus in the FareGo subsystemWho calculates the availability? TTC21
TTC Subway Fare Gates- The identification of broken fare gates is currently a manualprocess, requiring TTC staff to identify and report issues- TTC staff advised us that the next version of software that TTCplans to purchase will provide automatic identification – thisinformation will help TTC to address issues with fare gates in amore timely way22
TTC Subway Fare Gates23
TTC Subway Fare GatesIncident Management Process Issues Include: Not all out-of-service gates that are stuck in an open position arebarricaded to prevent customers from passing through, whichcould result in revenue loss to TTC (D.1) TTC does not receive automatic alerts from the current FareGosubsystem when the fare gate goes out of service (D.2) TTC staff currently have to manually identify the out-of-servicegates (D.2) At the automatic entrances where there is no TTC Staffpresence, the gate could potentially stay out of service for a longtime (D.2) Escalated issues to TTC's vendor to fix fare gates were notcompleted within the targeted timeline as per the SLA (D.3) TTC gets compensated up to a maximum of 25 per cent of theservice charge if the availability and maintenance targets are notmet by its vendor (D.3)24
2. Issues IdentifiedFARE EQUIPMENT & AVAILABILITY CALCULATIONCONTROL GAPS25
Controls and Assurance Provided byPRESTO Need Strengthening Audited 3416 report 12 control gaps Some key controls not included or not specified in the3416 report Risk exists between the device level andsubsystem level – controls need strengthening 3416 report does not specifically include TTC faregates, data concentrators and Farego subsystem Operational controls not included in report requireanother mechanism to address them, such as an SLA(e.g. availability of card readers) Retailer network controls No reconciliation controls (cash, debit/credit) forretailer network Metrolinx should receive cash reconciliations andrequest separate 3416 report from vendor26
Controls and Assurance Provided byPRESTO Need Strengthening A manual reconciliation is done by PRESTO staff, but only betweenthe subsystem and PRESTO's central system. The manualreconciliation identifies transactions missing in PRESTO's centralsystem from the subsystem, but could be prone to human error andmay not identify all items (E.1) There is no reconciliation between the device level (where thetransaction starts) and the subsystems and PRESTO's central system.This creates a higher risk if there are any missing transactions fromthe device level to the subsystems and to PRESTO's central system,as they may not be identified and pushed through (E.1) TTC has not been provided with device level data from PRESTO (datais encrypted and purged after seven days) (E.2) PRESTO is not provided with a cash and debit/credit cardreconciliation from PRESTO's third party retailer network for salestransactions of TTC's monthly passes and single-use tickets (E.4) TTC is not compensated for sales of TTC's monthly passes and singleuse tickets that do not get into PRESTO's central system (E.4)27
Flow of PRESTO Revenue Transactions from Device toPRESTO’s Systems to TTC’s Bank Account/PRESTO/central Sys(amto oa;1yMon ey gets deposited i n TTC .srevenue b a nk account::: ---Sett.l ernentMonthly,ec.onci 11at1oncon-1p1etea byPRESTOaooTTCSubSy.stewns. toCentxal SystemManual,econciliauon iSoone by PRESTObetwe,ensubsystems andcentral systemTransactionswnich 1000.unesuan.sacttonsmissing 1n thecentral systemfrom subsystemsFareGo3 5Precise PairklLinkSuDsy.s;a:eo,iSoosysteffiT f"a f'l&:tCOOn:Sfrom Pl-ecisePa.rkLinks ubs:ystenl donot ffl.2f.Ge n: 1rnoTransactions andEventsPRESTOoemntlsysmm TTCge,sthe money at'themonlhencLd e v ice lev e l datanot obtai n ed andis pureed after 7d a ys. norec:onc11,at ,onb e t'Ween dev i cesand subsystems 3416 rep0rt doesPRESTO VendingM aCt'l tnes.'·.:i ·ParkeonVendingM a.Ch ines./PREiizE}! ·. ·· ci ' . ·.- !. - . 'Auditor General's Office - Integrity, ExcellenceBuses and Streetcarsand Innovation28
3. Moving ForwardTogether29
Moving Forward1. It is not uncommon to have issues arise in large,complex outsourcing deals – Focus on how to resolve the issues2. Seek a win/win for everyone to win includingcustomers Acknowledge individual and shared accountabilities andresponsibilities in this arrangement – move from blame3. Keep moving forward Obtain the information to diagnose the root cause Put Strategies in place with key deliverables to address theissue30
FORESIGHT Vision for the rest of this contract and thenext – strategically - What matters most? Improving customer experience and maximizingrevenue?– Is everyone aligned to achieve success? Identify options to bring the deal the nextlevel, including for: Metrolinx - How and when it will complete strategicdeliverables, such as open payment TTC, What does it visualises for the citizens of Toronto define what is important, determine the information and the service levels it needs toachieve these goals for Torontonians32
INSIGHTBoth TTC and Metrolinx need better information toidentify, diagnose, and resolve issues1. Define clear, agreed upon, and formalizedoutcomes and Service Level Agreement (SLA)targets – set interim targets if necessary2. Data sets are missing (e.g. device level data)& analytics need improving What do the percentages mean? Unpack the numbers and definitions3. Improve Communication33
Examples of Points of view that need to be reconciledExampleMetrolinxTTCSLAsNot definedNot definedPresto Adoption66%81%Debit /Credit cardfunctionalityPermanent RemovalTemporary RemovalContracted DeliverablesOutstandingApprox. 20% (mainlyOpen Payment)At least 40%outstandingPresto Card Readeravailability targetNo SLA in place 99.99% percontractRevenue Loss Claim 0 7.5 M for 3 yearsended Dec 31, 2018Withdrawals – other than basically feel it isauthorizedFeel it isunauthorized PRESTO commissions34
OVERSIGHT1. Need: the right people at the right table with the right informationTo: Verify right things are getting done / targets are met Unpack issues if necessary make decisions to move the parties forward2. Governance gap may have resulted in issues beingneedlessly escalated3. Gap needs closing by reinstating Joint ExecutiveCommittee and operationalizing the Expert Panel tohelps set up SLAs when needed35
Governance and Contractual RelationshipsCard reader a '&il8ib II)' calcul&oooVendor ATTCBoard---i & Jb-Vendor AC&r I re&CJer mon1tor1ns tootLavailability cak:ulatian[Parl eon \lendlns mac54.Jb-Voodor B )r·····- ; - - ---·· ··::::: e METROLI NX lPRESTO/PRESTO RevenueSecban E[VendorDVendor EVendorF;:. . . . . 0 . . 1101 I I i 101 i I i 11 1 IFare G1111:es secuoo D A.1.4. B.1.2 A.'2liost data ooncentratar DetYrork forfare gac.es ancl 111eooor1 connectMtYfor a ll Pf?ESTO oe,,.,1ces- .[[A .1,B .1.1Determination of in-sarv,ce ,.-ehiclesPRESTO d a ."JCe monitaring))A.2. D.2t- orl connectiOII rrom fareD .'2gi;ite ta data concen11ratorColn &llll to1 .e11 00'".lectJon rromvend ing medmnes an neviA.2. B 2streetcarsPRESTO sales channelJE .4l I 011 1 , , . :W'etroi,1\1( vendlns maon- 11eavailab rt)' celcu5auonVendorG; . .B.2 . A.2Coou-act:ed maintenance for fare gateSFare g;rte .avarlabchty calculationC.5Schecme GcNemanc:e Structure( lPRESTO/( ;.v;( CCTranspo( (I UP :ID) 8.] ti] ti1 tl l· · s{II2lim' . J[]JC.I,J/,11-TTC \lendorsITC & PRESTO Vendo.raPRESTO SUl - rsContractual Ralatians.h!pVoting Member -PRESlO/Metrdlnx end A.ccanturavendorssecoooAudit Repo.rt.Secbon Refarance36
Concrete Steps to Move Forward in aBig Way some starting examples1. Reinstitute the Joint Executive Committee2. Set up the Expert Panel and provide information todefine and set key SLAs3. List all deliverables yet to be complete which ones will be delivered target timelines for delivery4. Address ‘laundry list’ of issues identified in the AGreport to help resolve matters5. Align data reports to support root cause analysis6. Communication the parties differ – bring impasses tothe Board37
34 Recommendations to MoveForwardThe Executive Leadership Teams of both TTCand Metrolinx conveyed their support for thisaudit and found that it brought new insightand perspectives. They have shared a renewedcommitment to achieving the visioncontemplated in the business arrangement forthe benefit of citizens and to putting thepillars in place to make this happen.Auditor General's Office - Integrity, Excellenceand Innovation38
TTC subway fare gates (stuck in open position) PRESTO vending machines on new streetcars (2 types) 2. Revenue transactions not captured properly by PRESTO's back -end systems 3 Introduction Audit Phase 1 Audit Phase 2 Focus was on: Fare evasion and the estimated loss of passenger revenue
