WIXLIB

Accuracy Audit of Cox Communications’Internet Data Usage MeterBy Peter SevcikMay 2015Cox Communications has deployed a data usage meter system to measure and report howmuch Internet data traffic a residential subscriber consumes and generates. Cox reportsInternet subscriber usage information on its customer portal, and notifies subscribers viaemail when they approach the monthly data usage limit associated with the servicepackage they have purchased. Cox engaged NetForecast to perform an independent auditof the accuracy of its data usage meter system.NetForecast generated test traffic, performed independent traffic measurements, obtaineddaily usage meter records from Cox for three test locations, and compared NetForecast’susage data with Cox’s data usage meter records. NetForecast performed tests in threelocations in September and October of 2013, and retested one location in October andNovember of 2014. Cox had no prior knowledge of the timing or volume of theNetForecast test traffic.Cox established an accuracy goal for its Internet data usage meter to correctly measuretraffic passing through a subscriber’s cable modem within plus or minus ( /-) 1.0% foreach month. Our analysis validates that the meter is accurate within plus or minus ( /-)0.1% (one tenth of one percent) for the month, and within plus or minus ( /-) 0.2% (onefifth of one percent) for the cumulative daily sum at each location—an outcome that fallswell within Cox’s 1% stated goal.The Cox Data Usage MeterCox’s Internet data usage meter provides subscribers with information about how muchtraffic has crossed their residential Internet connection. Subscribers can view a dataconsumption summary for the current period in a “gas gauge-like” view (Figure 1). Theycan also view more detailed usage reports as shown in Figures 2, 3, and 4.All of the reports sum traffic passing from the Internet into the household (downstream)with traffic passing from the household (upstream) to the Internet into a single value.NetForecast ReportNFR5118 2015NetForecast, Inc.Figure 1 – Sample Subscriber Summary View of Usage

Cox’s Internet usage reports are accessible via the Cox customer portalat http://www.cox.com/. Subscribers with online accounts can access their Internet usageinformation by logging in at the sign-in prompt at the top left of the portal menu bar, andhighlighting My Connection. This will open a navigation window. The Data UsageMeter link is located under Internet Tools in the bottom section of the navigationwindow.By scrolling down the data usage meter page, subscribers can view a detailed daily usagegraph for the current or previous usage cycles (Figure 2), and can choose other data usageviews, including: a table of daily consumption values (Figure 3); a cumulative monthlygraph showing how much traffic has been consumed relative to the subscriber’s monthlyusage limit (Figure 4), and a monthly history graph.Figure 2 – Sample Graphical Report of Daily Usage 2015, NetForecast, Inc.Report 5118Page 2

Figure 3 – Sample Table of Daily UsageFigure 4 – Sample Cumulative Daily Usage Report 2015, NetForecast, Inc.Report 5118Page 3

How the Cox Data Usage Meter WorksCox subscribers connect to the Internet through a cable modem at the subscriber’slocation, and from there data traffic travels over a local coaxial and Hybrid Fiber-Coaxial(HFC) cable system to a Cable Modem Termination System (CMTS). The traffic thencontinues through Cox’s network and into the Internet.The CMTS counts downstream and upstream traffic for each subscriber cable modem itserves. Downstream traffic flows from the Internet to the subscriber, and upstream trafficflows from the subscriber to the Internet. The three test sites were supported by a CiscouBR10000 series CMTS. The CMTS periodically reports the down and upstream countsin an Internet Protocol Detail Record (IPDR) as shown in Figure tor(ETL)EBIWarehouseDataUsageMeterFigure 5 – Usage Meter Process FlowFrom the CMTS, IPDRs are sent to the IPDR Collector (Active Resource Manager, orARM for short), which collects, processes, and stores IPDR data streamed from CMTSs.IPDR processing involves time normalization and usage data mediation, and the ARMalso performs checks and cross checks the completeness and correctness of IPDR data todetect anomalies that can result in inaccurate Internet usage data.In the next stage, the IPDR aggregator gathers data from multiple collectors and convertsincremental traffic counts into traffic data in a process referred to as ETL (extract,transform, and load). The IPDR aggregator extracts data from the CMTS’s, transforms itto fit operational needs, and loads it into the Cox EBI (enterprise business intelligence)warehouse. Within the EBI warehouse, traffic data is associated with subscriber accounts,and the meter value is calculated for each day to create a meter record. From here the datais fed to the Cox subscriber’s data usage meter within the Cox Web portal, whichdisplays the data usage views shown in the previous section.The requirements for how a cable modem communicates with the CMTS and for howsubscriber traffic is transported is defined in the Data over Cable Service InterfaceSpecification (DOCSIS) developed by CableLabs. The IPDR specification is managed bythe TeleManagement Forum (TM Forum). A DOCSIS Management Information Base(MIB) defines how traffic is stated in the IPDR. 2015, NetForecast, Inc.Report 5118Page 4

NetForecast Internet Data Usage Meter Accuracy ValidationMethodologyCox defined specifications for the accuracy and functionality of its data usage meter.NetForecast independently validated the meter relative to Cox’s stated goals for: trafficcounted, data accumulation period, error bounds, timeliness, exception handling,granularity, mathematical consistency, accessability, availability, and clarity as defined inthe meter specification. A comprehensive description of data usage meter specificationfactors is available in NetForecast’s report, ISP Data Usage Meter Specification BestPractices for MSOs, reference [1].As the data usage meter system auditor, NetForecast has no stake in the design orimplementation of the Cox data usage meter, nor do we endorse the Cox data usage meterspecification goals. The objective of our testing is to assess whether or not the systemcomplies with the meter specification.NetForecast obtained subscriber accounts and performed usage meter accuracy validationtesting for the three test locations. Both downstream and upstream testing wereperformed under a variety of conditions.NetForecast instrumented the accounts with a test laptop PC running NetForecastsoftware and a Linux-enabled NETGEAR router with NetForecast’s Usage Mapping(UMap) firmware. In addition, we used FTP accounts on various NetForecast servers onthe Internet. Figure 6 shows the NetForecast instrumentation. A more detailed descriptionof the NetForecast methodology is available in the NetForecast publication Validating theAccuracy of ISP Subscriber Traffic Usage Meters, reference [2].The NetForecast test involved scripts performing an FTP file transfer from one of our testservers to the laptop. The tests consisted of repeatedly transferring files of varying sizesin complex patterns. These tests were also performed as uploads from the laptop to theserver. The script generated a log file documenting the transfer results and capturingdetailed timing information for each transfer.Extreme care was taken to ensure that only test traffic was sent or received through thecable modem. The laptop and router were the only devices connected to the cablemodem. The laptop was cleansed of all applications that could generate traffic not neededfor the tests—and applications were configured to neither request nor receive anysoftware updates. Finally, remote management of the laptop was scheduled not to occurduring testing. 2015, NetForecast, Inc.Report 5118Page 5

Figure 6 – The NetForecast MethodologyFor each test, NetForecast produced three of our own carefully documented records of upand down traffic, illustrated by the pink boxes in the lower left of Figure 6. These recordsources included: File transfer logs and traffic data on the test PC Upstream and downstream traffic processed by the UMap router firmware Server records generated using detailed traffic instrumentation software.Because of their locations in the network, each of the NetForecast measurement pointssees different amounts of protocol overhead. NetForecast accounted for upstream anddownstream protocol overhead as counted at the CMTS.NetForecast also gathered data usage meter data from two places in the Cox meter systemduring testing phase as shown in Figure 6. The data sources were: Hourly traffic records from the data warehouse, which provided a preview ofmeasurements presented on the Cox meter portal The account portal for the test location, which reported the usage meter data tothe subscriber.Data usage meter system accuracy validation entailed comparing data from the two Coxreporting sources to the three sets of NetForecast-generated measurement data.Comparing the NetForecast and Cox data enabled an end-to-end view of the Cox datausage meter’s accuracy.The hour-by-hour data records collected by NetForecast from the three sources describedabove were not shared with Cox. Cox provided daily usage meter record files for each ofthe three sites. Cox did not know which tests NetForecast performed or when they wereperformed. 2015, NetForecast, Inc.Report 5118Page 6

NetForecast Independent Cox Data Usage Meter Validation ResultsThis report section summarizes NetForecast’s validation test results for Cox’s subscriberInternet data usage meter as seen from the subscriber perspective in the threegeographically separate measurement locations. NetForecast validated Cox’s specifiedgoals for the data usage meter against what we actually found.1. What the meter countsCox Goal: The Cox data usage meter counts all IP traffic crossing the cable modemCMTS HFC connection, except for Cox-specific service traffic: Video accessed through Cox TV ConnectCox Video on Demand and TV accessed through the Contour app and notaccessed via a 3rd party app like HBO GO or ESPN3Cox Digital Telephone[Note: NetForecast did not test using Cox-specific service traffic, so we did not validatewhether or not the meter counts it.]The Cox data usage meter also does not count the DOCSIS traffic, which results fromcommunication between the CMTS and the cable modem.The meter system counts the following traffic classes as subscriber usage: Subscriber-generated traffic (also called payload)Protocol overheadBackground traffic (non-subscriber-generated traffic)[Note: Overhead is contributed by a number of network protocols including: DHCP,DNS, Ethernet, IP, and TCP. These protocols are counted. The overhead varies byapplication, depending on the protocols the application employs to transmit user dataover the Internet. These protocols are necessary for your Internet service to functionproperly.]NetForecast Validation Result: NetForecast confirms that all traffictransiting the Ethernet port between the cable modem and home routeris counted. This includes overhead contributed by a number of networkprotocols such as: DHCP, DNS, Ethernet, IP, UDP, and TCP.Additional protocols may be in use based on the devices in the homeand applications they operate. DOCSIS traffic was not counted.2. Data accumulation periodCox Goal: The meter data accumulation period defines the minimum period over whichthe meter needs to gather a sufficient number of measurements to be accurate. Cox usestwo accuracy timespans: cumulative daily sums, and the monthly total.NetForecast Validation Result: NetForecast confirms the use of dailycumulative, and one month total data accumulation periods.NetForecast performed its analysis using the same data accumulationtime frameworks. 2015, NetForecast, Inc.Report 5118Page 7