WIXLIB

Taylor et al.Data Mining for Vehicle Telemetrywas used in Artificial Neural Networks, Decision Drees, and Support Vector Machines,achieving comparable performances.In this paper, we consider the driving conditions monitoring problem of road classification. Whereas driver monitoring focusses on driver state inside the vehicle, drivingconditions problems relate to the outside environment, including the traffic levels, androad type (Huang et al., 2011; Langari and Won, 2005; Wang and Lukic, 2011). Drivingconditions and road type can be defined in several ways, including level of service (Carlson and Austin, 1997; Langari and Won, 2005; Murphey et al., 2008), descriptive (Hauptmann et al., 1996; Huang et al., 2011; Qiao et al., 1995; Tang and Breckon, 2011; Tayloret al., 2012), and government classification (Taylor et al., 2012). Possibly the most useddefinition in research is that provided by Carlson and Austin (1997), based on level ofservice and driving cycles. Level of service and driving cycles are qualitative measuresdescribing observed operational conditions (Langari and Won, 2005), and therefore maybe subjective. Descriptive definitions are of most use, as they have a direct relationshipto the current situation and environment. For example, Huang et al. (2011) use the labels highway, urban road (both congested and flowing), and country road. Hauptmannet al. (1996) use an even more direct classification structure, based upon current carbehaviour. Their five labels range from very fast, straight line driving on flat roads, tovery low speeds or stop. These are used to represent further driving situations, such ashighway driving, and traffic lights or parking.Wang and Lukic (2011) provide a survey for driving conditions prediction, with thefocus on Hybrid Electric Vehicles. They recognise that many researchers use drive cyclesfor a road definition, and use only information from the vehicle speed in their models. Forexample, average velocity and acceleration, as well as peak accelerations and percentageof time in certain speed intervals are often used (Huang et al., 2011; Langari and Won,2005; Murphey et al., 2008; Park et al., 2008). These features are also often extractedfrom 150 seconds of data in order to produce good classification performances (Wangand Lukic, 2011). These approaches have clear limitations in determining the currentdriving conditions. First, steering wheel behaviour is likely to differ in different situations, providing additional predictive information. Second, if features are extracted fromlarge amounts of temporal history, the model is likely to be slow to react to changes inenvironment.5

Taylor et al.Data Mining for Vehicle TelemetryOther authors have used different features in addition to those extracted from speedcycles. Hauptmann et al. (1996), for example utilize engine speeds, accelerations, andgradient. Additionally, Qiao et al. (1995) extract features from the pedal positions,temperatures and selected gear. These features, however, although they contain differentinformation from the vehicle speed, are all related to it. Engine speed, for example, has aPearson correlation with vehicle speed of 0.96 on data we have collected, meaning that itis adding little new information into the system. Qiao et al. (1996) note that the lengthof the temporal window that features are extracted over is an important factor in thesystem’s reaction time and they use a much smaller window length of 6.25 seconds. Oneshortfall in their work, however, is that automatic feature selection is not performed andfeatures are selected based on the intuition of the researchers.Examples of feature selection being used in this domain are mainly those that usefeatures extracted from speed cycles.Murphey et al. (2008) and Park et al. (2008)proposed a selection procedure based on binary class separability of single features: if afeature is able to distinguish one class label from the others, then that feature is selected.Huang et al. (2011) also use a non-parametric, one-way analysis of variances to ensurethat features used are relevant, and use cross correlation analysis to remove redundancy.They investigate 11 features in total, with only 4 being manually selected for classification.When dealing with CAN-bus data, however, the number of signals and features can bein the order of 1000 seconds, meaning automatic approaches are necessary (Taylor et al.,2013a).A final approach to the problem of road classification is the use of visual inputs, e.g.from front mounted cameras, and applying image processing techniques (Jansen et al.,2005; Tang and Breckon, 2011). In their work, Tang and Breckon (2011) use color, textureand edge features from image sub-regions as inputs into a neural network, and using colouranalysis, Jansen et al. (2005) identify the terrain type. Such systems are limited becausethey rely on non-standard sensors, generally need greater computational processing andare severely affected by poor lighting conditions, such as night-time driving.6

Taylor et al.3Data Mining for Vehicle TelemetryData mining methodologyThe methodology we present is based on a general framework for data mining outlinedby John (1997). As in (Huang et al., 2011; John, 1997), and others, we use the termdata mining to refer to the process of collecting, processing, and learning from data as awhole. The methodology presented in this paper is of a similar form to those found inmany temporal data mining applications, including (Constantinescu et al., 2010; Huanget al., 2011; Kargupta et al., 2004; Manimala et al., 2012; Sagheer et al., 2006; Shaikhet al., 2011; Wollmer et al., 2011) and others, and is split into stages of: data collection;feature extraction; feature selection; classification and evaluation. In this paper, wealso consider selection of signals, prior to feature extraction. This has the advantage ofsaving computation, as only selected signals have to be processed later in the data miningprocess.3.1Data collectionThe data collection must be planned carefully for data mining to be successful. First, theconditions under which data is to be collected, as well as what data should be recordedmust be decided. It is important to control the acquisition conditions so that resultsbecome meaningful. Deciding on which data to record from vehicle telemetry is nontrivial, because of the thousands of signals available via the CAN-bus (Farsi et al., 1999).Recording and analysing all of them is an impossible task, so most researchers makeeducated guesses based on domain knowledge.Second, the data representation should be in a form that is suitable for subsequentprocessing. For instance, the CAN-bus is an event based communications network wheresensors broadcast data at varying rates (Farsi et al., 1999), so consequently, some datamining methods will not be directly applicable. It is typical therefore to re-sample thedata at a common rate, e.g. between 10 100Hz, producing M signals, S1 , S2 , . . . , SM ,with samples of the same frequency.Finally if the problem is to be posed as one of classification, the ground truth usedto derive the labels must be assigned in a consistent and reliable way. Improper labelassignment can lead to noise in the learning process leading to poorer classification results.Drive cycles can be generated for each label and treated as separate in order to simplify7

Taylor et al.Data Mining for Vehicle Telemetrylater processing (Huang et al., 2011). Treating the data in this way, however, ignores anytransition periods where a label change occurs. This may cause an evaluation to prefermodels that use large amounts of historical data, but have slow reactions to changes inenvironment. In this paper, we consider the more realistic scenario of journeys whichcontain several periods of differing labels. Although this introduces noise during labelchanges, we believe this approach will provide more accurate performance estimates thatdo not ignore these reaction times.3.2Feature extractionIn temporal data mining, it is advantageous to include historical information when performing classification (Antunes and Oliveira, 2001). Without this, an individual samplecontains only information about the exact point that sensor measurements were made,which may be noise. This means that no trend or statistical information can be usedin determining the classification. We refer to this process of incorporating historicalinformation into the current sample as temporal feature extraction.Consider a signal, S, of length T , such as the vehicle speed or SWA.f (S(t), S(t 1), ., S(t l 1)) f (S(t, l)),where f (S(t, l)) is a temporal summary of the values between times t and t l 1. Ift l, because it is at the beginning of the recorded signal, t samples are used. Featurescan generally be split into two categories, namely structural and statistical. Structuralfeatures describe the trend of the signals, whereas variations, peaks, and averages arerepresented by statistical features.In each time instance, m signals, S1 (t), S2 (t), ., Sm (t) are sampled, from each of whichk features, f1 , f2 , ., fk are extracted. Therefore, after feature extraction, a sample, x(t),at time t, is represented as,x(t) {f1 (S1 (t, l)), ., f1 (Sm (t, l));f2 (S1 (t, l)), ., f2 (Sm (t, l));.;fk (S1 (t, l)), ., fk (Sm (t, l))}.8

Taylor et al.Data Mining for Vehicle TelemetryIt should be noted here that in some cases, different features may be extracted overdifferent temporal windows from each signal, meaning that the value of k and l may varybetween signals and features in the same dataset. Finally, whereas Huang et al. (2011)extract features from windows with no overlap, in this paper features are extracted oversliding windows with an overlap of l 1. This means that a temporal dataset of lengthT , is a sequence of samples,X x(1), x(2), ., x(T 1), x(T ).This method both maximizes the number of samples and means their number is notdependent on window length. The overlap in windows does increase the autocorrelationin the data, however, which can be problematic for some data mining methods.3.3Signal and Feature selectionAs previously stated, signals and features are often hand selected using domain knowledge. This is sub-optimal and time consuming, however, and may introduce biases towardthe engineer’s preferences. We therefore use automatic selection of both signals, prior tofeature extraction, and features, after feature extraction. We consider two common feature selection methods, Principal Component Analysis (PCA), an unsupervised methodfor redundancy feature selection, and Mutual Information (MI), a supervised method forrelevancy feature selection (Witten and Frank, 2005).PCA transforms a dataset onto a set of orthogonal dimensions which are linearlyuncorrelated, referred to as principal components (PCs) (Witten and Frank, 2005). Thisis done through computing Eigen values from the covariance matrix of the data. Theidea is that because the dimensions produced are linearly uncorrelated, there is very littleredundancy in the dataset. Also, if the PCs with the highest variance are selected (i.e.those associated with the largest Eigen values), they are also likely to contain the highestentropy and be good predictors.Whereas PCA is an unsupervised method of feature selection, MI takes into accountrelationships between features and the class labels. MI is defined as,M I(fi , C) Xp(vi , vc ) log2vi vals(fi ),vc vals(C)9p(vi , vc ),p(vi )p(vc )

Taylor et al.Data Mining for Vehicle Telemetrywhere fi is a feature and C is the class labels. A high MI indicates that the feature is agood predictor of the class labels and that it should be included in a predictive model.Both of these feature selection methods are able to provide a ranking of features. PCAranks the PCs by their variance, where those with a larger variance are ranked higher.With MI, features are ranked by the closeness of their relationship with class labels.3.4ClassificationIn this paper, we employ three widely used machine learning algorithms: Naı̈ve Bayes,Decision Tree, and Random Forest, that are all available in the Waikato Environmentfor Knowledge Analysis (WEKA) machine learning suite (Witten and Frank, 2005). TheNaı̈ve Bayes algorithm learns class conditional distributions from the data and uses Bayesrule to make inferences. For the Decision Tree classifier, we use the C4.5 algorithm whichsplits nodes based on MI. Once the full tree is built, pruning of nodes with few applicablesamples is performed to prevent over-fitting. The Random Forest algorithm builds severalDecision Trees, each on different sub-samples of the data and sub-sets of features. Eachof these algorithms are chosen because of their wide-spread use and the ease with whichmodels produced by them can be understood by a domain expert.In road classification, there is an inherent class imbalance where one or more classlabels dominate the training data. For example, there is a 5:1 ratio of single lane road examples to multiple lane roads, and a smaller number of motorways than other road typesin our data. This imbalance can lead to biases in models, which tend to prefer to labelinstances that are a majority (He and Garcia, 2009). We consider two approaches to dealing with class imbalance, namely over-sampling and under-sampling. In over-sampling,samples of the minority class label are duplicated to increase their representation, whilein under-sampling, some proportion of the majority class samples are decimated. Duplication and decimation is performed by selecting samples at random.In the multi-class problem of road type classification, we adopt Error CorrectionOutput Coding (ECOC) (Berger, 1999; Escalera et al., 2008; Soda and Iannello, 2010),which has been shown to have resilience to class imbalance (Berger, 1999; Escalera et al.,2008; Soda and Iannello, 2010). ECOC is an ensemble classification algorithm whichsplits a multi-class classification task into several binary-class problems. A unique binarycode, Ci , is given to each of the classes as in Table 1. A classifier is built to predict each10

Taylor et al.Data Mining for Vehicle TelemetryClassCodeA Road1000111B Road0100100C Road0010010Motorway0001001Table 1: Example exhaustive coding for Road classification.of the bits in these codes, i.e. there will be as many models as there are bits in the codes.In this example, the classifier predicting the third digit of the codes would predict 1 forC roads, and 0 for the remainder. The true code with the smallest Hamming distancebetween itself and the predicted code is then output as the sample classification.Some of the binary class models will have better performance than others, because ofthe difficulty of distinguishing the classes. A and B roads, for example, are much moreclosely related than C roads and Motorways, so we would expect a model distinguishingbetween A and B roads to have worse performance. Because of this, it is sometimesbeneficial to take account of this in the Hamming distance calculation by weighting itwith expected performance (Zhang et al., 2012). This is done by updating the Hammingdistances by multiplying them by the expected performances and can be illustrated usingthe example in Table 1. Suppose, for example, that the expected success rate, estimatedusing the training data, for each of the dichotomies is W [0.75, 0.5, 1, 1, 1, 0.5, 0.5]. Ifthe base models then output a bit string of 1100101, the weighted Hamming distanceswould be 1, 1.25, 4.25, 3.25 for A road, B road, C road and Motorway respectively. Withthese distances, the output the classification is of type A road.3.5EvaluationFor evaluation, we use random sub-set validation over sub-datasets, a variation on crossfolds validation. Each iteration consists of a training and a testing phase, where themodel is built using data from a subset of the journeys and then used to label instancesfrom unseen journeys. In each training phase the same number of datasets are used toselect features and build a model, and the remainder of data is used as testing data.11

Taylor et al.Data Mining for Vehicle TelemetryThis is repeated for several combinations of training and testing data, producing a largenumber of predictions made by the models. These predictions are then compared againstthe ground truth to produce a performance metric. Because it is possible to use somesamples multiple times in an evaluation, the performance metrics exhibit Monte Carlovariation.Unlike other work in environment classification, we choose to not use accuracy or errorrates as a measure of performance. Instead, in this paper we use Area Under the ROC(Receiver Operating Characteristic) curve (AUC), as it is better suited in situations witha high class imbalance (Huang and Ling, 2005). This is because the imbalance may biasthe output of a classifier, which is not accounted for by accuracy. Consider a model thatoutputs a probability distribution over the class labels and trained with an imbalancedbinary classification dataset with numerous times more 0 labels than 1s. When usingaccuracy, an output of p(0) 0.7, p(1) 0.3 with a decision threshold of 0.5 would meanthat the prediction is 0. In this case, a model that outputs p(0) 1 for all inputs, alwayspredicting 0, may provide a very high accuracy on the dataset due to this being correctfor most of the samples. When used in the real world, however, predicting 0 regardlessof the situation is not useful. The ROC curve accounts for any class biases by computingtrue positive and false positive rates over several thresholds, ranging from 0 to 1. Athreshold of 1 for a class means that the class label is never predicted, producing no falsenegatives and no false positives. Conversely, a threshold of 0 would mean all instancesare predicted as the class label, producing a false negative rate and false positive rateof 1. The true positives are then plotted on the y-axis against the false positives on thex-axis, with the ideal curve following the y-axis as close as possible, having an AUC of 1.44.1Experimental SettingData collectionWe used a Video VBOX Pro for the data recording, which allowed for the recording ofvideo streams synchronized with selected CAN-bus signals. In order to have the CAN-bussignals at a constant frequency, the VBOX interpolates signals by taking the last-seenvalue. This method ensured that nominal, integer or binary signals are not averagedoutside of their domains. For instance, if a binary signal is only broadcast every second12

Taylor et al.Data Mining for Vehicle Telemetrybut sampled at 20Hz using linear interpolation, a value change would produce somesamples between 0 and 1. Also, using the last broadcast value ensures that the signal isas up-to-date as possible, although it may mean it is more susceptible to noise.The data used in this paper was collected over 16 drives across the Midlands, UK, intwo cars. Each journey involves at least one driver, with a mean journey length of 51minutes. Output from 15 CAN-bus sensors, listed with brief explanations in Table 2, wererecorded each at 20Hz for a total of 49403 seconds, which is comparable to the lengthof data used in (Huang et al., 2011). Some sensors used are expected to have very littlerelevance in determining the road type, and others are highly redundant. As previouslystated, these expectations may be incorrect, as is the case with the ambient temperaturesignal. Although it may initially be expected to be a poor predictor, it has one of thehigher MI scores (0.197 for carriageway type) in data we have collected. On furtherinspection we find that its Pearson correlation with vehicle speed, which is expected tobe a good predictor, is 0.774. This makes some intuitive sense, as the temperature nearthe engine will rise with vehicle speed as the engine works harder. With this insight wecan say that ambient temperature is a good predictor of road type, but that it is somewhatredundant to other signals. After signal and feature selection, only the features whichare useful for the problem should be used in classification.The ground truth for the dataset was achieved using GPS and applied by hand usingGoogle Earth. GPS coordinates are looked up in Google Earth and a label is decided, andassigned to samples. For the carriageway classification, the number of lanes is decidedby looking at the satellite images provided. If there is mor

toring via CAN-bus data that is based on the general data mining process. The . Sections 6 and 7, we discuss the results, draw conclusions and identify future steps. 2 Related work Data mining of CAN-bus data has been used in several applications, including fault de-tection (Crossman et al., 2003; Guo et al., 2000), driver monitoring (Mehler .