Transcription
Introduction to Machine Learningand Deep LearningConor Daly 2015 The MathWorks, Inc.1
Machine learning in actionCamVid Dataset1.2.Segmentation and Recognition Using Structure from Motion Point Clouds, ECCV 2008Semantic Object Classes in Video: A High-Definition Ground Truth Database, Pattern Recognition Letters2
Machine learning is everywhere Image recognitionSpeech recognitionStock predictionMedical diagnosisPredictive maintenanceLanguage translationand more [TBD]3
Agenda1.Machine learning – predictive maintenance2.Deep learning – build a digits classifier3.Predictive maintenance revisited – a deep learning approach4
What is machine learning?Machine learning uses data and produces a model to perform a taskMachine Learning ApproachStandard ngshortHand Written ProgramIf X 0.5then “LONG”If Y 4 and Z 5then “MEDIUM” mediumshortFormula or Equation𝑌health 𝛽1 𝑋 𝛽2 𝑌 𝛽3 𝑍 𝑚𝑜𝑑𝑒𝑙: Predictors 𝒎𝑚𝑜𝑑𝑒𝑙 (𝑠𝑒𝑛𝑠𝑜𝑟 𝑑𝑎𝑡𝑎, 𝑎𝑐𝑡𝑖𝑣𝑖𝑡𝑦)5
Machine Learning: problem specific overviewType of LearningCategories of onMachineLearningDevelop predictive modelbased on bothinput and output dataUnsupervisedLearningGroup and interpret databased onlyon input data6
Predictive maintenance of turbofan engineSensor data from 100 engines of the same modelMotivation– Import and analyze historical sensor data– Train model to predict when failures willoccur– Deploy model to run on live sensor data– Predict failures in real timeData provided by NASA ic-data-repository/7
Use historical data to predict when failures will occurLiveHistoricalInitial Use/Prior MaintenanceEngine 1Recording StartsFailureMaintenance?Engine 2⁞Engine 100Cycles(Time)Engine X?Schedule Maintenance8
Preprocessing and classifying our input dataStart ofRecordedEngine LifeDataRecording StartsFailureEngine 1Engine 2Engine 3⁞Engine 100Cycle 0Cycles(Time)9
Agenda1.Machine learning – predictive maintenance2.Deep learning – build a digits classifier3.Predictive maintenance revisited – a deep learning approach10
Can you tell the difference? Japanese or Blenheim Spaniel?Blenheim SpanielJapanese SpanielSource: ILSVRC ImageNet dataset11
Why is deep learning so popular now?HumanAccuracySource: ILSVRC Top-5 Error on ImageNet12
Deep learning enablersAcceleration with GPUsMassive sets of labeled dataAvailability of state of the art models from experts13
Machine learning vs deep learningDeep learning performs end-to-end learning by learning features, representations and tasks directlyfrom images, text and soundDeep learning algorithms also scale with data – traditional machine learning saturatesMachine LearningDeep Learning14
Deep learning and neural networks Deep learning neural networksData flows through network in layersLayers provide transformation of data15
Convolutional neural networks Train “deep” neural networks on structured data (e.g. images, signals, text)Implements Feature Learning: Eliminates need for “hand crafted” featuresTrained using GPUs for performancecartruckvan bicycleInputConvolution ReLuPoolingConvolution ReLuFeature ication16
Two approaches for deep learning1. Train a Deep Neural Network from ScratchConvolutional Neural Network (CNN)Learned features𝟗𝟓% 𝟑% 𝟐%Lots of dataCar Truck Bicycle 2. Fine-tune a pre-trained model ( transfer learning)Fine-tune network weights Truck CarPre-trained CNNMedium amountsof dataNew Task17
Two deep learning approachesApproach 1: Train a Deep Neural Network from ScratchConvolutional Neural Network (CNN)Learned features𝟗𝟓% 𝟑% 𝟐%Car Truck Bicycle Recommended when:Training data1000s to millions of labeled imagesComputationCompute intensive (requires GPU)Training TimeDays to Weeks for real problemsModel accuracyHigh (can over fit to small datasets)18
Two deep learning approachesApproach 2: Fine-tune a pre-trained model (transfer learning)CNN trained on massive sets of data Learned robust representations of images from larger data set Can be fine-tuned for use with new data or task with small – medium size datasetsFine-tune network weights Truck CarPre-trained CNNNew DataNew TaskRecommended when:Training data100s to 1000s of labeled images (small)ComputationModerate computation (GPU optional)Training TimeSeconds to minutesModel accuracyGood, depends on the pre-trained CNN model19
Digits classificationWhat?A set of ‘handwritten’ digitsfrom 0-9 (c.f. MNIST)Why?An easy task for machinelearning beginnersHow many?60,000 training images10,000 test imagesBest results?99.79% accuracy 720
Agenda1.Machine learning – predictive maintenance2.Deep learning – build a digits classifier3.Predictive maintenance revisited – a deep learning approach21
Tackle time series data with LSTM How can we apply deep learning to time series data? One approach is to use Long Short-Term Memory (LSTM) neural networks These networks learn long-term temporal dependencies LSTMs work well with sequential input data, for example:– Time series– Text– Video22
LSTM classification networksFeature extractionFully ConnectedlongSoftmaxLSTMLSTM is used to extract time series featuresInput mediumshortClassification23
Shakespearean LSTM Text generated fromdeep LSTM network Network has learnedlong-term text style ofShakespeare E.g. punctuation,character-namecapitalization24
Thanks for listening!Today we’ve looked at:1.2.3.Machine learning – predictive maintenanceDeep learning – build a digits classifierPredictive maintenance revisited – a deep learning approachWhat to learn more/try it for yourself?Try MATLAB Deep Learning Onramp25
Deep learning performs end-to-end learning by learning features, representations and tasks directly from images, text and sound . Try MATLAB Deep Learning Onramp. Title: PowerPoint Presentation Author: Marene Salzman Keywords: Version 13.0 Created Date: 9/27/2017 4:23:42 PM .
