Transcription
Master Class:Deep Learning for SignalsAbhijit BhattacharjeeSenior Application EngineerMathWorks 2015 The MathWorks, Inc.1
Music Genre Classification2
3
4
AgendaWhy deep learning?Deep learning with signal data(Demo) Speech Command Recognition(Demo) LSTM NetworksEnabling Features in MATLABDeploying deep learning5
What is Deep Learning?6
Deep learning is a type of machine learning in which a model learns toperform tasks directly from image, time-series or text data.Deep learning is usually implemented using a neural network architecture.7
What is Deep Learning? Subset of machine learning with automatic feature extraction– Can learn features directly from data– More Data better modelMachineLearningDeep LearningDeepLearning8
Types of DatasetsTabularDataTime Series/SignalML or LSTMLSTM or CNNImageDataCNN9
Image Example: Object recognition using deep learningTraining(GPU)Millions of images from 1000different categoriesPredictionReal-time object recognition usinga webcam connected to a laptop10
Signals Example: Analyzing signal data using deep learningSignal Classification using LSTMsSpeech Recognition using CNNs11
Deep Learning WorkflowACCESS AND EXPLOREDATALABEL AND PREPROCESSDATADEVELOP PREDICTIVEMODELSINTEGRATE MODELS WITHSYSTEMSFilesData iningDesktop AppsDatabasesLabeling AutomationHyperparameter TuningEnterprise Scale SystemsSensorsImport ReferenceModelsNetwork VisualizationEmbedded Devices andHardware12
How Does a Convolutional Neural Network Work?EdgesShapesObjects13
Speech Command RecognitionUsing Convolutional Neural NetworksNoUpRightOff14
Example: Speech Command Recognition Using DeepLearningCNN Network for Audio ClassificationaudioDatastore1. TrainStopUpYes none auditorySpectrogramGoogle speechcommand dataset2. PredictaudioDeviceReader"Up"auditorySpectrogram15
CommandRecognition Demo16
17
18
19
20
21
22
23
24
25
26
https://www.isca-speech.org/archive/interspeech 2015/papers/i15 1478.pdf27
28
29
30
31
32
33
34
35
MATLAB Based Algorithm Wins the 2017 PhysioNet/CinCChallenge to Automatically Detect Atrial FibrillationChallengeDesign an algorithm that uses machine learning to detectatrial fibrillation and other abnormal heart rhythms innoisy, single-lead ECG recordingsSolutionUse MATLAB to analyze ECG data, extract features usingsignal processing and wavelet techniques, and evaluatedifferent machine learning algorithms to train andimplement a best-in-class classifier to detect AFResults First place in PhysioNet/CinC Challenge achieved ECG data visualized in multiple domains Feature extraction accelerated with parallelprocessingBlock diagram for Black Swan’s atrial fibrillation detectionalgorithm.“I don’t think MATLAB has any strong competitors forsignal processing and wavelet analysis. When you add inits statistics and machine learning capabilities, it’s easy tosee why nonprogrammers enjoy using MATLAB,particularly for projects that require combining all thesemethods.”- Ali Bahrami Rad, Aalto UniversityLink to user story36
AgendaWhy deep learning?Deep learning with signal data(Demo) Speech Command Recognition(Demo) LSTM NetworksEnabling Features in MATLABDeploying deep learning37
I was born in France. I speak ?38
Recurrent Neural Networks Take previous data into account when making new predictionsSignals, text, time series39
I was born in France [2000 words] I speak ?40
Long Short Term Memory Networks RNN that carries a memory cell throughout the processSequence Problemsc0C1Ct41
LSTMDemo42
Time Series Classification (Human Activity Recognition)Long short-term memory networks Dataset is accelerometer and gyroscopesignals captured with a smartphoneData is a collection of time series with 9channelssequence to oneDeepLearning43
44
AgendaWhy deep learning?Deep learning with signal data(Demo) Speech Command Recognition(Demo) LSTM NetworksEnabling Features in MATLABDeploying deep learning45
Deep Learning on CPU, GPU, Multi-GPU and ClustersHOWSingleCPUTO TA R G E T ?Single CPUSingle GPUSingle CPU, Multiple GPUsOn-prem server withGPUsCloud GPUs(AWS)46
Audio Datastoreads audioDatastore('.\Dataset', .'IncludeSubfolders',true, .'FileExtensions','.wav', .'LabelSource','foldernames')ads Programmatic interface to largecollections of audio files (Optional) auto-generation oflabels from folder names Label-based indexing andpartitioning Random file sampling Automatic sequential file reading Parallel access for preprocessing, feature extraction, ordata augmentationDatastore with properties:Files: {' .\Temp\speech commands v0.01\ backgr' .\Local\Temp\speech commands v0.01\' .\Local\Temp\speech commands v0.01\. and 64724 more}Labels: [ background noise ; background noise ;ReadMethod: 'File'OutputDataType: 'double'ads getSubsetDatastore(ads,isCommand isUnknown);countEachLabel(ads)ans 11 2 sTrain,adsValidation,adsTest] splitData(ads,datafolder);47
“I love to label andpreprocess my data” Said no engineer, ever.48
Audio Labeler Work on collections ofrecordings or record new audiodirectly within the app Navigate dataset and playbackinteractively Define and apply labels to– Entire files– Regions within files Import and export audio folders,label definitions and datastores49
50
Signal Analyzer AppAnalyze multiple signals in time, frequency and time-frequency domains Navigate through signals Extract regions of interest Generate MATLAB scripts51
Feature Extraction: Time-Frequency AnalysisReassigned spectrogramand synchrosqueezingEmpirical mode decompositionPersistent spectrumKurtogramWavelet scalogramInstantaneous frequencyInstantaneous BWConstant Q transformHilbert-Huang transform52
Enabling Features Summary(For Audio Applications) Audio System Toolbox(For Signal Applications) Signal Processing ToolboxGroundTruthLabelingRead ctPrototype!Record &storeNeural Network TBStats & ML TBads Datastore with properties:Files: {' .\Temp\speech commands v0.01\ background noise \doing the dishes.wav';' .\Local\Temp\speech commands v0.01\ background noise \dude miaowing.wav';' .\Local\Temp\speech commands v0.01\ background noise \exercise bike.wav'. and 64724 more}Labels: [ background noise ; background noise ; background noise . and 64724 morecategorical]ReadMethod: 'File'OutputDataType: 'double'53
AgendaWhy deep learning?Deep learning with signal data(Demo) Speech Command Recognition(Demo) LSTM NetworksEnabling Features in MATLABDeploying deep learning54
Deploying Deep Learning Models for InferenceNVIDIATensorRT ntelMKL-DNNLibrary55
GPU Coder Fills a Gap in the Deep Learning SolutionTrainingInferenceAccess DataPreprocessSelect NetworkTrainDeployImage rVision56
With GPU Coder, MATLAB is fastGPU Coder is fasterthan TensorFlow,MXNet and PytorchTensorFlowMXNetGPU CoderPyTorchIntel Xeon CPU 3.6 GHz - NVIDIA libraries: CUDA9 - cuDNN 7 - Frameworks: TensorFlow 1.8.0, MXNet 1.2.1, PyTorch 0.3.157
MathWorkscan help you do Deep LearningFree resources Guided evaluations with aMathWorks deep learningengineerProof-of-concept projectsDeep learning hands-onworkshopSeminars and technical deepdivesDeep learning onramp courseMore options Consulting servicesTraining coursesTechnical supportAdvanced customer supportInstallation, enterprise, and clouddeploymentMATLAB for Deep Learning58
Thank you!59
Extra Slides60
Machine learning vs. deep learningDeep learning performs end-to-end learning by learning features, representations and tasks directlyfrom images, text, sound, and signalsMachine LearningMachineLearningDeep LearningDeepLearningSubset of machinelearning with automaticfeature extraction61
Feature Extraction: Spectral AnalysisWelch periodogramBW measurementsOctave spectrumHarmonic distortionLomb periodogramSpectral statistics62
Features Extraction and Signal SegmentationFor Machine and Deep Learning Mel Frequency Cepstral Coefficients (MFCC) Pitch Voice Activity Detection63
Mel Frequency Cepstral Coefficients (MFCC) Variation of "perceptually-adjusted" spectrum content over time Most common voice and speech features for Machine Learning applications Option of:– mfcc function,– cepstralFeatureExtractor System object– Cepstral Feature Extractor block (Simulink) Returns:– MFCC– "Delta" ( mfcc[n]-mfcc[n-1])– "Delta delta" ( delta[n]-delta[n-1])64
Pitch Extraction Estimate audio pitch over time Popular feature for machine learning invoice and speech processing Choice of 5 different popular algorithms– 'NCF' – Normalized Correlation Function– 'PEF' – Pitch Estimation Filter– 'CEP' – Cepstrum Pitch Determination– 'LHS' – Log-Harmonic Summation– 'SRH' – Summation of Residual Harmonics[ft, idx] pitch(audioIn, fs);65
Voice Activity Detection (VAD) Standard algorithm to segment voice and speech signals voiceActivityDetector System object for MATLAB Voice Activity Detector block for Simulink vad voiceActivityDetectorvad voiceActivityDetector with 'Hann'[]0.20000.100066
Images / SecTensorRT speeds up inference for TensorFlow and GPU CoderTensorFlowGPU CodercuDNNTensorRTIntel Xeon CPU 3.6 GHz - NVIDIA libraries: CUDA9 - cuDNN 7 - Frameworks: TensorFlow 1.6.0, MXNet 1.1.0, MATLAB 18a67
Images / SecGPU Coder with TensorRT provides best performance acrossall batch sizesGPU Coder/TensorRTGPU CoderTensorFlow/TensorRTMXNetTensorFlowBatch Size68
Images / SecEmbedded: GPU Coder also fastNVIDIA libraries: CUDA9 - cuDNN 7 – Jetpack 3.2.1 - Frameworks: TensorFlow 1.6.0, MXNet 1.1.0, MATLAB 18a69
Example: Speech Denoising with Deep LearningCNN & Fully Connected Networks for 2-D Regression1. Train 2. Predict70
Example 3: Speech Denoising with Deep LearningCNN & Fully Connected Networks for 2-D Regression71
Speech DenoisingDemo72
Extras74
Summary and Release Timeline75
77
Design an algorithm that uses machine learning to detect atrial fibrillation and other abnormal heart rhythms in noisy, single-lead ECG recordings Solution Use MATLAB to analyze ECG data, extract features using signal processing and wavelet techniques, and evaluate different machine learning algorithms to train and
