WIXLIB

Introduction to Deep Learning in Signal Processing &Communications with MATLABDr. Amod AnandkumarPallavi KarApplication Engineering Group, Mathworks India 2019 The MathWorks, Inc.1

Different Types of Machine LearningType of Machine LearningCategories of AlgorithmsClassification Output is a choice between classes(True, False) (Red, Blue, Green)SupervisedLearningMachineLearningDevelop predictivemodel based on bothinput and output dataUnsupervisedLearningRegressionClustering Output is a real number(temperature, stock prices) No output - find natural groups andpatterns from input data onlyDiscover an internalrepresentation frominput data only2

What is Deep Learning?3

Deep learning is a type of supervised machine learning in which amodel learns to perform classification tasks directly from images, text,or sound.Deep learning is usually implemented using a neural network.The term “deep” refers to the number of layers in the network—themore layers, the deeper the network.4

Why is Deep Learning So Popular Now?HumanAccuracySource: ILSVRC Top-5 Error on ImageNet5

Vision applications have been driving the progress in deep learningproducing surprisingly accurate systems6

Deep Learning success enabled by: Labeled public datasets Progress in GPU for acceleration World-class models andconnected ETRAINED MODELTensorFlowKerasIMPORTERONNX ConverterMODEL CONVERTERInception-v3MODELS7

Deep Learning is VersatileMATLAB Examples Available Here8

Many Network Architectures for Deep LearningSeries NetworkDirected AcyclicGraph NetworkRecurrent Networkand more(GAN, DQN, )AlexNetYOLOResNetR-CNNLSTM9

Convolutional Neural NetworksEdgesShapesObjectsA lot of datais requiredConvolution layersFully-connectedlayers10

Deep Learning Inference in 4 Lines of Code net alexnet; I imread('peacock.jpg') I1 imresize(I,[227 227]); classify(net,I1) ans categoricalpeacock11

Understanding network behavior using visualizationsFilters Deep DreamActivations Custom visualizations– Example: Class Activation Maps(See blog post)12

Visualization Technique – Deep DreamdeepDreamImage(.net, ‘fc8', channel,'NumIterations', 50, .'PyramidLevels', 4,.'PyramidScale', 1.25);Synthesizes images that strongly activatea channel in a particular layerExample Available Here13

What is Training?ImagesNetworkTrainingLabelsLarge Data SetTrained DeepNeural NetworkDuring training, neural network architectures learn features directlyfrom the data without the need for manual feature extraction14

What Happens During Training?AlexNet ExampleLabelsTraining DataLayer weights are learnedduring training15

Visualize Network Weights During TrainingAlexNet ExampleLabelsTraining DataFirst ConvolutionLayer16

Visualize Features Learned During TrainingAlexNet ExampleSample Training DataFeatures Learned by Network17

Visualize Features Learned During TrainingAlexNet ExampleSample Training DataFeatures Learned by Network18

Deep Learning WorkflowACCESS AND EXPLOREDATALABEL AND PREPROCESSDATADEVELOP PREDICTIVEMODELSINTEGRATE MODELS WITHSYSTEMSFilesData iningDesktop AppsDatabasesLabeling AutomationHyperparameter TuningEnterprise Scale SystemsSensorsImport ReferenceModelsNetwork VisualizationEmbedded Devices andHardware19

Deep Learning ChallengesData Handling large amounts of data Labeling thousands of signals, images & videos Transforming, generating, and augmenting data (for different domains)Training and Testing Deep Neural Networks Accessing reference models from research Understanding network behaviour Optimizing hyperparameters Training takes hours-daysRapid and Optimized Deployment Desktop, web, cloud, and embedded hardware20

Working with Signal ormFully ConnectedLayersTrainConvolutionalNetwork (CNN)Trained ModelTrained Model21

Long Short Term Memory Networks from RNNs Recurrent Neural Network that carries a memory cell throughout the processSequence Problems – Long term dependency does not work well with RNNsc0C1Ct22

RNN to LSTMCell stateInput gateForget gateUpdateOutput gate23

Example: Speaker Gender Recognition Using Deep LearningLSTM Network for Audio based Speaker ClassificationaudioDatastoreTraining Data1. TrainMozilla CommonVoice Dataset Male FemaleTime-varyingfeatures:- MFCC- Pitch- .Test Data2. Predict Male Female24

Some audio and speech applications following CNN workflows27

https://www.isca-speech.org/archive/interspeech 2015/papers/i15 1478.pdf28

Solution2: Speech Command Recognition with Deep Learning(MATLAB)Raw DataPreprocessedDataModels and AlgorithmsIntegrated AnalyticSystems.29

43

44

45

46

Data augmentation allows training more advanced networks andgenerating more robust modelsOriginalDatasetSimulate:-lower voicevarying level of environmental interferencerecording hardware effectsdistortions due to use of different microphonesdifferent speaker vs microphone positions or room sizes inenclosed environments.50

Serial% Cycle continuously and automatically through files in datastoremfccFile zeros(numel(ads.Files),numMfcc)while hasdata(ads)[data,info] read(ads);% do something with dataendParallel% Partition and explicit parfor parallel processing patternN numpartitions(ads);parfor index 1:Nsubads partition(ads,N,index);while hasdata(subads)data read(subads);% do something with dataendend% Tall array and "implicit" parallel processing patternT tall(ads);dcOffsets cellfun(@(x) mean(x), T,'UniformOutput',false);gather(dcOffsets);51

Package Speech Recognition App52

53

BREAK54

Deep Learning ChallengesDataNot a deep learning expert Handling large amounts of data Labeling thousands of signals, images & videos Transforming, generating, and augmenting data (for different domains)Training and Testing Deep Neural Networks Understanding network behavior Accessing reference models from research Optimizing hyperparameters Training takes hours-daysRapid and Optimized Deployment Desktop, web, cloud, and embedded hardware55

BREAK56

57

Speech 2 text intro58

59

60

61

62

Audio Labeler Work on collections ofrecordings or record new audiodirectly within the app Navigate dataset and playbackinteractively Define and apply labels to– Entire files– Regions within files Import and export audio folders,label definitions and datastores63

Apps for Data Labeling64

Label Images Using Image Labeler App65

Accelerate Labeling With Automation AlgorithmsLearn More66

Perform Bootstrapping to Label Large DatasetsImagesVideosTrainNetworkLabeling AppsImproveNetworkMoreGround TruthGround TruthPropose LabelsAutomationAlgorithm67

Example – Semantic Segmentation Available HereClassify pixels into 11 classes– Sky, Building, Pole, Road,Pavement, Tree, SignSymbol,Fence, Car, Pedestrian, Bicyclist CamVid datasetBrostow, Gabriel J., Julien Fauqueur, and Roberto Cipolla. "Semantic object classes invideo: A high-definition ground truth database." Pattern Recognition Letters Vol 30, Issue2, 2009, pp 88-97.68

Example: Singing Voice Separation Source separationBased on U-Net architecture69

Synthetically generating labeled data70

Modulation Classification with Deep Learning Generate synthetic modulated signals Apply channel impairments Train a CNN to classify modulation types71

Deep Learning ChallengesData Handling large amounts of data Labeling thousands of signals, images & videos Transforming, generating, and augmenting data (for different domains)Training and Testing Deep Neural Networks Understanding network behavior Accessing reference models from research Optimizing hyperparameters Training takes hours-daysRapid and Optimized Deployment Desktop, web, cloud, and embedded hardware72

Transfer Learning8 lines of MATLAB CodeLoad Training Data12Load Pre-TrainedNetwork345Replace Final Layers6Train Network onNew Data78%% Create a datastoreimds e,'LabelSource','foldernames');num classes numel(unique(imds.Labels));%% Load Referece Networknet alexnet;layers net.Layers%% Replace Final Layerslayers(end-2) fullyConnectedLayer(num classes,'Name',[‘fc5']);layers(end) classificationLayer('Name','classOut');%% Set Training Options & Train NetworktrainOpts trainingOptions('sgdm’,.'MiniBatchSize', 64);net trainNetwork(imds, layers, trainOpts);73

Tune Hyperparameters to Improve TrainingMany hyperparameters depth, layers, solver options,learning rates, regularization, Techniques Parameter sweep Bayesian optimization74

Keras-Tensorflow Importer75

Model Exchange and Co-executionMATLABModel ExchangePythonCo-execution76

Model Exchange With Deep Learning re MLChainerCognitiveToolkitTensorFlowKerasCaffeONNX Open Neural Network Exchange Format77

Interoperate With Deep Learning Frameworks – Use itVisualizationandDebuggingImportCore MLChainerONNX Open Neural Network Exchange Format78

Model Exchange With Deep Learning FrameworksCaffe Model Importer importCaffeLayers importCaffeNetworkTensorFlow-Keras Model Importer importKerasLayers importKerasNetworkONNX Converter importONNXNetwork exportONNXNetworkONNX Converter79

Model Exchange and Co-executionMATLABPythonModel ExchangeCo-execution80

MATLAB-Python Co-Execution – The ‘How’Call Python file fromMATLABCall TensorFlow commands fromMATLABTF code.py81

MATLAB-Python Co-Execution – Method A1.2.Copy the code into a .PY fileWrap entry point in a functionimport tensorflow as tffrom tf import kerasdef myTFCode():for x in y:a.foo()def foo(x):return x 1TF code.py3.Add module to Python path and then callfrom MATLAB via:py.myModule.myTFCode();82

Deep Learning on CPU, GPU, Multi-GPU and ClustersH O W T O TA R G E T ?SingleCPUSingle CPUSingle GPUSingle CPU, Multiple GPUsOn-prem server withGPUsCloud GPUs(AWS)83

MATLAB Containers for NVIDIA GPU Cloud & DGX84

Deep Learning ChallengesData Handling large amounts of data Labeling thousands of signals, images & videos Transforming, generating, and augmenting data (for different domains)Training and Testing Deep Neural Networks Accessing reference models from research Understanding network behaviour Optimizing hyperparameters Training takes hours-daysRapid and Optimized Deployment Desktop, web, cloud, and embedded hardware85

Deploying Deep Learning ApplicationEmbedded HardwareCodeGenerationApplicationlogicDesktop, Web, CloudApplicationDeployment86

MATLAB Production Server is an application server that publishesMATLAB code as APIs that can be called by other applicationsAnalytics DevelopmentMATLABCompiler SDKPackageMATLABDeployCode / testAccessEnterpriseApplicationIntegrateMATLAB Production ServerData sources /applicationsWorker processes RequestBrokerScale and secureMobile / WebApplication3rd partydashboard87

MATLAB support for Cloud88

89

IAAS to PAAS90

Automatic Cluster Creation91

Get Public IP for access92

Server Machine VM access MPS console endpoint: https://xxx.xx.xx.xx93

Deploying Deep Learning ApplicationEmbedded HardwareCodeGenerationApplicationlogicDesktop, Web, CloudApplicationDeployment94

Solution- GPU/MATLAB Coder for Deep Learning DeploymentTarget LibrariesGPU CoderApplicationlogicTensorRT mputeLibrary95

Deep Learning Deployment WorkflowsINFERENCE ENGINE DEPLOYMENTTrainedDNNINTEGRATED APPLICATION DEPLOYMENTPreprocessingcnncodegenPortable target codePostprocessingcodegenPortable target code96

Single Image Inference on Titan Xp using cuDNNTensorFlowMXNet(1.2.1)GPU CoderPyTorchIntel Xeon CPU 3.6 GHz - NVIDIA libraries: CUDA9 - cuDNN 7(1.8.0)(R2018b)(0.3.1)97

NVIDIA Hardware Support Package (HSP)Simple out-of-box targeting for NVIDIA boards:INFERENCE ENGINE DEPLOYMENTJetsonDrive platformTrainedDNNcnncodegenPortable target code98

Summary- MATLAB Deep Learning FrameworkDEPLOYMENTIntelMKL-DNNLibraryAccess Data Manage large image sets Automate image labeling Easy access to modelsDesign Train Acceleration with GPU’sScale to clustersNVIDIATensorRT &cuDNNLibrariesARMComputeLibrary99

Summary Create and validate Deep learning models Automate ground truth labeling Access large amount of data from cluster/cloud Interoperability with Deep learningframeworks Visualization and hyperparameter tuning Seamlessly scale training to GPUs, clusters and cloud Deployment on embedded targets and web services100

Available Here 101

102

103

4 Deep learning is a type of supervised machine learning in which a model learns to perform classification tasks directly from images, text, or sound. Deep learning is usually implemented using a neural network. The term “deep” refers to the number of layer