Deep Learning And Reinforcement Learning .

1y ago

31 Views

1 Downloads

2.86 MB

48 Pages

Report/dmca

Download PDF

Transcription

Deep Learning and ReinforcementLearning Workflows in A.I.Abhijit Bhattacharjee 2015 The MathWorks, Inc.1

Why MATLAB for Artificial Intelligence?2

Artificial IntelligenceDevelopment of computer systems to perform tasks that normallyrequire human intelligence3

A.I. ApplicationsObject ClassificationSpeech RecognitionPredictive MaintenanceSignal ClassificationAutomated DrivingStock Market Prediction4

Artificial Intelligence Development of computer systems to perform tasks thatnormally require human intelligenceMachine nK-meansNearestNeighborSVMReinforcement LearningGaussianMixture5

Machine Learning and Deep LearningMachine LearningUnsupervisedLearning[No Labeled Data]Clustering6

Machine Learning and Deep LearningMachine LearningUnsupervisedLearning[No Labeled Data]ClusteringSupervised Learning[Labeled Data]ClassificationRegression7

Machine Learning and Deep LearningMachine LearningUnsupervisedLearning[No Labeled Data]ClusteringSupervised Learning[Labeled Data]ClassificationSupervised learning typically involvesfeature extractionRegressionDeep LearningDeep learning typically does notinvolve feature extraction8

Deep Learning Subset of machine learning with automatic feature extraction– Learns features and tasks directly from data– More Data better modelMachineLearningDeep LearningDeepLearning9

Deep Learning Uses a Neural Network ArchitectureInputLayerHidden Layers (n)OutputLayer10

Deep Learning DatatypesImageNumericSignalText11

Deep Learning WorkflowPrepare DataData access andpreprocessingGround truth labelingTrain ModelModel design,HyperparametertuningModel exchangeacross frameworksHardwareaccelerated trainingDeployMultiplatform codegeneration (CPU, GPU)Edge deploymentEnterpriseDeployment12

Why MATLAB for A.I. Tasks?Increased productivity with interactive toolsGenerate simulation data for complex models and systemsEase of deployment and scaling to various platformsFull A.I. workflows that cannot be easilyreplicated by other toolchains13

Why MATLAB for A.I. Tasks?Increased productivity with interactive toolsLabelingTrainingModelExchangeFull A.I. workflows that cannot be easilyreplicated by other toolchains14

“I love to label andpreprocess my data” Said no engineer, ever.15

Labeling for deep learning is repetitive,tedious, and time-consuming but necessary16

Signal Labeler – annotate signals with labels/sublabels, export to workspace for trainingDefineLabelsInteractivelyLabel SignalsViewpropertiesof labels20

User Story – Veoneer (Autoliv) Automotive– Software and hardware for active safety,autonomous driving, occupant protection,and brake control Building radar sensor – checkaccuracy using LiDAR-basedverificationHuman analyzes hours of recordeddataUsed MATLAB to semi-automatelabeling and tracking of 3D LiDARpoint clouds.21

Manual Labeling for 25 events took over 20 minutes.After full automation with MATLAB’s tools, it took 5 minutes22

Use Deep Network Designer to Create Networks23

Transfer Learning with Pre-trained exNetResNet-50Import & Export Models Between FrameworksKeras-TensorflowImporterCaffe ModelImporterONNX ModelConverter24

Model Exchange with pen Neural Network ExchangeCore MLCNTK( )Caffe25

Why MATLAB for A.I. Tasks?Generate simulation data for complex models and systemsReinforcementLearningFull A.I. workflows that cannot be easilyreplicated by other toolchains27

Reinforcement Learning vs Machine Learning vs Deep LearningMachine LearningUnsupervisedLearning[No Labeled Data]ClusteringSupervised Learning[Labeled Data]ClassificationDeep LearningSupervised learning typically involvesfeature extractionRegressionDeep learning typically simplifiesfeature extraction28

Reinforcement Learning vs Machine Learning vs Deep LearningMachine LearningReinforcement learning:UnsupervisedLearning[No Labeled Data]ClusteringSupervised Learning[Labeled Data]ClassificationRegressionDeep LearningReinforcementLearning[Interaction Data]DecisionMaking Learning through trial & error[interaction]Controls It’s about learning abehavior or accomplishing atask29

What is Reinforcement Learning? What is ReinforcementLearning?– Type of machine learningthat trains an ‘agent’through repeatedinteractions with anenvironment How does it work?– Through a trial & errorprocess that uses a rewardsystem to maximizesuccess30

Reinforcement Learning enables the use of Deep Learning forControls and Decision Making ApplicationsRoboticsAutonomous drivingControlsA.I. Gameplay31

How Does Reinforcement Learning Work?STATEAGENTACTIONREWARDENVIRONMENT32

A Practical Example of Reinforcement LearningTraining a Self-Driving CarSTATEAGENTACTION REWARD Vehicle’s computer learns how to drive (agent)using sensor readings from LIDAR, cameras, (state)that represent road conditions, vehicle position, (environment)by generating steering, braking, throttle commands, (action)to avoid collisions and lane deviation (reward).ENVIRONMENTThe goal of Reinforcement learning is for the agent to find an optimal algorithm forperforming a task33

Deep Networks are commonly found in the agent, becausethey can model complex problems.AGENT Turn leftTurn rightBrakeAccelerate34

Reinforcement Learning WorkflowGeneratePrepare mulation-basedGround truth labelingdata generationTrain ModelReinforcement learningTraining agent toperform taskDeveloping rewardsystem to optimizeperformanceDeploymentMultiplatform codegeneration (CPU,GPU)Edge deploymentEnterpriseDeploymentSimulink –generate data fordynamic systems(planes, cars,robots, etc.)35

Why MATLAB and Simulink for Reinforcement Learning?Virtual models allow you to simulate conditionshard to emulate in the real world.36

Using MATLAB and Simulink for Reinforcement Learning Reinforcement learning is adynamic process Decision making problems– Financial trading, calibration, etc. Controls-based problems– Lane-keep assist, adaptive cruisecontrol, robotics, etc.37

Why MATLAB for A.I. Tasks?Increased productivity with interactive toolsGenerate simulation data for complex models and emsEase of deployment and scaling to various platformsFull A.I. workflows that cannot be easilyreplicated by other toolchains39

Deployment and Scaling for A.I.MATLABEmbedded DevicesEnterprise Systems40

Embedded Devices – Automatic Code GenerationMATLAB CodeAuto-generatedCode(C/C /CUDA)DeploymentTarget41

Deploying Deep Learning Models for sorRT &cuDNNLibrariesDeep LearningNetworksARMComputeLibrary42

GPU Coder is more than twice as fast as other compiled deeplearning s/secondTensorFlowXLA CompilePyTorch JITGPU CoderIntel Xeon CPU 3.6 GHz with NVIDIA Titan V GPU - NVIDIA libraries: CUDA – cuDNN - Frameworks: TensorFlow 1.13.0, PyTorch 1.1.0 – ResNet-50 pre-trained model43

Enterprise DeploymentCloudGPUMATLAB Parallel ComputingToolboxMATLAB ParallelServerMulticoreCPURun thousands of simulations in parallel with MATLAB Parallel Server to save hours oftraining time.44

Enterprise DeploymentDeployment to the cloud with MATLAB Compiler and MATLAB Production Server45

Musashi Seimitsu Industry Co.,Ltd.Detect Abnormalities in Automotive PartsMATLAB use in project: Preprocessing of captured images Image annotation for training Deep learning based analysis– Various transfer learning methods(Combinations of CNN models, Classifiers)– Estimation of defect area using Class Activation Map(CAM)– Abnormality/defect classificationAutomated visual inspection of 1.3 millionbevel gear per month Deployment to NVIDIA Jetson using GPU Coder46

Call to action Visit the Deep Learning Booth!Related upcoming talks:– AI Techniques for Signals, Time-series, and Text Data– Sensor Fusion and Tracking for Autonomous Systems– Deploying Deep Neural Networks to Embedded GPUs and CPUs48

Using MATLAB and Simulink for Reinforcement Learning Reinforcement learning is a dynamic process Decision making problems –Financial trading, calibration, etc. Controls-based problems –