WIXLIB

Workshop: Parallel Computing With MATLAB (Part I)Raymond NorrisApplication Engineer, MathWorksMay 2021 2021 The MathWorks, Inc.1

Agenda Part I – Parallel Computing with MATLAB on the Desktop– Parallel Computing Toolbox– MATLAB Online Part II – Scaling MATLAB to magnitUDE– MATLAB Parallel Server– MATLAB Desktop2

Agenda Part I – Parallel Computing with MATLAB on the Desktop– Parallel Computing Toolbox– MATLAB Online Part II – Scaling MATLAB to magnitUDE– MATLAB Parallel Server– MATLAB Desktop3

Chatting Send to at least the Host,Presenter & PanelistsIdeally, send to All AttendeesParticipants4

Save time and tackle increasingly complex problems Reduce computation time by using available compute cores and GPUs Scale and accelerate workflows with minimal code changes Scale computations to clusters and clouds Focus on your engineering and research, not the computation5

Optimize your code before parallelizing for best performance Find bottlenecks with profilerTechniques for accelerating MATLAB algorithms and applications6

Optimize your code before parallelizing for best performance Implement effective programming techniquesTechniques for accelerating MATLAB algorithms and applications7

Optimize your code before parallelizing for best performance (Advanced) Replace code with MEX functionsTechniques for accelerating MATLAB algorithms and applications8

MATLAB has built-in multithreadingMulti-core CPUMATLABMATLAB multicore9

Scale further with parallel computingMulti-core CPUMATLABParallel Computing ToolboxMATLAB multicore10

Run multiple iterations by utilizing multiple CPU coresMATLABParallel Computing ToolboxWorkers11

Scaling MATLAB applications and Simulink simulationsCommon programming constructsGreater ControlEase of UseAutomatic parallel support in toolboxesAdvanced programming constructs12

Scaling MATLAB applications and Simulink simulationsCommon programming constructs(parfor, parfeval, parsim, )Greater ControlEase of UseAutomatic parallel support in toolboxesAdvanced programming constructs13

Parallelism using parfor Run iterations in parallelExamples: parameter sweeps, Monte Carlo simulationsMATLABWorkersTimeTime14

Parallelism using parfora zeros(5, 1);a zeros(5, 1);b pi;b pi;for i 1:5a(i) i b;enddisp(a)parfor i 1:5a(i) i b;enddisp(a)MATLABWorkers15

Parallelize for loops with independent iterationsMATLABWorkersa zeros(10, 1);b pi;parfor i 1:10a(i) i b;enddisp(a)16

Optimizing parforUse moreKeep smallTypeCategorysliced inputinputbroadcastinputreductionoutputsliced outputoutputlooponly exist on workertemporaryonly exist on workerTroubleshooting variables in parfor-loops17

Parallelism using parfor18

Execute additional code as iterations completefunction a parforWaitbar Send data or messages from parallelworkers back to the MATLAB clientRetrieve intermediate values and trackcomputation progressD parallel.pool.DataQueue;h waitbar(0, 'Please wait .');afterEach(D, @nUpdateWaitbar)N 200;p 1;parfor i 1:Na(i) max(abs(eig(rand(400))));send(D, i)endfunction nUpdateWaitbar( )waitbar(p/N, h)p p 1;endend19

Execute functions in parallel asynchronously using parfevalfetchNextOutputsMATLABWorkers Asynchronous execution on parallel workersUseful for “needle in a haystack” problemsfor idx 1:10f(idx) parfeval(@magic,1,idx);endfor idx 1:10[completedIdx,value] fetchNext(f);magicResults{completedIdx} value;end20

Run multiple simulations in parallel with parsimWorkersTime TimeRun independent Simulinksimulations in parallel usingthe parsim function21

Scaling MATLAB applications and Simulink simulationsCommon programming constructsGreater ControlEase of UseAutomatic parallel support in toolboxesAdvanced programming constructs(spmd, labBarrier, )22

Leverage NVIDIA GPUs without learning CUDAMATLABParallel Computing Toolbox23

Leverage your GPU to accelerate your MATLAB code Ideal Problems– massively parallel and/orvectorized operations– computationally intensive 500 GPU-supportedfunctions Use gpuArray andgather to transfer databetween CPU and GPUMATLAB GPU computing24

Parallel computing on your desktop, clusters, and cloudsGPUGPUMulti-core CPUMulti-core CPUMATLABParallel Computing Toolbox MATLAB Parallel ServerPrototype on the desktopIntegrate with infrastructureAccess directly through MATLAB25

Scale to clusters and cloudsWith MATLAB Parallel Server, you can Change hardware with minimal code change Submit to on-premise or cloud clusters Support cross-platform submission– Windows client to Linux cluster26

Interactive parallel computingLeverage cluster resources in MATLAB parpool('cluster', 3); myscriptMATLABParallel Computing Toolboxmyscript.m:a zeros(5, 1);b pi;parfor i 1:5a(i) i b;end27

batch simplifies offloading computationsSubmit MATLAB jobs to the cluster job lel Computing Toolboxpool28

batch simplifies offloading simulationsSubmit Simulink jobs to the clusterjob batchsim(in,'Pool',3);parsimworkerMATLABParallel Computing Toolboxpool29

Big data workflowsACCESS DATAMore data and collectionsof files than fit in memoryDEVELOP & PROTOTYPE ON THE DESKTOPAdapt traditional processing tools orlearn new tools to work with Big DataSCALE PROBLEM SIZETo traditional clusters and BigData systems like Hadoop30

tall arrays New data type designed for data that doesn’t fit into memoryLots of observations (hence “tall”)Looks like a normal MATLAB array– Supports numeric types, tables, datetimes, strings, etc.– Supports several hundred functions for basic math, stats, indexing, etc.– Statistics and Machine Learning Toolbox support(clustering, classification, etc.)Working with tall arrays31

distributed arrays Distribute large matrices across workers running on a clusterSupport includes matrix manipulation, linear algebra, and signal processing Several hundred MATLAB functions overloaded for distributed arrays MATLABParallel Computing Toolbox11 26 4115 30 4520 35 5012 27 4216 31 4621 36 5113 28 4317 32 4722 37 52MATLAB Parallel Server32

tall arrays vs. distributed arrays tall arrays are useful for out-of-memory datasets with a “tall” shape– Can be used on a desktop, cluster, or with Spark/Hadoop– Low-level alternatives are MapReduce and MATLAB API for Spark distributed arrays are useful for in-memory datasets on a cluster– Can be any shape (“tall”, “wide”, or both)– Low-level alternative is SPMD gop (Global operation across all workers)33

Resources MATLAB Documentation– MATLAB Advanced Software Development Performance and Memory– Parallel Computing Toolbox Parallel and GPU Computing Tutorials– nd-gpu-computing-tutorials97719.html Parallel Computing with MATLAB– ting.html34

MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See www.mathworks.com/trademarks for a list of additional trademarks. Otherproduct or brand names may be trademarks or registered trademarks of their respective holders. 2020 The MathWorks, Inc. 2021 The MathWorks, Inc.35

Download Instructions e-aa71-1da85bf81870– Click on Add to my Files– Click Copy Folder– Log into MATLAB Drive with MathWorks Account PC 3442250– Enter MathWorks email address– Click Next– Click Access MATLAB Online (maybe prompted to sign-in again)– In Current Folder, double click on Parallel Computing Workshop– Right click on startWorkshop, select Run36

Start Workshop37

Chatting Send to at least the Host,Presenter & PanelistsIdeally, send to All Attendees38

Agenda Part I – Parallel Computing with MATLAB on the Desktop– Parallel Computing Toolbox– MATLAB Online Part II – Scaling MATLAB to magnitUDE– MATLAB Parallel Server– MATLAB Desktop Next Week– Thursday, May 27th 2-5pm– MATLAB / PCT installed (R2020b or R2021a)– magnitUDE account39

Required: MATLAB & Parallel Computing Toolbox40

Part I -Parallel Computing with MATLAB on the Desktop -Parallel Computing Toolbox -MATLAB Online Part II -Scaling MATLAB to magnitUDE -MATLAB Parallel Server -MATLAB Desktop Next Week -Thursday, May 27th 2-5pm -MATLAB / PCT installed (R2020b or R2021a) -magnitUDE account