WIXLIB

A Quick Introduction to Pythonand Machine LearningLing ZhangApr 17th, 20191

Objective Help on getting started with PythonIntroduce the basic concepts of neural network andhow it is trainedHelp on getting stated with machine learningprogramming using PyTorch (or Tensorflow)2

Part I – Introduction to Python What is PythonCommonly-used Python editorsData Structures in PythonControl Structures (Selection and Loop)Python FunctionsPython ClassesAn Python package example: skrfThere are many Python tutorials available online. Here is an sp3

Part I – Introduction to Python What is Python?Commonly-used Python editorsData Structures in PythonControl Structures in PythonPython FunctionsPython ClassesAn Python package example: skrf4

What is Python Python is a general purpose programming languagewith the following nice features:–––––An object-based high-level programming languageA free software released under an open-source licenseCross-platform: Windows, Linux/Unix, MaxOS X etcA very readable and concise language with clear non-verbosesyntaxA large variety of high-quality packages are available forvarious applications online and free to use Don’t reinvent the wheel!5

Part I – Introduction to Python What is PythonCommonly-used Python editorsData Structures in PythonControl Structures (Selection and Loop)Python FunctionsPython ClassesAn Python package example: skrf6

Python Terminal You can use the Pythonterminal for interactivecomputing in Python A preferred way is to useother editors like JupyterNotebook or PyCharm7

Jupyter Notebookhttps://jupyter.org/ A web-based interactivecomputational environment Can contain code, text,mathematics, plots and richmedia. Can work interactively Easy to visualize results Easy to share with others8

PyCharmhttps://www.jetbrains.com/pycharm/ PyCharm: easy to work with when working on a largeproject containing different Python files Can see intermediate results in debugging mode9

Part I – Introduction to Python What is PythonCommonly-used Python editorsData Structures in PythonControl Structures (Selection and Loop)Python FunctionsPython ClassesAn Python package example: skrf10

Overview of Data Structures in Python11

Lists, Tuple, Set, DictionaryUsually used in ‘for’ loopList:Tuple:Set:Dictionary:Attention: Python index begins with 0 !!!12

Use NumPy for Array/Matrix Oriented Computing13

NumPy OverviewFor a comprehensive reference of NumPy methods, refer ines.html14

Part I – Introduction to Python What is PythonCommonly-used Python editorsData Structures in PythonControl Structures (Selection and Loop)Python FunctionsPython ClassesAn Python package example: skrf15

If-elif-else Selection Structure in PythonIndentationis necessaryin Python!The Nearest Rule (if else ambiguity)The else/elif clause matches the nearest preceding if/elif clause in the sameblock.16

Iteration/Loop StructureWhile loopbreakcontinue17

Part I – Introduction to Python What is PythonCommonly-used Python editorsData Structures in PythonControl Structures (Selection and Loop)Python FunctionsPython ClassesAn Python package example: skrf18

Function Definition in Python General syntax: Lambda expression: Lambda functions are usually used temporally without aformal name.19

Part I – Introduction to Python What is PythonCommonly-used Python editorsData Structures in PythonControl Structures (Selection and Loop)Python FunctionsPython ClassesAn Python package example: skrf20

Python: Object-based Language Matlab is a function-based programming language. Differently, Python is a object-based language. We can define objectsunder classes. Then and the properties and methods (functions) can beeasily called. Also, for Python, many packages online well-written by others are freeto download and be installed.21

Python Class Definitionpropertiesmethods The init () function is called automatically every time the class is being used tocreate a new object. The self parameter is a reference to the current instance of the class, and is used toaccess variables that belongs to the class. It does not have to be named self , you can call it whatever you like, but it has to bethe first parameter of any function in the class. There are other properties of Python class such as inheritance, which will not becovered in detail here.22

Part I – Introduction to Python What is PythonCommonly-used Python editorsData Structures in PythonControl Structures (Selection and Loop)Python FunctionsPython ClassesAn Python package example: skrf23

RF Library in Python Found a library in Python called skrf, which is free toinstall and use. als/Networks.html#Introduction)This library can handle S-parameters and Z-parameters.24

Basic PropertiesImport the packageLoad PDN fileNow asus brd is a rf.Network object, and it can directly useall functions and properties within rf.Network class.25

PlottingObtain frequencyObtain Z-para26

Interpolation and ConcatenationDefine a Frequency objectDirectly call ‘interpolate’ function for frequency interpolationCombine Networks which cover different frequency ranges27

Connecting Multi-portsConnect two ports of two networkInner connect two ports of one network31-port 29-port28

Summary of Part I Python is a high-level programming language which isfree and open-source, and convenient to use.When you want some functions, try to search anddownload existing packages online first.29

Part II – Introduction to Machine Learning Brief introduction to neural network and deep learningHow neural network is trained – gradient descent andbackpropagationA simple tutorial of using Pytorch for machine learningprogramming in Python30

Part II – Introduction to Machine Learning Brief introduction to neural network and deep learningHow neural network is trained – gradient descent andbackpropagationA simple tutorial of using Pytorch for machine learningprogramming in Python31

Neural Network (NN) The structure of neural network is similar toneuron structure in human brain. The model has an input layer, an output layerand an arbitrary number of hidden layers. Each neuron can be regarded as a nonlinearfunction (activation function) of the weightedsum of its inputs. Pros: High accuracy, capacity and robustness. Can achieve complex non-linearity. Cons: Need large data size Need high computational ability. Black-box model and hard to understand. Applications: regression and classification. How can neural network be trained?32

Image Recognition For human beings, it is intuitive and easy to recognize the kid in the picture. Even ifthe position and environment changes, we don’t need to learn the concept again. But for computers, it is hard to recognize it if the picture changes for a little bit. And itis hard to build a formal rule about how a kid looks like. Intuitively, we can feel there exists some hierarchy or conceptual structure in thepicture, like from pixels to edges, and ears, mouth, nose, head and etc. Human beings33recognize pictures mainly by high-level features, not by individual pixels.

2D Convolution A window smaller than image sizeslides through the image and multiplywith each small area it goes through. We can detect small and meaningfulfeatures such as edges with kernelsthat occupy only tens or hundreds ofpixels. Using a couple of convolutionallayers in series helps extract morecomplex features. This also reduces the memoryrequirements of the model andimprove its statistical efficiency.34

Max Pooling Max pooling partitions the input image into a set of non-overlapping rectangles and,for each such sub-region, outputs the maximum. The pooling layer serves to progressively reduce the spatial size of the representation,to reduce the number of parameters and amount of computation in the network, andhence to also control overfitting. It is common to periodically insert a pooling layer between successive convolutionallayers in a CNN architecture.35

Convolutional Neural Network (CNN) A CNN consists of an input and an output layer, as well as multiple hidden layers. Thehidden layers of a CNN typically consist of convolutional layers, pooling layers, fullyconnected layers and normalization layers. A CNN contains so many parameters that it needs large amounts of data to be trainedwell. Usually at least millions of images are required to train a good image classifier likeGoogle. In deep learning, having more data is more significant than having a bettermodel!36

Part II – Introduction to Machine Learning Brief introduction to neural network and deep learningHow neural network is trained – gradient descent andbackpropagationA simple tutorial of using Pytorch for machine learningprogramming in Python37

Optimization Method – Gradient Descent When there is no closed form, we have to use computationalapproach such as gradient descent.38

Training Method: Backpropagation (1 / 2)Review: Chain rule of calculus: Gradient descent method to update weights: 𝜃𝑗 𝜃𝑗 𝜂 𝜃 𝐸(𝜃)𝑗where 𝜂 is learning rate, and 𝐸(𝜃) is loss function that needs to be minimized.Derivative of f(u):Loss function:t: actual output(logistic function)39

Training Method: Backpropagation (2 / 2)40

Part II – Introduction to Machine Learning Brief introduction to neural network and deep learningHow neural network is trained – gradient descent andbackpropagationA simple tutorial of using Pytorch for machine learningprogramming in Python41

Example of Pytorch – Linear Regression ob/master/tutorials/01-basics/linear regression/main.pyImport modulesDefine hyper-parametersTraining dataset42

Example of Pytorch – Linear Regression (2/3)Directly use nn.linear moduleUse nn.MSELoss() function as objective functionUse SGD optimizerCreate a Tensor from numpy arrayCalculate outputCalculate lossCalculate gradient and update parameters43

Example of Pytorch – Linear Regression (3/3)model has been trainedSave the trained model When using Python for machine learning applications, we don’tneed to rewrite the gradient descent and backpropagation algorithmby ourselves. We just need to use the existing modules in the packages such asPyTorch, Tensorflow, which makes it convenient.44

Example of Pytorch – CNNComplete code can be found at: ter/tutorials/02-intermediate/convolutional neural network/main.py Class inheritanceSelf-defined neural network45

Example of Tensorflow Can refer to this tutorial on ow-Examples46

Summary of Part II There are many available code example or template onGitHub, which can be downloaded freely.You just need to slightly modify the code to fit yourdesired applications.47

Questions?48

What is Python Python is a general purpose programming language with the following nice features: -An object-based high-level programming language -A free software released under an open-source license -Cross-platform: Windows, Linux/Unix, MaxOS X etc -A very readable and concise language with clear non-verbose syntax -A large variety of high-quality packages are available for