WIXLIB

UNIVERSITY OF CENTRAL FLORIDATensorFlowTutorialby Astrid JacksonUNIVERSITY OF CENTRAL FLORIDA

UNIVERSITY OF CENTRAL FLORIDATENSORFLOW Tensors: n-dimensional arrays Vector: 1-D tensor Matrix: 2-D tensor Flow: data flow computation framework A sequence of tensor operations2

3UNIVERSITY OF CENTRAL FLORIDASIMPLE FULLY CONNECTED NETWORK𝑾𝟏,𝟏𝒙𝟏𝒚𝟏 �𝟐 ��weights 𝒃𝟑𝒚𝟑biasesoutputs

4UNIVERSITY OF CENTRAL FLORIDANETWORK AS MATRIX OPERATIONS𝒚𝟏𝒚𝟐𝒚𝟑 ��𝟑,𝟐𝑾𝟑,𝟑 𝒙𝟏𝒙𝟐𝒙𝟑𝒃𝟏 𝒃𝟐𝒃𝟑

5UNIVERSITY OF CENTRAL FLORIDANETWORK WITH TENSORFLOW𝒚𝟏𝒚𝟐𝒚𝟑 ��𝟑,𝟐𝑾𝟑,𝟑 𝒙𝟏𝒙𝟐𝒃𝟏 𝒙𝟑import tensorflow as tflogits tf.matmul(x, w) by tf.nn.softmax(logits)𝒃𝟐𝒃𝟑

6UNIVERSITY OF CENTRAL FLORIDADEFINE Variable( initial value ,name optional name )import tensorflow as tfw tf.Variable( tf.random normal([3, 3]), name 'w' )b tf.Variable( tf.zeros([3]), name 'b' )y tf.nn.softmax( tf.matmul( x, w ) b )

7UNIVERSITY OF CENTRAL FLORIDATENSORFLOWCode defines data flow graphEach variable corresponds to anode in the graph, not the resultSoftmaxVariableAddVariableMatMul𝑥import tensorflow as tfw tf.Variable( tf.random normal([3, 3]), name 'w' )b tf.Variable( tf.zeros([3]), name 'b' )y tf.nn.softmax( tf.matmul( x, w ) b )

8UNIVERSITY OF CENTRAL FLORIDACOMPUTATION AS DATA FLOW sGraph of Nodes, also called operations (ops)

9UNIVERSITY OF CENTRAL FLORIDADATA FLOW GRAPH rgetsEdges are N-dimensional arrays: Tensors

10UNIVERSITY OF CENTRAL FLORIDADATA FLOW GRAPH (BACKWARD GRAPH ANDUPDATES)Backward graph and update are added automatically to graph'Biases' are variable Some ops compute gradients- updates biasesbiases learningrateAdd Mul-

11UNIVERSITY OF CENTRAL FLORIDATENSORFLOWFetchGraphSoftmaxCode defines data flow graph How to execute the graph?VariableSoftmaxVariableMatMulSession Manage resource for graphexecutionimport tensorflow as tfsess tf.Session()w tf.Variable( tf.random normal([3, 3]), name 'w' )b tf.Variable( tf.zeros([3]), name 'b' )y tf.nn.softmax( tf.matmul( x, w ) b )print sess.run(y)𝑥

12UNIVERSITY OF CENTRAL FLORIDAINITIALIZE VARIABLEFetchGraphSoftmaxVariable is an empty node Fill in the content ofVariable nodeVariableSoftmaxVariableMatMulimport tensorflow as tfsess tf.Session()w tf.Variable( tf.random normal([3, 3]), name 'w' )b tf.Variable( tf.zeros([3]), name 'b' )y tf.nn.softmax( tf.matmul( x, w ) b )sess.run( tf.initialize all variables() )print sess.run(y)𝑥

13UNIVERSITY OF CENTRAL FLORIDAPLACEHOLDERFetchGraphSoftmaxHow about x?placeholder( data type ,shape optional shape ,name optional name )VariableSoftmaxVariableMatMulimport tensorflow as tfsess tf.Session()x tf.placeholder( "float", [1, 3] )w tf.Variable( tf.random normal([3, 3]), name 'w' )b tf.Variable( tf.zeros([3]), name 'b' )y tf.nn.softmax( tf.matmul( x, w ) b )sess.run( tf.initialize all variables() )print sess.run(y, feed dict {x: np.array([[1., 2., 3.]])})𝑥Feed

UNIVERSITY OF CENTRAL FLORIDASESSION MANAGEMENT Needs to release resource after usesess.close() Using context managerwith tf.Session() as sess: Interactive sessionsess InteractiveSession()14

UNIVERSITY OF CENTRAL FLORIDA15LOSSSelect loss functionLoss function for softmaxsoftmax cross entropy with logits(logits, labels,name optional name )labels tf.placeholder("float", [1, 3])logits tf.matmul( x, w ) bcross entropy tf.nn.softmax cross entropy with logits(logits, labels, name 'xentropy')

UNIVERSITY OF CENTRAL FLORIDAOPTIMIZATIONGradient descentclass GradientDescentOptimizer(learning rate,use locking False,name 'GradientDescent')labels tf.placeholder("float", [1, 3])cross entropy tf.nn.softmax cross entropy with logits(tf.matmul( x, w ) b, labels, name 'xentropy')optimizer tf.train.GradientDescentOptimizer(0.1)train op optimizer.minimize(cross entropy)sess.run(train op, feed dict {x:np.array([[1., 2., 3.]]),labels:np.aray([[0., 1., 0.]])})16

UNIVERSITY OF CENTRAL FLORIDA17ITERATIVE UPDATEGradient descent requires multiple steps to convergelabels tf.placeholder("float", [1, 3])cross entropy tf.nn.softmax cross entropy with logits(tf.matmul( x, w ) b, labels, name 'xentropy')optimizer tf.train.GradientDescentOptimizer(0.1)train op optimizer.minimize(cross entropy)for step in range(10):sess.run(train op, feed dict {x:np.array([[1., 2., 3.]]),labels:np.array([[0., 1., 0.]])})

UNIVERSITY OF CENTRAL FLORIDA18EVALUATIONHow well does model do?correct prediction tf.equal( tf.argmax(logits, 1),tf.argmax(labels, 1) )accuracy tf.reduce mean( tf.cast(correct prediction,tf.float32) )print sess.run(accuracy, feed dict {x:np.array([[1., 2., 3.]]),labels:np.array([[0., 1., 0.]])})

UNIVERSITY OF CENTRAL FLORIDAADDING LAYERSx tf.placeholder("float", [1, 3])out xnum layers 2for layer in range(num layers):w tf.Variable( tf.random normal([3, 3]) )b tf.Variable( tf.zeros([1, 3]) )out tf.nn.relu( tf.matmul(out, w) b ) 19

UNIVERSITY OF CENTRAL FLORIDAGRAPH VISUALIZATIONTensorBoardwriter tf.train.SummaryWriter('/tmp/tf logs', sess.graph)Launch TensorBoardtensorboard --logdir /tmp/tf logsNavigate web browser to:localhost:600620

21UNIVERSITY OF CENTRAL FLORIDAVISUALIZE STATESAdd summariesscalar summaryhistogram summarysummary op tf.merge all summaries()for step in range(10):, summary sess.run([train op, summary op],feed dict )writer.add summary(summary, step)

UNIVERSITY OF CENTRAL FLORIDASAVE AND LOAD MODELStf.train.Saver( )Default will associate all variablesall variables()save(sess, save path, )restore(sess, save path, )Replace initialization22

UNIVERSITY OF CENTRAL FLORIDA23ADDING NEW OP Familiarity with C required Build TensorFlow from source Eigen leviewer file-view-default

UNIVERSITY OF CENTRAL FLORIDA24RESOURCES http://tensorflow.org https://github.com/tensorflow/tensorflowTensorFlow wrappers Keras:https://keras.io/Eigen Tensor library r file-view-default

Tutorial by Astrid Jackson. UNIVERSITY OF CENTRAL FLORIDA 2 . node in the graph, not the result Softmax. UNIVERSITY OF CENTRAL FLORIDA 8 COMPUTATION AS DATA FLOW GRAPH 8 biases weights inputs targets MatMul Add Softmax Xent Graph of Nodes, als