Transcription
A Brief Introduction toNeural NetworksDavid Krieseldkriesel.comDownload location:http://www.dkriesel.com/en/science/neural networksNEW – for the programmers:Scalable and efficient NN framework, written in JAVAhttp://www.dkriesel.com/en/tech/snipe
A small preface"Originally, this work has been prepared in the framework of a seminar of theUniversity of Bonn in Germany, but it has been and will be extended (afterbeing presented and published online under www.dkriesel.com on5/27/2005). First and foremost, to provide a comprehensive overview of thesubject of neural networks and, second, just to acquire more and moreknowledge about LATEX . And who knows – maybe one day this summary willbecome a real preface!"Abstract of this work, end of 2005The above abstract has not yet become apreface but at least a little preface, eversince the extended text (then 40 pageslong) has turned out to be a downloadhit.Ambition and intention of thismanuscriptThe entire text is written and laid outmore effectively and with more illustrations than before. I did all the illustrations myself, most of them directly inLATEX by using XYpic. They reflect whatI would have liked to see when becomingacquainted with the subject: Text and illustrations should be memorable and easyto understand to offer as many people aspossible access to the field of neural networks.stand the definitions without reading therunning text, while the opposite holds forreaders only interested in the subject matter; everything is explained in both colloquial and formal language. Please let meknow if you find out that I have violatedthis principle.The sections of this text are mostlyindependent from each otherThe document itself is divided into different parts, which are again divided intochapters. Although the chapters containcross-references, they are also individuallyaccessible to readers with little previousknowledge. There are larger and smallerchapters: While the larger chapters shouldprovide profound insight into a paradigmof neural networks (e.g. the classic neuralnetwork structure: the perceptron and itsNevertheless, the mathematically and for- learning procedures), the smaller chaptersmally skilled readers will be able to under- give a short overview – but this is also ex-v
dkriesel.complained in the introduction of each chapter.In addition to all the definitions and explanations I have included some excursusesto provide interesting information not directly related to the subject.Unfortunately, I was not able to find freeGerman sources that are multi-facetedin respect of content (concerning theparadigms of neural networks) and, nevertheless, written in coherent style. Theaim of this work is (even if it could notbe fulfilled at first go) to close this gap bitby bit and to provide easy access to thesubject.the original high-performance simulationdesign goal. Those of you who are up forlearning by doing and/or have to use afast and stable neural networks implementation for some reasons, should definetelyhave a look at Snipe.However, the aspects covered by Snipe arenot entirely congruent with those coveredby this manuscript. Some of the kindsof neural networks are not supported bySnipe, while when it comes to other kindsof neural networks, Snipe may have lotsand lots more capabilities than may everbe covered in the manuscript in the formof practical hints. Anyway, in my experience almost all of the implementation reWant to learn not only byquirements of my readers are covered well.reading, but also by coding?On the Snipe download page, look for theUse SNIPE!section "Getting started with Snipe" – youwill find an easy step-by-step guide conSNIPE 1 is a well-documented JAVA li- cerning Snipe and its documentation, asbrary that implements a framework for well as some examples.neural networks in a speedy, feature-richand usable way. It is available at noSNIPE: This manuscript frequently incorporates Snipe. Shaded Snipe-paragraphscost for non-commercial purposes. It waslike this one are scattered among largeoriginally designed for high performanceparts of the manuscript, providing inforsimulations with lots and lots of neuralmation on how to implement their connetworks (even large ones) being trainedtext in Snipe. This also implies thatthose who do not want to use Snipe,simultaneously. Recently, I decided tojust have to skip the shaded Snipegive it away as a professional reference imparagraphs! The Snipe-paragraphs asplementation that covers network aspectssume the reader has had a close look athandled within this work, while at thethe "Getting started with Snipe" section.same time being faster and more efficientOften, class names are used. As Snipe consists of only a few different packages, I omitthan lots of other implementations due to1 Scalable and Generalized Neural Information Processing Engine, downloadable at http://www.dkriesel.com/tech/snipe, online JavaDoc athttp://snipe.dkriesel.comvited the package names within the qualifiedclass names for the sake of readability.D. Kriesel – A Brief Introduction to Neural Networks (ZETA2-EN)
dkriesel.comIt’s easy to print thismanuscriptSpeaking headlines throughout thetext, short ones in the table ofcontentsThis text is completely illustrated incolor, but it can also be printed as is inmonochrome: The colors of figures, tablesand text are well-chosen so that in addition to an appealing design the colors arestill easy to distinguish when printed inmonochrome.The whole manuscript is now pervaded bysuch headlines. Speaking headlines arenot just title-like ("Reinforcement Learning"), but centralize the information givenin the associated section to a single sentence. In the named instance, an appropriate headline would be "Reinforcementlearning methods provide feedback to thenetwork, whether it behaves good or bad".However, such long headlines would bloatthe table of contents in an unacceptableway. So I used short titles like the first onein the table of contents, and speaking ones,like the latter, throughout the text.There are many tools directlyintegrated into the textDifferent aids are directly integrated in thedocument to make reading more flexible: Marginal notes are a navigationalHowever, anyone (like me) who prefers aidreading words on paper rather than onscreen can also enjoy some features.The entire document contains marginalnotes in colloquial language (see the example in the margin), allowing you to "scan"the document quickly to find a certain pasIn the table of contents, differentsage in the text (including the titles).Hypertexton paper:-)types of chapters are markedNew mathematical symbols are marked byspecific marginal notes for easy findingDifferent types of chapters are directly (see the example for x in the margin).marked within the table of contents. Chapters, that are marked as "fundamental"are definitely ones to read because almost There are several kinds of indexingall subsequent chapters heavily depend onthem. Other chapters additionally depend This document contains different types ofon information given in other (preceding) indexing: If you have found a word inchapters, which then is marked in the ta- the index and opened the correspondingpage, you can easily find it by searchingble of contents, too.D. Kriesel – A Brief Introduction to Neural Networks (ZETA2-EN)viiJx
dkriesel.comfor highlighted text – all indexed wordsare highlighted like this.3. You must maintain the author’s attri-Mathematical symbols appearing in several chapters of this document (e.g. Ω foran output neuron; I tried to maintain aconsistent nomenclature for regularly recurring elements) are separately indexedunder "Mathematical Symbols", so theycan easily be assigned to the corresponding term.4. You may not use the attribution tobution of the document at all times.imply that the author endorses youor your document use.For I’m no lawyer, the above bullet-pointsummary is just informational: if there isany conflict in interpretation between thesummary and the actual license, the actuallicense always takes precedence. Note thatthis license does not extend to the sourceNames of persons written in small capsfiles used to produce the document. Thoseare indexed in the category "Persons" andare still mine.ordered by the last names.Terms of use and licenseBeginning with the epsilon edition, thetext is licensed under the Creative Commons Attribution-No Derivative Works3.0 Unported License 2 , except for somelittle portions of the work licensed undermore liberal licenses as mentioned (mainlysome figures from Wikimedia Commons).A quick license summary:How to cite this manuscriptThere’s no official publisher, so you needto be careful with your citation. Pleasefind more information in English andGerman language on my homepage, respectively the subpage concerning themanuscript3 .Acknowledgement1. You are free to redistribute this docu- Now I would like to express my grati-tude to all the people who contributed, inwhatever manner, to the success of thiswork, since a work like this needs manyhelpers. First of all, I want to thankthe proofreaders of this text, who helped2. You may not modify, transform, or me and my readers very much. In albuild upon the document except for phabetical order: Wolfgang Apolinarski,personal use.Kathrin Gräve, Paul Imhoff, Thomasment (even though it is a much betteridea to just distribute the URL of myhomepage, for it always contains themost recent version of the text).2 3 http://www.dkriesel.com/en/science/neural networksD. Kriesel – A Brief Introduction to Neural Networks (ZETA2-EN)
dkriesel.comKühn, Christoph Kunze, Malte Lohmeyer,Joachim Nock, Daniel Plohmann, DanielRosenthal, Christian Schulz and TobiasWilken.Additionally, I want to thank the readersDietmar Berger, Igor Buchmüller, MarieChrist, Julia Damaschek, Jochen Döll,Maximilian Ernestus, Hardy Falk, AnneFeldmeier, Sascha Fink, Andreas Friedmann, Jan Gassen, Markus Gerhards, Sebastian Hirsch, Andreas Hochrath, NicoHöft, Thomas Ihme, Boris Jentsch, TimHussein, Thilo Keller, Mario Krenn, MirkoKunze, Maikel Linke, Adam Maciak,Benjamin Meier, David Möller, AndreasMüller, Rainer Penninger, Lena Reichel,Alexander Schier, Matthias Siegmund,Mathias Tirtasana, Oliver Tischler, Maximilian Voit, Igor Wall, Achim Weber,Frank Weinreis, Gideon Maillette de BuijWenniger, Philipp Woock and many others for their feedback, suggestions and remarks.of the University of Bonn – they all madesure that I always learned (and also hadto learn) something new about neural networks and related subjects. Especially Dr.Goerke has always been willing to respondto any questions I was not able to answermyself during the writing process. Conversations with Prof. Eckmiller made me stepback from the whiteboard to get a betteroverall view on what I was doing and whatI should do next.Globally, and not only in the context ofthis work, I want to thank my parents whonever get tired to buy me specialized andtherefore expensive books and who havealways supported me in my studies.For many "remarks" and the very specialand cordial atmosphere ;-) I want to thankAndreas Huber and Tobias Treutler. Sinceour first semester it has rarely been boringwith you!Now I would like to think back to myschool days and cordially thank someteachers who (in my opinion) had imparted some scientific knowledge to me –although my class participation had notalways been wholehearted: Mr. WilfriedHartmann, Mr. Hubert Peters and Mr.Especially, I would like to thank BeateFrank Nökel.Kuhl for translating the entire text fromGerman to English, and for her questionsFurthermore I would like to thank thewhich made me think of changing thewhole team at the notary’s office of Dr.phrasing of some paragraphs.Kemp and Dr. Kolb in Bonn, where I haveI would particularly like to thank Prof. always felt to be in good hands and whoRolf Eckmiller and Dr. Nils Goerke as have helped me to keep my printing costswell as the entire Division of Neuroinfor- low - in particular Christiane Flamme andmatics, Department of Computer Science Dr. Kemp!Additionally, I’d like to thank SebastianMerzbach, who examined this work in avery conscientious way finding inconsistencies and errors. In particular, he clearedlots and lots of language clumsiness fromthe English version.D. Kriesel – A Brief Introduction to Neural Networks (ZETA2-EN)ix
dkriesel.comThanks go also to the Wikimedia Commons, where I took some (few) images andaltered them to suit this text.Last but not least I want to thank twopeople who made outstanding contributions to this work who occupy, so to speak,a place of honor: My girlfriend VerenaThomas, who found many mathematicaland logical errors in my text and discussed them with me, although she haslots of other things to do, and Christiane Schultze, who carefully reviewed thetext for spelling mistakes and inconsistencies.David KrieselxD. Kriesel – A Brief Introduction to Neural Networks (ZETA2-EN)
ContentsA small prefaceIvFrom biology to formalization – motivation, philosophy, history andrealization of neural models11 Introduction, motivation and history1.1 Why neural networks? . . . . . . . . . . . .1.1.1 The 100-step rule . . . . . . . . . . .1.1.2 Simple application examples . . . . .1.2 History of neural networks . . . . . . . . . .1.2.1 The beginning . . . . . . . . . . . .1.2.2 Golden age . . . . . . . . . . . . . .1.2.3 Long silence and slow reconstruction1.2.4 Renaissance . . . . . . . . . . . . . .Exercises . . . . . . . . . . . . . . . . . . . . . .2 Biological neural networks2.1 The vertebrate nervous system . . . . . . . . . .2.1.1 Peripheral and central nervous system . .2.1.2 Cerebrum . . . . . . . . . . . . . . . . . .2.1.3 Cerebellum . . . . . . . . . . . . . . . . .2.1.4 Diencephalon . . . . . . . . . . . . . . . .2.1.5 Brainstem . . . . . . . . . . . . . . . . . .2.2 The neuron . . . . . . . . . . . . . . . . . . . . .2.2.1 Components . . . . . . . . . . . . . . . .2.2.2 Electrochemical processes in the neuron .2.3 Receptor cells . . . . . . . . . . . . . . . . . . . .2.3.1 Various types . . . . . . . . . . . . . . . .2.3.2 Information processing within the nervous2.3.3 Light sensing organs . . . . . . . . . . . .2.4 The amount of neurons in living organisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .system. . . . . . . . .3356889111212.131313141515161616192424252628xi
Contentsdkriesel.com2.5 Technical neurons as caricature of biology . . . . . . . . . . . . . . . . . 30Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 Components of artificial neural networks (fundamental)3.1 The concept of time in neural networks . . . . . .3.2 Components of neural networks . . . . . . . . . . .3.2.1 Connections . . . . . . . . . . . . . . . . . .3.2.2 Propagation function and network input . .3.2.3 Activation . . . . . . . . . . . . . . . . . . .3.2.4 Threshold value . . . . . . . . . . . . . . . .3.2.5 Activation function . . . . . . . . . . . . . .3.2.6 Common activation functions . . . . . . . .3.2.7 Output function . . . . . . . . . . . . . . .3.2.8 Learning strategy . . . . . . . . . . . . . . .3.3 Network topologies . . . . . . . . . . . . . . . . . .3.3.1 Feedforward . . . . . . . . . . . . . . . . . .3.3.2 Recurrent networks . . . . . . . . . . . . . .3.3.3 Completely linked networks . . . . . . . . .3.4 The bias neuron . . . . . . . . . . . . . . . . . . .3.5 Representing neurons . . . . . . . . . . . . . . . . .3.6 Orders of activation . . . . . . . . . . . . . . . . .3.6.1 Synchronous activation . . . . . . . . . . .3.6.2 Asynchronous activation . . . . . . . . . . .3.7 Input and output of data . . . . . . . . . . . . . .Exercises . . . . . . . . . . . . . . . . . . . . . . . . . .4 Fundamentals on learning and training samples4.1 Paradigms of learning . . . . . . . . . . . .4.1.1 Unsupervised learning . . . . . . . .4.1.2 Reinforcement learning . . . . . . .4.1.3 Supervised learning . . . . . . . . .4.1.4 Offline or online learning? . . . . . .4.1.5 Questions in advance . . . . . . . . .4.2 Training patterns and teaching input . . . .4.3 Using training samples . . . . . . . . . . . .4.3.1 Division of the training set . . . . .4.3.2 Order of pattern representation . . .4.4 Learning curve and error measurement . . .4.4.1 When do we stop learning? . . . . 3454545464848.51515253535454545657575859D. Kriesel – A Brief Introduction to Neural Networks (ZETA2-EN)
dkriesel.com4.5Gradient optimization procedures . . . .4.5.1 Problems of gradient procedures4.6 Exemplary problems . . . . . . . . . . .4.6.1 Boolean functions . . . . . . . .4.6.2 The parity function . . . . . . .4.6.3 The 2-spiral problem . . . . . . .4.6.4 The checkerboard problem . . . .4.6.5 The identity function . . . . . .4.6.6 Other exemplary problems . . .4.7 Hebbian rule . . . . . . . . . . . . . . .4.7.1 Original rule . . . . . . . . . . .4.7.2 Generalized form . . . . . . . . .Exercises . . . . . . . . . . . . . . . . . . . .IIContents.Supervised learning network paradigms5 The perceptron, backpropagation and its variants5.1 The singlelayer perceptron . . . . . . . . . . . . . . . . . . . . .5.1.1 Perceptron learning algorithm and convergence theorem5.1.2 Delta rule . . . . . . . . . . . . . . . . . . . . . . . . . .5.2 Linear separability . . . . . . . . . . . . . . . . . . . . . . . . .5.3 The multilayer perceptron . . . . . . . . . . . . . . . . . . . . .5.4 Backpropagation of error . . . . . . . . . . . . . . . . . . . . . .5.4.1 Derivation . . . . . . . . . . . . . . . . . . . . . . . . . .5.4.2 Boiling backpropagation down to the delta rule . . . . .5.4.3 Selecting a learning rate . . . . . . . . . . . . . . . . . .5.5 Resilient backpropagation . . . . . . . . . . . . . . . . . . . . .5.5.1 Adaption of weights . . . . . . . . . . . . . . . . . . . .5.5.2 Dynamic learning rate adjustment . . . . . . . . . . . .5.5.3 Rprop in practice . . . . . . . . . . . . . . . . . . . . . .5.6 Further variations and extensions to backpropagation . . . . .5.6.1 Momentum term . . . . . . . . . . . . . . . . . . . . . .5.6.2 Flat spot elimination . . . . . . . . . . . . . . . . . . . .5.6.3 Second order backpropagation . . . . . . . . . . . . . .5.6.4 Weight decay . . . . . . . . . . . . . . . . . . . . . . . .5.6.5 Pruning and Optimal Brain Damage . . . . . . . . . . .5.7 Initial configuration of a multilayer perceptron . . . . . . . . .5.7.1 Number of layers . . . . . . . . . . . . . . . . . . . . . .5.7.2 The number of neurons . . . . . . . . . . . . . . . . . .D. Kriesel – A Brief Introduction to Neural Networks 486879192939494959696979898989999100xiii
Contentsdkriesel.com5.7.3 Selecting an activation function . . . . . . .5.7.4 Initializing weights . . . . . . . . . . . . . .5.8 The 8-3-8 encoding problem and related problemsExercises . . . . . . . . . . . . . . . . . . . . . . . . . .1001011011026 Radial basis functions6.1 Components and structure . . . . . . . . . . . . . . . .6.2 Information processing of an RBF network . . . . . .6.2.1 Information processing in RBF neurons . . . .6.2.2 Analytical thoughts prior to the training . . . .6.3 Training of RBF networks . . . . . . . . . . . . . . . .6.3.1 Centers and widths of RBF neurons . . . . . .6.4 Growing RBF networks . . . . . . . . . . . . . . . . .6.4.1 Adding neurons . . . . . . . . . . . . . . . . . .6.4.2 Limiting the number of neurons . . . . . . . . .6.4.3 Deleting neurons . . . . . . . . . . . . . . . . .6.5 Comparing RBF networks and multilayer perceptronsExercises . . . . . . . . . . . . . . . . . . . . . . . . . . . .105. 105. 106. 108. 111. 114. 115. 118. 118. 119. 119. 119. 1207 Recurrent perceptron-like networks (depends on chapter 5)7.1 Jordan networks . . . . . . . . . . . . . . . . . . . .7.2 Elman networks . . . . . . . . . . . . . . . . . . . . .7.3 Training recurrent networks . . . . . . . . . . . . . .7.3.1 Unfolding in time . . . . . . . . . . . . . . . .7.3.2 Teacher forcing . . . . . . . . . . . . . . . . .7.3.3 Recurrent backpropagation . . . . . . . . . .7.3.4 Training with evolution . . . . . . . . . . . .129. 129. 129. 130. 131. 131. 132. 133. 134. 135. 135. 136.8 Hopfield networks8.1 Inspired by magnetism . . . . . . . . . . . . . . . . . .8.2 Structure and functionality . . . . . . . . . . . . . . .8.2.1 Input and output of a Hopfield network . . . .8.2.2 Significance of weights . . . . . . . . . . . . . .8.2.3 Change in the state of neurons . . . . . . . . .8.3 Generating the weight matrix . . . . . . . . . . . . . .8.4 Autoassociation and traditional application . . . . . .8.5 Heteroassociation and analogies to neural data storage8.5.1 Generating the heteroassociative matrix . . . .8.5.2 Stabilizing the heteroassociations . . . . . . . .8.5.3 Biological motivation of heterassociation . . . .xiv.121122123124125127127127D. Kriesel – A Brief Introduction to Neural Networks (ZETA2-EN)
dkriesel.comContents8.6 Continuous Hopfield networks . . . . . . . . . . . . . . . . . . . . . . . . 136Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1379 Learning vector quantization9.1 About quantization . . . . . . . .9.2 Purpose of LVQ . . . . . . . . . .9.3 Using codebook vectors . . . . .9.4 Adjusting codebook vectors . . .9.4.1 The procedure of learning9.5 Connection to neural networks .Exercises . . . . . . . . . . . . . . . .139. 139. 140. 140. 141. 141. 143. 143III Unsupervised learning network paradigms14510 Self-organizing feature maps10.1 Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10.2 Functionality and output interpretation . . . . . . . . . . . . . .10.3 Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10.3.1 The topology function . . . . . . . . . . . . . . . . . . . .10.3.2 Monotonically decreasing learning rate and neighborhood10.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10.4.1 Topological defects . . . . . . . . . . . . . . . . . . . . . .10.5 Adjustment of resolution and position-dependent learning rate .10.6 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10.6.1 Interaction with RBF networks . . . . . . . . . . . . . . .10.7 Variations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10.7.1 Neural gas . . . . . . . . . . . . . . . . . . . . . . . . . . .10.7.2 Multi-SOMs . . . . . . . . . . . . . . . . . . . . . . . . . .10.7.3 Multi-neural gas . . . . . . . . . . . . . . . . . . . . . . .10.7.4 Growing neural gas . . . . . . . . . . . . . . . . . . . . . .Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11 Adaptive resonance theory11.1 Task and structure of an ART network . . .11.1.1 Resonance . . . . . . . . . . . . . . .11.2 Learning process . . . . . . . . . . . . . . .11.2.1 Pattern input and top-down learning11.2.2 Resonance and bottom-up learning .11.2.3 Adding an output neuron . . . . . .D. Kriesel – A Brief Introduction to Neural Networks 163163164164.165. 165. 166. 167. 167. 167. 167xv
Contentsdkriesel.com11.3 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167IV Excursi, appendices and registers169A Excursus: Cluster analysis and regional and online learnable fieldsA.1 k-means clustering . . . . . . . . . . . . . . . . . . . . . . . . .A.2 k-nearest neighboring . . . . . . . . . . . . . . . . . . . . . . .A.3 ε-nearest neighboring . . . . . . . . . . . . . . . . . . . . . . .A.4 The silhouette coefficient . . . . . . . . . . . . . . . . . . . . . .A.5 Regional and online learnable fields . . . . . . . . . . . . . . . .A.5.1 Structure of a ROLF . . . . . . . . . . . . . . . . . . . .A.5.2 Training a ROLF . . . . . . . . . . . . . . . . . . . . . .A.5.3 Evaluating a ROLF . . . . . . . . . . . . . . . . . . . .A.5.4 Comparison with popular clustering methods . . . . . .A.5.5 Initializing radii, learning rates and multiplier . . . . . .A.5.6 Application examples . . . . . . . . . . . . . . . . . . .Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .171172172173173175176177178179180180180B Excursus: neural networks used for predictionB.1 About time series . . . . . . . . . . . . . . . . . . .B.2 One-step-ahead prediction . . . . . . . . . . . . . .B.3 Two-step-ahead prediction . . . . . . . . . . . . . .B.3.1 Recursive two-step-ahead prediction . . . .B.3.2 Direct two-step-ahead prediction . . . . . .B.4 Additional optimization approaches for prediction .B.4.1 Changing temporal parameters . . . . . . .B.4.2 Heterogeneous prediction . . . . . . . . . .B.5 Remarks on the prediction of share prices . . . . .181181183185185185185185187187C Excursus: reinforcement learningC.1 System structure . . . . . . . . . . .C.1.1 The gridworld . . . . . . . . .C.1.2 Agent und environment . . .C.1.3 States, situations and actionsC.1.4 Reward and return . . . . . .C.1.5 The policy . . . . . . . . . .C.2 Learning process . . . . . . . . . . .C.2.1 Rewarding strategies . . . . .C.2.2 The state-value function . . .191192192193194195196198198199xvi.D. Kriesel – A Brief Introduction to Neural Networks (ZETA2-EN)
dkriesel.comC.2.3 Monte Carlo method . . . . .C.2.4 Temporal difference learningC.2.5 The action-value function . .C.2.6 Q learning . . . . . . . . . . .C.3 Example applications . . . . . . . . .C.3.1 TD gammon . . . . . . . . .C.3.2 The car in the pit . . . . . .C.3.3 The pole balancer . . . . . .C.4 Reinforcement learning in connectionExercises . . . . . . . . . . . . . . . . . .Contents. . . . . . . . . . . . . . . . .with. . . . . . . . . . . . . . . . . . . . . . . . . . .neural. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .networks. . . . . .201202203204205205205206207207Bibliography209List of Figures215Index219D. Kriesel – A Brief Introduction to Neural Networks (ZETA2-EN)xvii
Part IFrom biology to formalization –motivation, philosophy, history andrealization of neural models1
Chapter 1Introduction, motivation and historyHow to teach a computer? You can either write a fixed program – or you canenable the computer to learn on its own. Living beings do not have anyprogrammer writing a program for developing their skills, which then only hasto be executed. They learn by themselves – without the previous knowledgefrom external impressions – and thus can solve problems better than anycomputer today. What qualities are needed to achieve such a behavior fordevices like computers? Can such cognition be adapted from biology? History,development, decline and resurgence of a wide approach to solve problems.1.1 Why neural networks?There are problem categories that cannotbe formulated as an algorithm. Problemsthat depend on many subtle factors, for example the purchase price of a real estatewhich our brain can (approximately) calculate. Without an algorithm a computercannot do the same. Therefore the question to be asked is: How do we learn toexplore such problems?ComputerscannotlearnExactly – we learn; a capability computers obviously do not have. Humans havea brain that can learn. Computers havesome processing units and memory. Theyallow the computer to perform the mostcomplex numerical calculations in a veryshort time, but they are not adaptive.If we compare computer and brain1 , wewill note that, theoretically, the computershould be more powerful than our brain:It comprises 109 transistors with a switching time of 10 9 seconds. The brain contains 1011 neurons, but these only have aswitching time of about 10 3 seconds.The largest part of the brain is working continuously, while the largest part ofthe computer is
long) has turned out to be a download hit. Ambitionandintentionofthis manuscript The entire text is written and laid out more effectively and with more illustra-tions than before. I did all the illustra-tions myself, most of them directly in LATEXbyusingXYpi
