Transcription
Machine Learningwith the Raspberry PiExperiments with Data andComputer VisionDonald J. Norris
Machine Learning with the Raspberry Pi: Experiments with Data andComputer VisionDonald J. NorrisBarrington, NH, USAISBN-13 (pbk): 2-5174-4ISBN-13 (electronic): 978-1-4842-5174-4Copyright 2020 by Donald J. NorrisThis work is subject to copyright. All rights are reserved by the Publisher, whether the wholeor part of the material is concerned, specifically the rights of translation, reprinting, reuse ofillustrations, recitation, broadcasting, reproduction on microfilms or in any other physicalway, and transmission or information storage and retrieval, electronic adaptation, computersoftware, or by similar or dissimilar methodology now known or hereafter developed.Trademarked names, logos, and images may appear in this book. Rather than use atrademark symbol with every occurrence of a trademarked name, logo, or image we use thenames, logos, and images only in an editorial fashion and to the benefit of the trademarkowner, with no intention of infringement of the trademark.The use in this publication of trade names, trademarks, service marks, and similar terms,even if they are not identified as such, is not to be taken as an expression of opinion as towhether or not they are subject to proprietary rights.While the advice and information in this book are believed to be true and accurate at thedate of publication, neither the authors nor the editors nor the publisher can accept anylegal responsibility for any errors or omissions that may be made. The publisher makes nowarranty, express or implied, with respect to the material contained herein.Managing Director, Apress Media LLC: Welmoed SpahrAcquisitions Editor: Aaron BlackDevelopment Editor: James MarkhamCoordinating Editor: Jessica VakiliDistributed to the book trade worldwide by Springer Science Business Media New York,233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201)348-4505, e-mail orders-ny@springer-sbm.com, or visit www.springeronline.com. ApressMedia, LLC is a California LLC and the sole member (owner) is Springer Science BusinessMedia Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation.For information on translations, please e-mail rights@apress.com, or visit http://www.apress.com/rights-permissions.Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBookversions and licenses are also available for most titles. For more information, reference ourPrint and eBook Bulk Sales web page at http://www.apress.com/bulk-sales.Any source code or other supplementary material referenced by the author in this book isavailable to readers on GitHub via the book's product page, located at www.apress.com/978-1-4842-5173-7. For more detailed information, please visit http://www.apress.com/source-code.Printed on acid-free paper
Table of ContentsAbout the Author viiAbout the Technical Reviewer ixChapter 1: Introduction to machine learning (ML) with theRaspberry Pi (RasPi) 1RasPi introduction 1Writing the Raspbian Image to a micro SD card 4Updating and upgrading the Raspbian distribution 15ML facts 24Chapter 2: Exploration of ML data models: Part 1 49Installing OpenCV 4 49Download OpenCV 4 source code 51Building the OpenCV software 52Seaborn data visualization library 59Underlying big principle 77Naive Bayes 91k-nearest neighbor (k-NN) model 101KNN demonstration 102Decision tree classifier 106Decision tree algorithm 107Decision tree classifier demonstration with scikit-learn 122iii
Table of ContentsChapter 3: Exploration of ML data models: Part 2 135Principal component analysis 136PCA script discussion 137Linear discriminant analysis 150LDA script discussion 152Support vector machines 162SVM demonstration – Part 1 166SVM demonstration – Part 2 169Learning vector quantization 177LVQ basic concepts 178LVQ demonstration 180Bagging and random forests 190Introduction to bagging and random forest 190Bootstrap resampling demonstration 193Bagging demonstration 195Random forest demonstration 204Chapter 4: Preparation for deep learning 213DL basics 213Machine learning from data patterns 214Loss functions 221Optimizer algorithm 225Artificial neural network 235How ANNs are trained and function 238Practical ANN weight modification example 258ANN Python demonstration – Part 1 262ANN Python demonstration – Part 2 267iv
Table of ContentsChapter 5: Practical deep learning ANN demonstrations 279Parts list 280Recognizing handwritten number demonstration 280Project history and preparatory details 284Adjusting the input datasets 293Interpreting ANN output data values 295Creating an ANN that does handwritten number recognition 297Initial ANN training script demonstration 299ANN test script demonstration 301ANN test script demonstration using the full training dataset 309Recognizing your own handwritten numbers 313Handwritten number recognition using Keras 324Introduction to Keras 324Installing Keras 325Downloading the dataset and creating a model 326Chapter 6: CNN demonstrations 335Parts list 335Introduction to the CNN model 336History and evolution of the CNN 342Fashion MNIST demonstration 358More complex Fashion MNIST demonstration 369VGG Fashion MNIST demonstration 373Jason’s Fashion MNIST demonstration 378Chapter 7: Predictions using ANNs and CNNs 387Pima Indian Diabetes demonstration 388Background for the Pima Indian Diabetes study 388Preparing the data 389v
Table of ContentsUsing the scikit-learn library with Keras 408Grid search with Keras and scikit-learn 411Housing price regression predictor demonstration 416Preprocessing the data 417The baseline model 421Improved baseline model 425Another improved baseline model 428Predictions using CNNs 431Univariate time series CNN model 433Chapter 8: Predictions using CNNs and MLPs formedical research 453Parts list 454Downloading the breast cancer histology Image dataset 455Preparing the project environment 459Using a MLP model for breast cancer prediction 491Chapter 9: Reinforcement learning 501Markov decision process 503Discounted future reward 505Q-learning 506Q-learning and neural networks 547Index 555vi
About the AuthorDonald J. Norris is an avid electronics hobbyist and maker. He is also anelectronics engineer with an advanced degree in Production Management.Don is retired from civilian government service with the US Navy, wherehe specialized in acoustics and digital signal processing. He also has morethan a dozen years’ experience as a professional software developerusing C, C#, C , Python, and Java, and 5 years’ experience as a certifiedIT security consultant.vii
About the Technical ReviewerAhmed Fawzy Gad is a machine learningengineer who holds B.Sc. and M.Sc. inInformation Technology. Ahmed is a teachingassistant and a researcher who is interestedin machine/deep learning, computer vision,and Python. He is a machine learningtechnical reviewer and consultant helpingothers do their projects. Ahmed contributeswritten tutorials and articles to a number ofblogs including Paperspace, Real Python,KDnuggets, Heartbeat, and Towards DataScience.Ahmed has authored three books titled TensorFlow: A Guide toBuild Artificial Neural Networks Using Python (Lambert 2017), PracticalComputer Vision Applications Using Deep Learning with CNNs (Apress,2018), and Building Android Apps in Python Using Kivy with AndroidStudio (Apress, 2019).He welcomes you to connect with him through LinkedIn(linkedin.com/in/AhmedFGad), Facebook (fb.com/AhmedFGadd), ande-mail (ahmed.f.gad@gmail.com).ix
Ahmed Fawzy Gad is a machine learning engineer who holds B.Sc. and M.Sc. in Information Technology. Ahmed is a teaching assistant and a researcher who is interested in machine/deep learning, computer vision, and Python. He is a machine learning technical reviewer and consultant helping others do their projects. Ahmed contributes
