Transcription
DEGREE PROJECT IN COMPUTER SCIENCE AND ENGINEERING,SECOND CYCLE, 30 CREDITSSTOCKHOLM, SWEDEN 2017Embedded Vision MachineLearning on Embedded Devicesfor Image classification inIndustrial Internet of thingsBILAL PARVEZKTH ROYAL INSTITUTE OF TECHNOLOGYSCHOOL OF INFORMATION AND COMMUNICATION TECHNOLOGY
Embedded VisionMachine Learning on EmbeddedDevices for Image classificationin Industrial Internet of thingsBilal Parvez2017-10-22Master’s ThesisExaminerAhmed HemaniSupervisorDimitrios StathisIndustrial adviserSebastian Petruch, Workaround GmbH, Munich, GermanyKTH Royal Institute of TechnologySchool of Information and Communication Technology (ICT)Department of Communication SystemsSE-100 44 Stockholm, Sweden
Abstract iAbstractBecause of Machine Learning, machines have become extremely good at image classification in nearreal time. With using significant training data, powerful machines can be trained to recognizeimages as good as any human would. Till now the norm has been to have pictures sent to a serverand have the server recognize them. With increasing number of sensors the trend is moving towardsedge computing to curb the increasing rate of data transfer and communication bottlenecks. Theidea is to do the processing locally or as close to the sensor as possible and then only transmitactionable data to the server. While, this does solve plethora of communication problems, speciallyin industrial settings, it creates a new problem. The sensors need to do this computationallyintensive image classification which is a challenge for embedded/wearable devices, due to theirresource constrained nature.This thesis analyzes Machine Learning algorithms and libraries from the motivation of portingimage classifiers to embedded devices. This includes, comparing different supervised MachineLearning approaches to image classification and figuring out which are most suited for being portedto embedded devices. Taking a step forward in making the process of testing and implementingMachine Learning algorithms as easy as their desktop counterparts. The goal is to ease the processof porting new image recognition and classification algorithms on a host of different embeddeddevices and to provide motivations behind design decisions.The final proposal goes through all design considerations and implements a prototype that ishardware independent. Which can be used as a reference for designing and then later porting ofMachine Learning classifiers to embedded devices.KeywordsMachine Learning, Image classification, Embedded devices
Sammanfattning iiiSammanfattningMaskiner har blivit extremt bra på bildklassificering i nära realtid. På grund av maskininlärningmed kraftig träningsdata, kan kraftfulla maskiner utbildas för att känna igen bilder så bra som allamänniskor skulle. Hittills har trenden varit att få bilderna skickade till en server och sedan fåservern att känna igen bilderna. Men eftersom sensorerna ökar i antal, går trenden mot så kallad"edge computing" för att stryka den ökande graden av dataöverföring ochkommunikationsflaskhalsar. Tanken är att göra bearbetningen lokalt eller så nära sensorn sommöjligt och sedan bara överföra aktiv data till servern. Samtidigt som detta löser överflöd avkommunikationsproblem, speciellt i industriella inställningar, skapar det ett nytt problem.Sensorerna måste kunna göra denna beräkningsintensiva bildklassificering ombord vilket specielltär en utmaning för inbyggda system och bärbara enheter, på grund av sin resursbegränsade natur.Denna avhandling analyserar maskininlärningsalgoritmer och biblioteken från motivationenatt portera generiska bildklassificatorer till inbyggda system. Att jämföra olika övervakademaskininlärningsmetoder för bildklassificering, utreda vilka som är mest lämpade för att bliporterade till inbyggda system, för att göra processen att testa och implementeramaskininlärningsalgoritmer lika enkelt som sina skrivbordsmodeller. Målet är att underlättaprocessen för att portera nya bildigenkännings och klassificeringsalgoritmer på en mängd olikainbyggda system och att ge motivation bakom designbeslut som tagits och för att beskriva isiondesign".Det slutliga förslaget går igenom all hänsyn till konstruktion och implementerar en prototyp somär maskinvaruoberoende och kan användas för snabb framtagning av prototyper och sedan senareöverföring av maskininlärningsklassificatorer till inbyggda system.NyckelordMaskininlärning, Bildklassificering, Inbyggda system
Acknowledgments vAcknowledgmentsThis report is a result of Master's thesis project at Workaround GmbH (Proglove) in the embeddedsoftware department in Munich, Germany, during the period of 20 th March 2017 to the mid ofSeptember.I would like to take this time and thank the following people to help me carry out this thesisto its completion. Professor Gerald Q. Maguire Jr. for having written this draft of this template on2014.06.03. Professor Vlado Handziski at TU Berlin for the initial idea and direction. Elecia White from the embedded.fm blog, for the continuous motivation and foranswering all my questions. All the people over at Proglove who helped me with equipment and with generalsupport all along the way. Laurent Itti, Professor of computer science, psychology and neuroscienceUniversity of Southern California for all the help with answering all the questionsrelated to Jevois (the hardware platform). My industrial supervisor, Sebastian Petruch for his extremely valuable feedbackthroughout the entire process, specially about “roter faden”. Finally, my family and friends for rooting for me all along the way.I would also like to thank Olga Russakovsky, for permission to use Figure 2-4 and AlfredoCanziani, for permission to use Figure 3-2, as part of this thesis.Another special thanks to Ammar Talic and Franco Saworski for helping me with revisingthe report. The many comments,insights and discussions greatly helped in shaping the report.Stockholm, September 2017Bilal Parvez
Table of contents viiTable of elord.iiiAcknowledgments.vTable of contents.viiList of Figures.xList of Tables.xiList of acronyms and abbreviations.xiii1. roblem definition.2Purpose.3Goals.3Research Methodology.4Delimitations.4Structure of the thesis.4Background.62.12.2Machine Learning .62.1.1Background of Machine Learning.62.1.2Types of Machine Learning.72.1.3Machine Learning for image classification.8Computer vision.92.2.1Definition of computer vision.92.2.22.3Embedded systems.102.3.1Definition of embedded systems.112.3.22.42.5Embedded vision.11Jevois Platform.112.4.1Software platform .112.4.2Hardware platform.122.4.3Major Concepts / Using rithms and Applications.10Using tiny-dnn.14Related work.142.6.1Embeddable libraries.142.6.2Developing kits.152.6.3Modeling neural nets over FPGA's.152.6.4Developing neural compute chips.152.6.5Practical deep learning.152.6.6AI as a service.15
2.6.72.73.Summary.16Methodology .183.13.23.33.4Research process.18Research paradigm.19Rationale and choice of technology.193.3.1Jevois .193.3.2Machine Learning paradigms.203.3.3Tiny-dnn.22Experiment design and planned measurements.233.4.1Experimental test bed.233.4.23.53.64.Data analysis/ System characterization.24Evaluation framework.24Proposed Methodology.27Setting up Jevois.274.2.1Using the camera.274.2.24.34.44.5Programming the camera .29Setting up tiny-dnn.29Integrating tiny-dnn into 1Using the mizations.355.2.1Reducing floating-point precision.355.2.25.35.45.5Optimizing computations.35Domain Adaptations.36Comparison .36Discussions.375.5.1Considerations when Porting to an Embedded MCU:.375.5.26.Hardware and software used.24Implementation.274.14.25.ARM Optimizations.16Lab Experiments vs Real World Products.37Conclusion and Future ure work.
This thesis analyzes Machine Learning algorithms and libraries from the motivation of porting image classifiers to embedded devices. This includes, comparing different supervised Machine Learning approaches to image classification and figuring out which are most suited for being ported to embedded devices. Taking a step forward in making the process of testing and implementing
