Transcription
Machine Learning forEconomics andFinance inTensorFlow 2Deep Learning Models forResearch and IndustryIsaiah Hull
Machine Learning for Economics and Finance in TensorFlow 2:Deep Learning Models for Research and IndustryIsaiah HullNacka, SwedenISBN-13 (pbk): 978-1-4842-6372-3 ISBN-13 (electronic): 2-6373-0Copyright 2021 by Isaiah HullThis work is subject to copyright. All rights are reserved by the Publisher, whether the whole orpart of the material is concerned, specifically the rights of translation, reprinting, reuse ofillustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way,and transmission or information storage and retrieval, electronic adaptation, computer software,or by similar or dissimilar methodology now known or hereafter developed.Trademarked names, logos, and images may appear in this book. Rather than use a trademarksymbol with every occurrence of a trademarked name, logo, or image we use the names, logos,and images only in an editorial fashion and to the benefit of the trademark owner, with nointention of infringement of the trademark.The use in this publication of trade names, trademarks, service marks, and similar terms, even ifthey are not identified as such, is not to be taken as an expression of opinion as to whether or notthey are subject to proprietary rights.While the advice and information in this book are believed to be true and accurate at the date ofpublication, neither the authors nor the editors nor the publisher can accept any legalresponsibility for any errors or omissions that may be made. The publisher makes no warranty,express or implied, with respect to the material contained herein.Managing Director, Apress Media LLC: Welmoed SpahrAcquisitions Editor: Aaron BlackDevelopment Editor: James MarkhamCoordinating Editor: Jessica VakiliDistributed to the book trade worldwide by Springer Science Business Media New York, 233Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mailorders-ny@springer-sbm.com, or visit www.springeronline.com. Apress Media, LLC is aCalifornia LLC and the sole member (owner) is Springer Science Business Media Finance Inc(SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation.For information on translations, please e-mail booktranslations@springernature.com; forreprint, paperback, or audio rights, please e-mail bookpermissions@springernature.com.Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBookversions and licenses are also available for most titles. For more information, reference our Printand eBook Bulk Sales web page at http://www.apress.com/bulk-sales.Any source code or other supplementary material referenced by the author in this book isavailable to readers on GitHub via the book's product page, located at www.apress.com/978-1-4842-6372-3. For more detailed information, please visit http://www.apress.com/source-code.Printed on acid-free paper
For my wife, Jamie; my son, Moses;and my parents, James and Gale
Table of ContentsAbout the Author xiAbout the Technical Reviewer xiiiChapter 1: T ensorFlow 2 1Installing TensorFlow 1Changes in TensorFlow 2 2TensorFlow for Economics and Finance 12Machine Learning 14Theoretical Models 19Introduction to Tensors 20Linear Algebra and Calculus in TensorFlow 22Constants and Variables 22Linear Algebra 24Differential Calculus 34Loading Data for Use in TensorFlow 55Summary 58Bibliography 59Chapter 2: M achine Learning and Economics 61“Big Data: New Tricks for Econometrics” (Varian 2014) 62“Prediction Policy Problems” (Kleinberg et al. 2015) 64“Machine Learning: An Applied Econometric Approach” (Mullainathan andSpiess 2017) 66v
Table of Contents“The Impact of Machine Learning on Economics” (Athey 2019) 69Machine Learning and Traditional Econometric Methods 69Off-the-Shelf ML Routines 72Policy Analysis 73Active Research and Predictions 74“Machine Learning Methods Economists Should Know About”(Athey and Imbens 2019) 75“Text as Data” (Gentzkow et al. 2019) 77Representing Text As Data 77Statistical Methods 78Applications 80“How is Machine Learning Useful for Macroeconomic Forecasting”(Coulombe et al. 2019) 80Summary 83Bibliography 84Chapter 3: R egression 87Linear Regression 87Overview 88Ordinary Least Squares (OLS) 91Least Absolute Deviations (LAD) 94Other Loss Functions 101Partially Linear Models 102Non-linear Regression 108Logistic Regression 114Loss Functions 115Discrete Dependent Variables 116Continuous Dependent Variables 117Optimizers 119vi
Table of ContentsStochastic Gradient Descent (SGD) 119Modern Optimizers 122Summary 124Bibliography 125Chapter 4: T rees 127Decision Trees 127Overview 128Feature Engineering 131Training 132Regression Trees 134Random Forests 136Gradient Boosted Trees 138Classification Trees 140Regression Trees 143Model Tuning 144Summary 146Bibliography 147Chapter 5: I mage Classification 149Image Data 150Neural Networks 155Keras 158The Sequential API 159The Functional API 168Estimators 171Convolutional Neural Networks 174The Convolutional Layer 174Training a Convolutional Neural Network 177vii
Table of ContentsPretrained Models 181Feature Extraction 181Model Fine-Tuning 184Summary 185Bibliography 186Chapter 6: T ext Data 189Data Cleaning and Preparation 190Collecting the Data 192Text Data Notation 195Data Preparation 196The Bag-of-Words Model 203Dictionary-Based Methods 209Word Embeddings 215Topic Modeling 217Text Regression 226Text Classification 243Summary 245Bibliography 246Chapter 7: T ime Series 249Sequential Models of Machine Learning 250Dense Neural Networks 250Recurrent Neural Networks 257Long Short-Term Memory (LSTM) 264Intermediate Hidden States 268Multivariate Forecasts 271LSTM 272Gradient Boosted Trees 275viii
Table of ContentsSummary 278Bibliography 278Chapter 8: D imensionality Reduction 281Dimensionality Reduction in Economics 282Principal Component Analysis 283Partial Least Squares 293The Autoencoder Model 296Summary 305Bibliography 306Chapter 9: G enerative Models 307Variational Autoencoders 309Generative Adversarial Networks 318Applications in Economics and Finance 327Summary 329Bibliography 330Chapter 10: T heoretical Models 331Solving Theoretical Models 332The Cake-Eating Problem 332The Neoclassical Business Cycle Model 341Deep Reinforcement Learning 348Summary 355Bibliography 355 Index 357ix
About the AuthorIsaiah Hull is a senior economist at theResearch Division of Sweden's CentralBank. He holds a PhD in economics fromBoston College and conducts research oncomputational economics, machine learning,macro-finance, and fintech. He also teachescourses on the DataCamp platform, includingIntroduction to TensorFlow in Python, andis working on an interdisciplinary researchproject to introduce quantum computing andquantum money to the economics discipline.xi
About the Technical ReviewerVishwesh Ravi Shrimali graduated from BITS Pilani in 2018, where hestudied mechanical engineering. Since then, he has worked with BigVision LLC on deep learning and computer vision and was involved increating official OpenCV AI courses. Currently, he is working at MercedesBenz Research and Development India Pvt. Ltd. He has a keen interestin programming and AI and has applied that interest in mechanicalengineering projects. He has also written multiple blogs about OpenCVand deep learning on Learn OpenCV, a leading blog on computer vision.He has also coauthored Machine Learning for OpenCV 4 (second edition)by Packt. When he is not writing blogs or working on projects, he likes to goon long walks or play his acoustic guitar.xiii
and deep learning on Learn OpenCV, a leading blog on computer vision. He has also coauthored Machine Learning for OpenCV 4 (second edition) by Packt. When he is not writing blogs or working on projects, he likes to go on long walks or play his acoustic guitar.
