Transcription
Python for EverybodyExploring Data Using Python 3Dr. Charles R. Severance
CreditsEditorial Support: Elliott Hauser, Sue BlumenbergCover Design: Aimee AndrionPrinting History 2016-Jul-05 First Complete Python 3.0 version 2015-Dec-20 Initial Python 3.0 rough conversionCopyright DetailsCopyright 2009- Dr. Charles R. Severance.This work is licensed under a Creative Commons Attribution-NonCommercialShareAlike 3.0 Unported License. This license is available /You can see what the author considers commercial and non-commercial uses of thismaterial as well as license exemptions in the Appendix titled “Copyright Detail”.
iiiPrefaceRemixing an Open BookIt is quite natural for academics who are continuously told to “publish or perish”to want to always create something from scratch that is their own fresh creation.This book is an experiment in not starting from scratch, but instead “remixing”the book titled Think Python: How to Think Like a Computer Scientist writtenby Allen B. Downey, Jeff Elkner, and others.In December of 2009, I was preparing to teach SI502 - Networked Programming atthe University of Michigan for the fifth semester in a row and decided it was time towrite a Python textbook that focused on exploring data instead of understandingalgorithms and abstractions. My goal in SI502 is to teach people lifelong datahandling skills using Python. Few of my students were planning to be professionalcomputer programmers. Instead, they planned to be librarians, managers, lawyers,biologists, economists, etc., who happened to want to skillfully use technology intheir chosen field.I never seemed to find the perfect data-oriented Python book for my course, so Iset out to write just such a book. Luckily at a faculty meeting three weeks beforeI was about to start my new book from scratch over the holiday break, Dr. AtulPrakash showed me the Think Python book which he had used to teach his Pythoncourse that semester. It is a well-written Computer Science text with a focus onshort, direct explanations and ease of learning.The overall book structure has been changed to get to doing data analysis problemsas quickly as possible and have a series of running examples and exercises aboutdata analysis from the very beginning.Chapters 2–10 are similar to the Think Python book, but there have been majorchanges. Number-oriented examples and exercises have been replaced with dataoriented exercises. Topics are presented in the order needed to build increasinglysophisticated data analysis solutions. Some topics like try and except are pulledforward and presented as part of the chapter on conditionals. Functions are givenvery light treatment until they are needed to handle program complexity ratherthan introduced as an early lesson in abstraction. Nearly all user-defined functionshave been removed from the example code and exercises outside of Chapter 4. Theword “recursion”1 does not appear in the book at all.In chapters 1 and 11–16, all of the material is brand new, focusing on real-worlduses and simple examples of Python for data analysis including regular expressionsfor searching and parsing, automating tasks on your computer, retrieving dataacross the network, scraping web pages for data, object-oriented programming,using web services, parsing XML and JSON data, creating and using databasesusing Structured Query Language, and visualizing data.The ultimate goal of all of these changes is to shift from a Computer Science to anInformatics focus and to only include topics into a first technology class that canbe useful even if one chooses not to become a professional programmer.1 Except,of course, for this line.
ivStudents who find this book interesting and want to further explore should lookat Allen B. Downey’s Think Python book. Because there is a lot of overlap between the two books, students will quickly pick up skills in the additional areas oftechnical programming and algorithmic thinking that are covered in Think Python.And given that the books have a similar writing style, they should be able to movequickly through Think Python with a minimum of effort.As the copyright holder of Think Python, Allen has given me permission to changethe book’s license on the material from his book that remains in this book from theGNU Free Documentation License to the more recent Creative Commons Attribution — Share Alike license. This follows a general shift in open documentationlicenses moving from the GFDL to the CC-BY-SA (e.g., Wikipedia). Using theCC-BY-SA license maintains the book’s strong copyleft tradition while making iteven more straightforward for new authors to reuse this material as they see fit.I feel that this book serves as an example of why open materials are so importantto the future of education, and I want to thank Allen B. Downey and CambridgeUniversity Press for their forward-looking decision to make the book availableunder an open copyright. I hope they are pleased with the results of my effortsand I hope that you, the reader, are pleased with our collective efforts.I would like to thank Allen B. Downey and Lauren Cowles for their help, patience,and guidance in dealing with and resolving the copyright issues around this book.Charles Severancewww.dr-chuck.comAnn Arbor, MI, USASeptember 9, 2013Charles Severance is a Clinical Associate Professor at the University of MichiganSchool of Information.
Contents1 Why should you learn to write programs?11.1Creativity and motivation . . . . . . . . . . . . . . . . . . . . . . .21.2Computer hardware architecture . . . . . . . . . . . . . . . . . . .31.3Understanding programming . . . . . . . . . . . . . . . . . . . . .41.4Words and sentences . . . . . . . . . . . . . . . . . . . . . . . . . .51.5Conversing with Python . . . . . . . . . . . . . . . . . . . . . . . .61.6Terminology: Interpreter and compiler . . . . . . . . . . . . . . . .81.7Writing a program . . . . . . . . . . . . . . . . . . . . . . . . . . .101.8What is a program? . . . . . . . . . . . . . . . . . . . . . . . . . .101.9The building blocks of programs . . . . . . . . . . . . . . . . . . . . 111.10What could possibly go wrong? . . . . . . . . . . . . . . . . . . . .121.11Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .141.12The learning journey . . . . . . . . . . . . . . . . . . . . . . . . . .151.13Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .151.14Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .162 Variables, expressions, and statements192.1Values and types . . . . . . . . . . . . . . . . . . . . . . . . . . . .192.2Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .202.3Variable names and keywords . . . . . . . . . . . . . . . . . . . . . . 212.4Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.5Operators and operands . . . . . . . . . . . . . . . . . . . . . . . .222.6Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .232.7Order of operations . . . . . . . . . . . . . . . . . . . . . . . . . .232.8Modulus operator . . . . . . . . . . . . . . . . . . . . . . . . . . .242.9String operations . . . . . . . . . . . . . . . . . . . . . . . . . . . .24v
viCONTENTS2.10Asking the user for input . . . . . . . . . . . . . . . . . . . . . . .252.11Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .262.12Choosing mnemonic variable names . . . . . . . . . . . . . . . . .272.13Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .282.14Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .292.15Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .303 Conditional execution313.1Boolean expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.2Logical operators . . . . . . . . . . . . . . . . . . . . . . . . . . . .323.3Conditional execution . . . . . . . . . . . . . . . . . . . . . . . . .323.4Alternative execution . . . . . . . . . . . . . . . . . . . . . . . . .333.5Chained conditionals . . . . . . . . . . . . . . . . . . . . . . . . . .343.6Nested conditionals . . . . . . . . . . . . . . . . . . . . . . . . . .353.7Catching exceptions using try and except . . . . . . . . . . . . . .363.8Short-circuit evaluation of logical expressions . . . . . . . . . . . .383.9Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .393.10Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .393.11Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .404 Functions434.1Function calls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .434.2Built-in functions. . . . . . . . . . . . . . . . . . . . . . . . . . .434.3Type conversion functions . . . . . . . . . . . . . . . . . . . . . . .444.4Math functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . .454.5Random numbers . . . . . . . . . . . . . . . . . . . . . . . . . . .464.6Adding new functions . . . . . . . . . . . . . . . . . . . . . . . . .474.7Definitions and uses . . . . . . . . . . . . . . . . . . . . . . . . . .484.8Flow of execution . . . . . . . . . . . . . . . . . . . . . . . . . . .494.9Parameters and arguments . . . . . . . . . . . . . . . . . . . . . .494.10Fruitful functions and void functions . . . . . . . . . . . . . . . . . . 514.11Why functions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . .524.12Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .524.13Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .534.14Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .54
CONTENTSvii5 Iteration575.1Updating variables . . . . . . . . . . . . . . . . . . . . . . . . . . .575.2The while statement . . . . . . . . . . . . . . . . . . . . . . . . .575.3Infinite loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .585.4Finishing iterations with continue . . . . . . . . . . . . . . . . . .595.5Definite loops using for . . . . . . . . . . . . . . . . . . . . . . . .605.6Loop patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615.6.1Counting and summing loops . . . . . . . . . . . . . . . . . . 615.6.2Maximum and minimum loops . . . . . . . . . . . . . . . .625.7Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .645.8Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .645.9Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .646 Strings676.1A string is a sequence . . . . . . . . . . . . . . . . . . . . . . . . .676.2Getting the length of a string using len . . . . . . . . . . . . . . .686.3Traversal through a string with a loop . . . . . . . . . . . . . . . .686.4String slices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .696.5Strings are immutable . . . . . . . . . . . . . . . . . . . . . . . . .706.6Looping and counting . . . . . . . . . . . . . . . . . . . . . . . . .706.7The in operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 716.8String comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . 716.9String methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 716.10Parsing strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . .746.11Format operator . . . . . . . . . . . . . . . . . . . . . . . . . . . .746.12Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .756.13Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .766.14Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .777 Files797.1Persistence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .797.2Opening files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .807.3Text files and lines . . . . . . . . . . . . . . . . . . . . . . . . . . . . 817.4Reading files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .827.5Searching through a file . . . . . . . . . . . . . . . . . . . . . . . .83
viiiCONTENTS7.6Letting the user choose the file name . . . . . . . . . . . . . . . . .857.7Using try, except, and open . . . . . . . . . . . . . . . . . . . .867.8Writing files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .877.9Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .887.10Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .897.11Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .898 Lists918.1A list is a sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 918.2Lists are mutable . . . . . . . . . . . . . . . . . . . . . . . . . . . .928.3Traversing a list . . . . . . . . . . . . . . . . . . . . . . . . . . . .928.4List operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . .938.5List slices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .948.6List methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .948.7Deleting elements . . . . . . . . . . . . . . . . . . . . . . . . . . .958.8Lists and functions . . . . . . . . . . . . . . . . . . . . . . . . . . .968.9Lists and strings . . . . . . . . . . . . . . . . . . . . . . . . . . . .978.10Parsing lines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .988.11Objects and values . . . . . . . . . . . . . . . . . . . . . . . . . . .998.12Aliasing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1008.13List arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1008.14Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1028.15Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1058.16Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1059 Dictionaries1099.1Dictionary as a set of counters . . . . . . . . . . . . . . . . . . . . . 1119.2Dictionaries and files . . . . . . . . . . . . . . . . . . . . . . . . . .1129.3Looping and dictionaries . . . . . . . . . . . . . . . . . . . . . . .1139.4Advanced text parsing . . . . . . . . . . . . . . . . . . . . . . . . .1159.5Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1169.6Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1179.7Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .117
CONTENTSix10 Tuples11910.1Tuples are immutable . . . . . . . . . . . . . . . . . . . . . . . . .11910.2Comparing tuples . . . . . . . . . . . . . . . . . . . . . . . . . . .12010.3Tuple assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . .12210.4Dictionaries and tuples . . . . . . . . . . . . . . . . . . . . . . . .12310.5Multiple assignment with dictionaries . . . . . . . . . . . . . . . .12410.6The most common words . . . . . . . . . . . . . . . . . . . . . . .12510.7Using tuples as keys in dictionaries . . . . . . . . . . . . . . . . . .12610.8Sequences: strings, lists, and tuples - Oh My! . . . . . . . . . . . .12610.9List comprehension. . . . . . . . . . . . . . . . . . . . . . . . . .12710.10 Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12710.11 Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12810.12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12811 Regular expressions13111.1Character matching in regular expressions . . . . . . . . . . . . . .13211.2Extracting data using regular expressions . . . . . . . . . . . . . .13311.3Combining searching and extracting . . . . . . . . . . . . . . . . .13611.4Escape character . . . . . . . . . . . . . . . . . . . . . . . . . . . .14011.5Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14011.6Bonus section for Unix / Linux users . . . . . . . . . . . . . . . . . . 14111.7Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14211.8Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14211.9Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14312 Networked programs14512.1Hypertext Transfer Protocol - HTTP . . . . . . . . . . . . . . . .14512.2The world’s simplest web browser . . . . . . . . . . . . . . . . . .14612.3Retrieving an image over HTTP . . . . . . . . . . . . . . . . . . .14812.4Retrieving web pages with urllib . . . . . . . . . . . . . . . . . .15012.5Reading binary files using urllib . . . . . . . . . . . . . . . . . . . 15112.6Parsing HTML and scraping the web . . . . . . . . . . . . . . . .15212.7Parsing HTML using regular expressions . . . . . . . . . . . . . .15212.8Parsing HTML using BeautifulSoup . . . . . . . . . . . . . . . . .15412.9Bonus section for Unix / Linux users . . . . . . . . . . . . . . . . .15712.10 Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15712.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .158
xCONTENTS13 Using Web Services15913.1eXtensible Markup Language - XML . . . . . . . . . . . . . . . . .15913.2Parsing XML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16013.3Looping through nodes . . . . . . . . . . . . . . . . . . . . . . . . . 16113.4JavaScript Object Notation - JSON . . . . . . . . . . . . . . . . .16213.5Parsing JSON . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16313.6Application Programming Interfaces . . . . . . . . . . . . . . . . .16413.7Security and API usage . . . . . . . . . . . . . . . . . . . . . . . .16513.8Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16613.9Application 1: Google geocoding web service . . . . . . . . . . . .16613.10 Application 2: Twitter . . . . . . . . . . . . . . . . . . . . . . . . .17014 Object-oriented programming17514.1Managing larger programs . . . . . . . . . . . . . . . . . . . . . . .17514.2Getting started . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17614.3Using objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17614.4Starting with programs . . . . . . . . . . . . . . . . . . . . . . . .17714.5Subdividing a problem . . . . . . . . . . . . . . . . . . . . . . . . .17914.6Our first Python object . . . . . . . . . . . . . . . . . . . . . . . .17914.7Classes as types . . . . . . . . . . . . . . . . . . . . . . . . . . . .18214.8Object lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18314.9Multiple instances . . . . . . . . . . . . . . . . . . . . . . . . . . .18414.10 Inheritance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18514.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18614.12 Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18715 Using Databases and SQL18915.1What is a database? . . . . . . . . . . . . . . . . . . . . . . . . . .18915.2Database concepts . . . . . . . . . . . . . . . . . . . . . . . . . . .18915.3Database Browser for SQLite . . . . . . . . . . . . . . . . . . . . .19015.4Creating a database table . . . . . . . . . . . . . . . . . . . . . . .19015.5Structured Query Language summary . . . . . . . . . . . . . . . .19315.6Spidering Twitter using a database . . . . . . . . . . . . . . . . . .19515.7Basic data modeling . . . . . . . . . . . . . . . . . . . . . . . . . .20015.8Programming with multiple tables . . . . . . . . . . . . . . . . . . . 201
CONTENTSxi15.8.1 Constraints in database tables . . . . . . . . . . . . . . . .20415.8.2 Retrieve and/or insert a record . . . . . . . . . . . . . . . .20515.8.3 Storing the friend relationship . . . . . . . . . . . . . . . . .206Three kinds of keys . . . . . . . . . . . . . . . . . . . . . . . . . .20715.10 Using JOIN to retrieve data . . . . . . . . . . . . . . . . . . . . . .20815.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21015.915.12 Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21115.13 Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21116 Visualizing data21316.1Building a OpenStreetMap from geocoded data . . . . . . . . . . .21316.2Visualizing networks and interconnections . . . . . . . . . . . . . .21516.3Visualizing mail data . . . . . . . . . . . . . . . . . . . . . . . . .218A Contributions225A.1Contributor List for Python for Everybody . . . . . . . . . . . . .225A.2Contributor List for Python for Informatics . . . . . . . . . . . . .225A.3Preface for “Think Python” . . . . . . . . . . . . . . . . . . . . . .225A.3.1 The strange history of “Think Python” . . . . . . . . . . .225A.3.2 Acknowledgements for “Think Python” . . . . . . . . . . .227Contributor List for “Think Python” . . . . . . . . . . . . . . . . .227A.4B Copyright Detail229
xiiCONTENTS
Chapter 1Why should you learn towrite programs?Writing programs (or programming) is a very creative and rewarding activity. Youcan write programs for many reasons, ranging from making your living to solvinga difficult data analysis problem to having fun to helping someone else solve aproblem. This book assumes that everyone needs to know how to program, andthat once you know how to program you will figure out what you want to do withyour newfound skills.We are surrounded in our daily lives with computers ranging from laptops to cellphones. We can think of these computers as our “personal assistants” who can takecare of many things on our behalf. The hardware in our current-day computers isessentially built to continuously ask us the question, “What would you like me todo xt?WhatNext?Figure 1.1: Personal Digital AssistantProgrammers add an operating system and a set of applications to the hardwareand we end up with a Personal Digital Assistant that is quite helpful and capableof helping us do many different things.Our computers are fast and have vast amounts of memory and could be very helpfulto us if we only knew the language to speak to explain to the computer what wewould like it to “do next”. If we knew this language, we could tell the computerto do tasks on our behalf that were repetitive. Interestingly, the kinds of thingscomputers can do best are often the kinds of things that we humans find boringand mind-numbing.1
2CHAPTER 1. WHY SHOULD YOU LEARN TO WRITE PROGRAMS?For example, look at the first three paragraphs of this chapter and tell me themost commonly used word and how many times the word is used. While you wereable to read and understand the words in a few seconds, counting them is almostpainful because it is not the kind of problem that human minds are designed tosolve. For a computer, the opposite is true, reading and understanding text froma piece of paper is hard for a computer to do but counting the words and tellingyou how many times the most used word was used is very easy for the computer:python words.pyEnter file:words.txtto 16Our “personal information analysis assistant” quickly told us that the word “to”was used sixteen times in the first three paragraphs of this chapter.This very fact that computers are good at things that humans are not is why youneed to become skilled at talking “computer language”. Once you learn this newlanguage, you can delegate mundane tasks to your partner (the computer), leavingmore time for you to do the things that you are uniquely suited for. You bringcreativity, intuition, and inventiveness to this partnership.1.1Creativity and motivationWhile this book is not intended for professional programmers, professional programming can be a very rewarding job both financially and personally. Buildinguseful, elegant, and clever programs for others to use is a very creative activity.Your computer or Personal Digital Assistant (PDA) usually contains many different programs from many different groups of programmers, each competing foryour attention and interest. They try their best to meet your needs and give you agreat user experience in the process. In some situations, when you choose a pieceof software, the programmers are directly compensated because of your choice.If we think of programs as the creative output of groups of programmers, perhapsthe following figure is a more sensible version of our PDA:PickMe!PickMe!PickMe!PickMe!PickMe!BuyMe :)Figure 1.2: Programmers Talking to YouFor now, our primary motivation is not to make money or please end users, butinstead for us to be more productive in handling the data and information that wewill encounter in our lives. When you first start, you will be both the programmerand the end user of your programs. As you gain skill as a programmer and programming feels more creative to you, your thoughts may turn toward developingprograms for others.
1.2. COMPUTER HARDWARE ARCHITECTURE1.23Computer hardware architectureBefore we start learning the language we speak to give instructions to computersto develop software, we need to learn a small amount about how computers arebuilt. If you were to take apart your computer or cell phone and look deep inside,you would find the following parts:SoftwareInput MemoryNetworkSecondaryMemoryFigure 1.3: Hardware ArchitectureThe high-level definitions of these parts are as follows: The Central Processing Unit (or CPU) is the part of the computer that isbuilt to be obsessed with “what is next?” If your computer is rated at 3.0Gigahertz, it means that the CPU will ask “What next?” three billion timesper second. You are going to have to learn how to talk fast to keep up withthe CPU. The Main Memory is used to store information that the CPU needs in ahurry. The main memory is nearly as fast as the CPU. But the informationstored in the main memory vanishes when the computer is turned off. The Secondary Memory is also used to store information, but it is muchslower than the main memory. The advantage of the secondary memory isthat it can store information even when there is no power to the computer.Examples of secondary memory are disk drives or flash memory (typicallyfound in USB sticks and portable music players). The Input and Output Devices are simply our screen, keyboard, mouse, microphone, speaker, touchpad, etc. They are all of the ways we interact withthe computer. These days, most computers also have a Network Connection to retrieveinformation over a network. We can think of the network as a very slowplace to store and retrieve data that might not always be “up”. So in a sense,the network is a slower and at times unreliable form of Secondary Memory.While most of the detail of how these components work is best left to computerbuilders, it helps to have some terminology so we can talk about these differentparts as we write our programs.
4CHAPTER 1. WHY SHOULD YOU LEARN TO WRITE PROGRAMS?As a programmer, your job is to use and orchestrate each of these resources tosolve the problem that you need to solve and analyze the data you get from thesolution. As a programmer you will mostly be “talking” to the CPU and tellingit what to do next. Sometimes you will tell the CPU to use the main memory,secondary memory, network, or the input/output devices.SoftwareInput MemoryNetworkSecondaryMemoryFigure 1.4: Where Are You?You need to be the person who answers the CPU’s “What next?” question. But itwould be very uncomfortable to shrink you down to 5mm tall and insert you intothe computer just so you could issue a command three billion times per second. Soinstead, you must write down your instructions in advance. We call these storedinstructions a program and the act of writing these instructions down and gettingthe instructions to be correct programming.1.3Understanding programmingIn the rest of this book, we will try to turn you into a person who is skilled in the artof programming. In the end you will be a programmer - perhaps not a professionalprogrammer, but at least you will have the skills to look at a data/informationanalysis problem and develop a program to solve the problem.In a sense, you need two skills to be a programmer: First, you need to know the programming language (Python) - you need toknow the vocabulary and the grammar. You need to be able to spell thewords in this new language properly and know how to construct well-formed“sentences” in this new language. Second, you need to “tell a story”. In writing a story, you combine wordsand sentences to convey an idea to the reader. There is a skill and art inconstructing the story, and skill in story writing is improved by doing somewriting and getting some feedback. In programming, our program is the“story” and the problem you are trying to solve is the “idea”.Once you learn one programming language such as Python, you will find it mucheasier to learn a second programming language such as JavaScript or C . The
1.4. WORDS AND SENTENCES5new programming language has very different vocabulary and grammar but theproblem-solving skills will be the same across all programming languages.You will learn the “vocabulary” and “sentences” of Python pretty quickly. It willtake longer for you to be able to write a coherent program to solve a brand-newproblem. We teach programming much like we teach writing. We start readingand explaining programs, then we write simple programs, and then we write increasingly complex programs over time. At some point you “get your muse” andsee the patterns on your own and can see more naturally how to take a problemand write a program that solves that problem. And once you get to that point,programming becomes a very pleasant and creative process.We start with the vocabulary and structure of Python programs. Be patient asthe simple examples remind you of when you started reading for the first time.1.4Words and sentencesUnlike human languages, the Python vocabulary is actually pretty small. We callthis “vocabulary” the “reserved words”. These are words that have very specialmeaning to Pytho
data analysis from the very beginning. Chapters 2–10 are similar to the Think Python book, but there have been major changes. Number-oriented examples and exercises have been replaced with data-oriented exercises. Topics are presented in the order needed to build increasingl
