Transcription
Extracted from:Intuitive PythonProductive Development for Projects that LastThis PDF file contains pages extracted from Intuitive Python, published by thePragmatic Bookshelf. For more information or to purchase a paperback or PDFcopy, please visit http://www.pragprog.com.Note: This extract contains some colored text (particularly in code listing). Thisis available only in online versions of the books. The printed versions are blackand white. Pagination might vary between the online and printed versions; thecontent is otherwise identical.Copyright 2021 The Pragmatic Programmers, LLC.All rights reserved.No part of this publication may be reproduced, stored in a retrieval system, or transmitted,in any form, or by any means, electronic, mechanical, photocopying, recording, or otherwise,without the prior consent of the publisher.The Pragmatic BookshelfRaleigh, North Carolina
Intuitive PythonProductive Development for Projects that LastDavid MullerThe Pragmatic BookshelfRaleigh, North Carolina
Many of the designations used by manufacturers and sellers to distinguish their productsare claimed as trademarks. Where those designations appear in this book, and The PragmaticProgrammers, LLC was aware of a trademark claim, the designations have been printed ininitial capital letters or in all capitals. The Pragmatic Starter Kit, The Pragmatic Programmer,Pragmatic Programming, Pragmatic Bookshelf, PragProg and the linking g device are trademarks of The Pragmatic Programmers, LLC.Every precaution was taken in the preparation of this book. However, the publisher assumesno responsibility for errors or omissions, or for damages that may result from the use ofinformation (including program listings) contained herein.For our complete catalog of hands-on, practical, and Pragmatic content for software developers, please visit https://pragprog.com.The team that produced this book includes:CEO: Dave RankinCOO: Janet FurlowManaging Editor: Tammy CoronDevelopment Editor: Adaobi Obi TultonCopy Editor: Karen GalleLayout: Gilson GraphicsFounders: Andy Hunt and Dave ThomasFor sales, volume licensing, and support, please contact support@pragprog.com.For international rights, please contact rights@pragprog.com.Copyright 2021 The Pragmatic Programmers, LLC.All rights reserved. No part of this publication may be reproduced, stored in a retrieval system,or transmitted, in any form, or by any means, electronic, mechanical, photocopying, recording,or otherwise, without the prior consent of the publisher.ISBN-13: 978-1-68050-823-9Encoded using the finest acid-free high-entropy binary digits.Book version: P1.0—June 2021
Keeping Your Source OrganizedJust like the dinosaurs in Jurassic Park, Python codebases have a habit ofbreaking out from inside the fenced pens we try to create for them. One particularly vexing problem can be keeping project code organized. As timepasses, code starts to live in places it shouldn’t and it can become increasinglydifficult to find what files code should live in or where new code should beplaced. In this section, we’ll explore a general strategy that uses Python’sbuilt-in unittest module to keep a directory organized.Maintaining Organization in a tests/ DirectoryMany Python codebases contain tests to help verify that they are workingcorrectly. Frequently, these tests are defined in a directory named tests/ at thetop level of the codebase. The tests/ directory structure typically matches thedirectory structure in the corresponding source directory that is being tested.Let’s consider a Python project with the following directory structure: j park init .py dinosaurs init .py raptor.py t rex.py fences init .py cable.py electrified.pytests init .py fences init .py test cable.py test electrified.py test raptor.pyThis source tree has a j park/ directory with source code and a tests/ directorywith test *.py files that correspond to source files in j park/.If you look at this tree, you’ll see that the layout of the j park/ directory hasdiverged from the layout in tests/. Among other inconsistencies, test electrified.pyis at the root level of tests/ instead of in the tests/fences/ subdirectory. Additionally, test raptor.py is orphaned at the root level of tests/ instead of occupying aplace in a tests/dinosaurs/ subdirectory. Click HERE to purchase this book now. discuss
6Python unittest Test Discovery Still Requires init .pyIn Python 3, you generally do not need to create init .py files likeyou did in Python 2. The init .py files are included in this example,however, because a longstanding bug prevents Python unittest discovery (for example, python3 -m unittest discover --help) from finding testfiles that don’t have init .py siblings.10With the existing disorganization, it would be unsurprising to receive a changerequest that updates the directory layout so that jurassic park/ and tests/ divergefurther: jurassic park init .py dinosaurs init .py raptor.py t rex.py fences init .py cable.py electrified.pytests init .py fences init .py test cable.py test electrified.py test raptor.py test t rex.pyNotice in the example that a new file test t rex.py has been added directly undertests/. This doesn’t match j park/ where t rex.py lives under the dinosaurs/ subdirectory.The tests/ directory is becoming increasingly divergent from the j park/ sourcedirectory. As the project accumulates more files, it becomes harder andharder to find where a source code is tested. It’s true that this disorganizationmay not be the end of the world, but it increases the barrier to entry to yourproject. Where does test code live? Where should new contributors find tests?How do they know if something is already tested? You notice this and mayfeel a bit wary—let’s address this wariness and capitalize on the opportunityto improve the code’s organization.10. https://stackoverflow.com/a/53976736 Click HERE to purchase this book now. discuss
Keeping Your Source Organized 7You can mandate that the j park/ and tests/ directory layouts match using aunittest TestCase in a new file tests/test directory layout.py:test directory layout.pyimport unittestfrom pathlib import Pathclass TestDirectoryLayout(unittest.TestCase):def test tests layout matches j park(self):# verify that this file is - itself - in tests/this files path Path( file )tests dir this files path.parentself.assertEqual(tests dir.name, "tests")# get a path to the j park/ source directoryj park path Path(tests dir.parent, "j park")# loop through all test *.py files in tests/# (and its subdirectories)for test file path in tests dir.glob("**/test *.py"):# skip this file: we don't expect there to be a# corresponding source file for this layout enforcerif test file path this files path:continue# construct the expected source pathsource rel dir test file path.relative to(tests dir).parentsource name test file path.name.split("test ", maxsplit 1)[1]source path Path(j park path, source rel dir, source name)error msg (f"{test file path} found, but {source path} missing.")self.assertTrue(source path.is file(), msg error msg)The TestCase class TestDirectoryLayout defines a single test method namedtest tests layout matches j park. Using the pathlib standard library module, the testmethod loops through every test *.py in the tests/ directory and ensures thatthe test *.py file corresponds to a source file. If some of the functions used inthe test shown look unfamiliar, that’s OK. The most important thing to keepin mind is the strategy of writing a test case that forces you and your teammates to follow a pattern and stay organized—that you keep holes out of yourfences. I encourage you to adapt the test into any of your own projects.If you were to duplicate the preceding directory structure and run the testusing the unittest module❮ python3 -m unittest tests/test directory layout.pyyou might see a failure message like the following:AssertionError: False is not true : /home/user/code/tests/test raptor.py Click HERE to purchase this book now. discuss
8found, but /home/user/code/j park/raptor.py missing.The output indicates that test raptor.py does not correspond to an actual sourcefile: it is an orphan. You have successfully added a test that automaticallydetects when other tests are out of position and do not correspond to asource file.If you repeatedly run the test—fixing the failure messages as you go—yourtests/ directory will eventually match the layout of j park/ and will be continuallyenforced. The eventual output of your corrections will look like the following: j park init .py dinosaurs init .py raptor.py t rex.py fences init .py cable.py electrified.pytests init .py dinosaurs init .py test raptor.py test t rex.py fences init .py test cable.py test electrified.pyThink of other ways you can add unit tests that improve your day-to-day livingexperience in a code base. It might be useful to, for example, write a unit testthat enforces any configuration files, CSV files, and so on in your code baseto store their contents in alphabetical order. Having their contents in ordermakes the files more pleasant to read and edit. Anytime you catch yourselfwriting a comment like # Please keep this list in alphabetical order, consider usingPython’s high level tooling to write a test that mandates the constraint instead.In the final section of this chapter, we’ll explore a risk a little more direct thandisorganized file systems: wildcard variable shadowing. Click HERE to purchase this book now. discuss
Python unittest Test Discovery Still Requires _init_.py In Python 3, you generally do not need to create _init_.py files like you did in Python 2. The _init_.py files are included in this example, however, because a longstanding bug prevents Python unittest dis- covery (for example, python3-munittestdiscover--help) from finding test files that don't have _init_.py siblings.10
