Transcription
ANL-21/39 Rev 3.17Mathematics and Computer Science DivisionArgonne National Laboratory9700 South Cass Avenue, Bldg. 240Argonne, IL 60439www.anl.govPETSc/TAO Users ManualRevision 3.17Mathematics and Computer Science DivisionArgonne National Laboratory is a U.S. Department of Energylaboratory managed by UChicago Argonne, LLC
About Argonne National LaboratoryArgonne is a U.S. Department of Energy laboratory managed by UChicago Argonne, LLCunder contract DE-AC02-06CH11357. The Laboratory’s main facility is outside Chicago, at9700 South Cass Avenue, Argonne, Illinois 60439. For information about Argonneand its pioneering science and technology programs, see www.anl.gov.DOCUMENT AVAILABILITYOnline Access: U.S. Department of Energy (DOE) reports produced after 1991 and agrowing number of pre-1991 documents are available free at OSTI.GOV(http://www.osti.gov/), a service of the US Dept. of Energy’s Office of Scientific andTechnical Information.Reports not in digital format may be purchased by the public from theNational Technical Information Service (NTIS):U.S. Department of CommerceNational Technical Information Service5301 Shawnee RdAlexandria, VA 22312www.ntis.govPhone: (800) 553-NTIS (6847) or (703) 605-6000Fax: (703) 605-6900Email: orders@ntis.govReports not in digital format are available to DOE and DOE contractors from theOffice of Scientific and Technical Information (OSTI):U.S. Department of EnergyOffice of Scientific and Technical InformationP.O. Box 62Oak Ridge, TN 37831-0062www.osti.govPhone: (865) 576-8401Fax: (865) 576-5728Email: reports@osti.govDisclaimerThis report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United StatesGovernment nor any agency thereof, nor UChicago Argonne, LLC, nor any of their employees or officers, makes any warranty, express orimplied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus,product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specificcommercial product, process, or service by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply itsendorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of documentauthors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof, ArgonneNational Laboratory, or UChicago Argonne, LLC.
PETSc/TAO Users ManualArgonne National LaboratoryMathematics and Computer Science DivisionPrepared byS. Balay 1 , S. Abhyankar 1,2 , M. F. Adams 3 , S. Benson 1 , J. Brown 1,10 , P. Brune 1 , K. Buschelman 1 , E. M. Constantinescu 1 , L. Dalcin 4 , A. Dener 1 , V. Eijkhout 6 , W. D. Gropp 1 , V. Hapla8, T. Isaac 14 , P. Jolivet 12 , D. Karpeev 1 , D. Kaushik 1 , M. G. Knepley 1,9 , F. Kong 1,11 , S.Kruger 15 , D. A. May 7 , L. Curfman McInnes 1 , R. Tran Mills 1 , L. Mitchell 13 , T. Munson 1 ,J. E. Roman 16 , K. Rupp 1 , P. Sanan 1,8 , J. Sarich 1 , B. F. Smith 17 , S. Zampini 4 , H. Zhang 1,5 ,H. Zhang 1 , and J. Zhang 11Mathematics and Computer Science Division, Argonne National LaboratoryElectricity Infrastructure and Buildings Division, Pacific Northwest National Laboratory3Computational Research, Lawrence Berkeley National Laboratory4Extreme Computing Research Center, King Abdullah University of Science and Technology5Department of Computer Science, Illinois Institute of Technology6Texas Advanced Computing Center, University of Texas at Austin7Department of Earth Sciences, University of Oxford8Institute of Geophysics, ETH Zurich9Department of Computer Science and Engineering, University at Buffalo10Department of Computer Science, University of Colorado, Boulder11Computational Frameworks, Idaho National Laboratory12Toulouse Institute of Compute Science Research13Department of Computer Science, Durham University14College of Computing, Georgia Tech15Tech-X Corporation16DSIC, Universitat Politècnica de València17Flatiron Institute, Simons Foundation2This work was supported by the Office of Advanced Scientific Computing Research, Office of Science, U.S.Department of Energy, under Contract DE-AC02-06CH11357.i
PETSc/TAO Users Manual, Release 3.17.2ii
CONTENTS1 Introduction to PETSc1.1 About This Manual . . . . . . . . . . . .1.2 Getting Started . . . . . . . . . . . . . .1.2.1 Suggested Reading . . . . . . . .1.2.2 Running PETSc Programs . . . .1.2.3 Writing PETSc Programs . . . .1.2.4 Simple PETSc Examples . . . . .1.3 Parallel and GPU Programming . . . . .1.3.1 MPI Parallelism . . . . . . . . . .1.3.2 CPU SIMD parallelism . . . . . .1.3.3 CPU OpenMP parallelism . . . .1.3.4 GPU kernel parallelism . . . . . .1.3.5 GPU stream parallelism . . . . .1.4 Compiling and Running Programs . . . .1.5 Profiling Programs . . . . . . . . . . . . .1.6 Writing C/C or Fortran Applications1.7 Directory Structure . . . . . . . . . . . .1122456121216161616171719202 Programming with PETSc/TAO2.1 Vectors and Parallel Data . . . . . . . . . . . . . . . . . . . . . . .2.1.1 Creating Vectors . . . . . . . . . . . . . . . . . . . . . . .2.1.2 Assembling (putting values in) vectors . . . . . . . . . . .2.1.3 Basic Vector Operations . . . . . . . . . . . . . . . . . . .2.1.4 Local/global vectors and communicating between vectors .2.1.5 DM - Local/global vectors and ghost updates . . . . . . .2.1.6 Application Orderings . . . . . . . . . . . . . . . . . . . .2.2 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.2.1 Creating matrices . . . . . . . . . . . . . . . . . . . . . . .2.2.2 Low-level matrix creation routines . . . . . . . . . . . . .2.2.3 Assembling (putting values into) matrices . . . . . . . . .2.2.4 Basic Matrix Operations . . . . . . . . . . . . . . . . . . .2.2.5 Matrix-Free Matrices . . . . . . . . . . . . . . . . . . . . .2.2.6 Other Matrix Operations . . . . . . . . . . . . . . . . . . .2.2.7 Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . .2.3 KSP: Linear System Solvers . . . . . . . . . . . . . . . . . . . . .2.3.1 Using KSP . . . . . . . . . . . . . . . . . . . . . . . . . . .2.3.2 Solving Successive Linear Systems . . . . . . . . . . . . .2.3.3 Krylov Methods . . . . . . . . . . . . . . . . . . . . . . . .2.3.4 Preconditioners . . . . . . . . . . . . . . . . . . . . . . . .2.3.5 Solving Block Matrices . . . . . . . . . . . . . . . . . . . .23232427313232384041414151525355575758596477.iii
PETSc/TAO Users Manual, Release 3.17.22.3.6 Solving Singular Systems . . . . . . . . . . . . . . . . . . . . . . . . .2.3.7 Using External Linear Solvers . . . . . . . . . . . . . . . . . . . . . .2.4 SNES: Nonlinear Solvers . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.4.1 Basic SNES Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.4.2 The Nonlinear Solvers . . . . . . . . . . . . . . . . . . . . . . . . . .2.4.3 General Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.4.4 Inexact Newton-like Methods . . . . . . . . . . . . . . . . . . . . . .2.4.5 Matrix-Free Methods . . . . . . . . . . . . . . . . . . . . . . . . . . .2.4.6 Finite Difference Jacobian Approximations . . . . . . . . . . . . . . .2.4.7 Variational Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . .2.4.8 Nonlinear Preconditioning . . . . . . . . . . . . . . . . . . . . . . . .2.5 TS: Scalable ODE and DAE Solvers . . . . . . . . . . . . . . . . . . . . . . .2.5.1 Basic TS Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.5.2 DAE Formulations . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.5.3 Using Implicit-Explicit (IMEX) Methods . . . . . . . . . . . . . . . .2.5.4 GLEE methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.5.5 Using fully implicit methods . . . . . . . . . . . . . . . . . . . . . . .2.5.6 Using the Explicit Runge-Kutta timestepper with variable timesteps2.5.7 Special Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.5.8 Monitoring and visualizing solutions . . . . . . . . . . . . . . . . . .2.5.9 Error control via variable time-stepping . . . . . . . . . . . . . . . .2.5.10 Handling of discontinuities . . . . . . . . . . . . . . . . . . . . . . . .2.5.11 Explicit integrators with finite flement mass matrices . . . . . . . . .2.5.12 Using TChem from PETSc . . . . . . . . . . . . . . . . . . . . . . .2.5.13 Using Sundials from PETSc . . . . . . . . . . . . . . . . . . . . . . .2.6 Performing sensitivity analysis . . . . . . . . . . . . . . . . . . . . . . . . . .2.6.1 Using the discrete adjoint methods . . . . . . . . . . . . . . . . . . .2.6.2 Checkpointing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.7 Solving Steady-State Problems with Pseudo-Timestepping . . . . . . . . . .2.8 TAO: Optimization Solvers . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.8.1 Getting Started: A Simple TAO Example . . . . . . . . . . . . . . .2.8.2 TAO Workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.8.3 TAO Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.8.4 Advanced Options . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.8.5 Adding a Solver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.9 High Level Support for Multigrid with KSPSetDM() and SNESSetDM() . .2.9.1 Adaptive Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . .2.10 DMPlex: Unstructured Grids in PETSc . . . . . . . . . . . . . . . . . . . . .2.10.1 Representing Unstructured Grids . . . . . . . . . . . . . . . . . . . .2.10.2 Data on Unstructured Grids (PetscSection) . . . . . . . . . . . . . .2.10.3 Evaluating Residuals . . . . . . . . . . . . . . . . . . . . . . . . . . .2.10.4 Saving and Loading Data with HDF5 . . . . . . . . . . . . . . . . . .2.10.5 Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.10.6 Metric-based mesh adaptation . . . . . . . . . . . . . . . . . . . . . .2.11 PetscDT: Discretization Technology in PETSc . . . . . . . . . . . . . . . . .2.11.1 Quadrature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.11.2 Probability Distributions . . . . . . . . . . . . . . . . . . . . . . . . .2.12 PetscFE: Finite Element Infrastructure in PETSc . . . . . . . . . . . . . . .2.12.1 Using Pointwise Functions to Specify Finite Element Problems . . .2.12.2 Describing a particular finite element problem to PETSc . . . . . . .2.12.3 Assembling finite element residuals and Jacobians . . . . . . . . . . 91791811811831861871901941951951951961961961993 Additional Information2013.1 PETSc for Fortran Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201ivCONTENTS
PETSc/TAO Users Manual, Release 3.17.23.23.33.43.53.63.73.83.93.1.1 Synopsis . . . . . . . . . . . . . . . . . . . . . . . . . . .3.1.2 C vs. Fortran Interfaces . . . . . . . . . . . . . . . . . .3.1.3 Sample Fortran Programs . . . . . . . . . . . . . . . . .Using MATLAB with PETSc . . . . . . . . . . . . . . . . . . . .3.2.1 Dumping Data for MATLAB . . . . . . . . . . . . . . .3.2.2 Sending Data to an Interactive MATLAB Session . . . .3.2.3 Using the MATLAB Compute Engine . . . . . . . . . .3.2.4 Licensing the MATLAB Compute Engine on a cluster .Profiling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.3.1 Basic Profiling Information . . . . . . . . . . . . . . . .3.3.2 Profiling Application Codes . . . . . . . . . . . . . . . .3.3.3 Profiling Multiple Sections of Code . . . . . . . . . . . .3.3.4 Restricting Event Logging . . . . . . . . . . . . . . . . .3.3.5 Interpreting -log info Output: Informative Messages3.3.6 Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.3.7 Saving Output to a File . . . . . . . . . . . . . . . . . .3.3.8 Accurate Profiling and Paging Overheads . . . . . . . .Hints for Performance Tuning . . . . . . . . . . . . . . . . . . .3.4.1 Maximizing Memory Bandwidth . . . . . . . . . . . . .3.4.2 Performance Pitfalls and Advice . . . . . . . . . . . . . .The Use of BLAS and LAPACK in PETSc and external libraries3.5.1 32 or 64 bit BLAS/LAPACK integers . . . . . . . . . . .3.5.2 Shared memory BLAS/LAPACK parallelism . . . . . . .3.5.3 Available BLAS/LAPACK libraries . . . . . . . . . . . .Other PETSc Features . . . . . . . . . . . . . . . . . . . . . . .3.6.1 PETSc on a process subset . . . . . . . . . . . . . . . .3.6.2 Runtime Options . . . . . . . . . . . . . . . . . . . . . .3.6.3 Viewers: Looking at PETSc Objects . . . . . . . . . . .3.6.4 Using SAWs with PETSc . . . . . . . . . . . . . . . . . .3.6.5 Debugging . . . . . . . . . . . . . . . . . . . . . . . . . .3.6.6 Error Handling . . . . . . . . . . . . . . . . . . . . . . .3.6.7 Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . .3.6.8 Parallel Communication . . . . . . . . . . . . . . . . . .3.6.9 Graphics . . . . . . . . . . . . . . . . . . . . . . . . . . .3.6.10 Developer Environments . . . . . . . . . . . . . . . . . .3.6.11 Emacs Users . . . . . . . . . . . . . . . . . . . . . . . . .3.6.12 Visual Studio Code Users . . . . . . . . . . . . . . . . .3.6.13 Vi and Vim Users . . . . . . . . . . . . . . . . . . . . . .3.6.14 Eclipse Users . . . . . . . . . . . . . . . . . . . . . . . .3.6.15 Qt Creator Users . . . . . . . . . . . . . . . . . . . . . .3.6.16 Visual Studio Users . . . . . . . . . . . . . . . . . . . . .3.6.17 XCode Users (The Apple GUI Development System) . .Unimportant and Advanced Features of Matrices and Solvers . .3.7.1 Extracting Submatrices . . . . . . . . . . . . . . . . . .3.7.2 Matrix Factorization . . . . . . . . . . . . . . . . . . . .3.7.3 Unimportant Details of KSP . . . . . . . . . . . . . . . .3.7.4 Unimportant Details of PC . . . . . . . . . . . . . . . .Running PETSc Tests . . . . . . . . . . . . . . . . . . . . . . . .3.8.1 Quick start with the tests . . . . . . . . . . . . . . . . .3.8.2 Understanding test output and more information . . . .Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 5265266268268269269270271275v
PETSc/TAO Users Manual, Release 3.17.2viCONTENTS
CHAPTERONEINTRODUCTION TO PETSC1.1 About This ManualThis manual describes the use of the Portable, Extensible Toolkit for Scientific Computation (PETSc) andthe Toolkit for Advanced Optimization (TAO) for the numerical solution of partial differential equations andrelated problems on high-performance computers. PETSc/TAO is a suite of data structures and routinesthat provide the building blocks for the implementation of large-scale application codes on parallel (andserial) computers. PETSc uses the MPI standard for all distributed memory communication.PETSc/TAO includes a large suite of parallel linear solvers, nonlinear solvers, time integrators, and optimization that may be used in application codes written in Fortran, C, C , and Python (via petsc4py; seeGetting Started ). PETSc provides many of the mechanisms needed within parallel application codes, suchas parallel matrix and vector assembly routines. The library is organized hierarchically, enabling users toemploy the level of abstraction that is most appropriate for a particular problem. By using techniques ofobject-oriented programming, PETSc provides enormous flexibility for users.PETSc is a sophisticated set of software tools; as such, for some users it initially has a much steeper learningcurve than packages such as MATLAB or a simple subroutine library. In particular, for individuals withoutsome computer science background, experience programming in C, C , python, or Fortran and experienceusing a debugger such as gdb or lldb, it may require a significant amount of time to take full advantage ofthe features that enable efficient software use. However, the power of the PETSc design and the algorithmsit incorporates may make the efficient implementation of many application codes simpler than “rolling them”yourself. For many tasks a package such as MATLAB is often the best tool; PETSc is not intended for theclasses of problems for which effective MATLAB code can be written. There are several packages (listed on https://petsc.org/), built on PETSc, that may satisfy your needswithout requiring directly using PETSc. We recommend reviewing these packages functionality beforestarting to code directly with PETSc. PETSc should not be used to attempt to provide a “parallel linear solver” in an otherwise sequentialcode. Certainly all parts of a previously sequential code need not be parallelized but the matrixgeneration portion must be parallelized to expect any kind of reasonable performance. Do not expectto generate your matrix sequentially and then “use PETSc” to solve the linear system in parallel.Since PETSc is under continued development, small changes in usage and calling sequences of routines willoccur. PETSc has been supported for twenty-five years; see mailing list information on our website forinformation on contacting support.1
PETSc/TAO Users Manual, Release 3.17.21.2 Getting StartedPETSc consists of a collection of classes, which are discussed in detail in later parts of the manual (Programming with PETSc/TAO and Additional Information). The important PETSc classes include index sets (IS), including permutations, for indexing into vectors, renumbering, etc; vectors (Vec); matrices (Mat) (generally sparse); over thirty Krylov subspace methods (KSP); dozens of preconditioners, including multigrid, block solvers, and sparse direct solvers (PC); nonlinear solvers (SNES); timesteppers for solving time-dependent (nonlinear) PDEs, including support for differential algebraicequations, and the computation of adjoints (sensitivities/gradients of the solutions) (TS); managing interactions between mesh data structures and vectors, matrices, and solvers (DM); scalable optimization algorithms (Tao).Each class consist of an abstract interface (simply a set of calling sequences; an abstract base class in C )and an implementation for each algorithm and data structure. Thus, PETSc provides clean and effectivecodes for the various phases of solving PDEs, with a uniform approach for each type of problem. Thisdesign enables easy comparison and use of different algorithms (for example, to experiment with differentKrylov subspace methods, preconditioners, or truncated Newton methods). Hence, PETSc provides a richenvironment for modeling scientific applications as well as for rapid algorithm design and prototyping.The classes enable easy customization and extension of both algorithms and implementations. This approachpromotes code reuse and flexibility, and also separates the issues of parallelism from the choice of algorithms.The PETSc infrastructure creates a foundation for building large-scale applications.It is useful to consider the interrelationships among different pieces of PETSc. Numerical Libraries inPETSc is a diagram of some of these pieces. The figure illustrates the library’s hierarchical organization,which enables users to employ the solvers that are most appropriate for a particular problem.1.2.1 Suggested ReadingThe manual is divided into three parts: Introduction to PETSc Programming with PETSc/TAO Additional InformationIntroduction to PETSc describes the basic procedure for using the PETSc library and presents two simpleexamples of solving linear systems with PETSc. This section conveys the typical style used throughout thelibrary and enables the application programmer to begin using the software immediately.Programming with PETSc/TAO explains in detail the use of the various PETSc libraries, such as vectors,matrices, index sets, linear and nonlinear solvers, and graphics. Additional Information describes a varietyof useful information, including profiling, the options database, viewers, error handling, and some details ofPETSc design.PETSc has evolved to become quite a comprehensive package, and therefore this manual can be ratherintimidating for new users. Bear in mind that PETSc can be used efficiently before one understands allof the material presented here. Furthermore, the definitive reference for any PETSc function is always the2Chapter 1. Introduction to PETSc
PETSc/TAO Users Manual, Release 3.17.2Fig. 1.1: Numerical Libraries in PETSc1.2. Getting Started3
PETSc/TAO Users Manual, Release 3.17.2online manual page. Manual pages for all PETSc functions can be accessed here. The manual pages providehyperlinked indices (organized by both concept and routine name) to the tutorial examples and enable easymovement among related topics.Visual Studio Code, Eclipse, Emacs, and Vim users may find their development environment’s options forsearching in the source code (for example, etags and ctags for Emacs and Vim) are extremely useful forexploring the PETSc source code. Details of these feature are provided in Developer Environments.The complete PETSc distribution, manual pages, and additional information are available via the PETSchome page. The PETSc home page also contains details regarding installation, new features and changes inrecent versions of PETSc, machines that we currently support, and a frequently asked questions (FAQ) list.Note to Fortran Programmers: In most of the manual, the examples and calling sequences are given forthe C/C family of programming languages. However, Fortran programmers can use all of the functionalityof PETSc from Fortran, with only minor differences in the user interface. PETSc for Fortran Users providesa discussion of the differences between using PETSc from Fortran and C, as well as several complete Fortranexamples.Note to Python Programmers: To program with PETSc in Python you need to enable Python bindings(i.e petsc4py) with the configure option --with-petsc4py 1. See the PETSc installation guide for moredetails.1.2.2 Running PETSc ProgramsBefore using PETSc, the user must first set the environmental variable PETSC DIR, indicating the fullpath of the PETSc home directory. For example, under the Unix bash shell a command of the form export PETSC DIR HOME/petsccan be placed in the user’s .bashrc or other startup file. In addition, the user may need to set theenvironment variable PETSC ARCH to specify a particular configuration of the PETSc libraries. Note that PETSC ARCH is just a name selected by the installer to refer to the libraries compiled for a particularset of compiler options and machine type. Using different values of PETSC ARCH allows one to switchbetween several different sets (say debug and optimized) of libraries easily. To determine if you need to set PETSC ARCH, look in the directory indicated by PETSC DIR, if there are subdirectories beginning witharch then those subdirectories give the possible values for PETSC ARCH.All PETSc programs use the MPI (Message Passing Interface) standard for message-passing communication[For94]. Thus, to execute PETSc programs, users must know the procedure for beginning MPI jobs on theirselected computer system(s). For instance, when using the MPICH implementation of MPI and many others,the following command initiates a program that uses eight processors: mpiexec -n 8 ./petsc program name petsc optionsPETSc also comes with a script that automatically uses the correct mpiexec for your configuration. PETSC DIR/lib/petsc/bin/petscmpiexec -n 8 ./petsc program name petsc optionsAll PETSc-compliant programs support the use of the -help option as well as the -version option.Certain options are supported by all PETSc programs. We list a few particularly useful ones below; acomplete list can be obtained by running any PETSc program with the option -help. -log view - summarize the program’s performance (see Profiling) -fp trap - stop on floating-point exceptions; for example divide by zero -malloc dump - enable memory tracing; dump list of unfreed memory at conclusion of the run, seeDetecting Memory Allocation Problems,4Chapter 1. Introduction to PETSc
PETSc/TAO Users Manual, Release 3.17.2 -malloc debug - enable memory debugging (by default this is activated for the debugging versionof PETSc), see Detecting Memory Allocation Problems, -start in debugger [noxterm,gdb,lldb] [-display name] - start all processes in debugger. See Debugging, for more information on debugging PETSc programs. -on error attach debugger [noxterm,gdb,lldb] [-display name] - start debugger onlyon encountering an error -info - print a great deal of information about what the program is doing as it runs1.2.3 Writing PETSc ProgramsMost PETSc programs begin with a call toierr PetscInitialize(int *argc,char ***argv,char *file,char *help);if (ierr) return , ierr;which initializes PETSc and MPI. The arguments argc and argv are the command line arguments deliveredin all C and C programs. The argument file optionally indicates an alternative name for the PETScoptions file, .petscrc, which resides by default in the user’s home directory. Runtime Options providesdetails regarding this file and the PETSc options database, which can be used for runtime customization.The final argument, help, is an optional character string that will be printed if the program is run with the-help option. In Fortran the initialization command has the formcall PetscInitialize(character(*) file,integer ierr)PetscInitialize() automatically calls MPI Init() if MPI has not been not previously initialized. Incertain circumstances in which MPI needs to be initialized directly (or is initialized by some other library),the user can first call MPI Init() (or have the other library do it), and then call PetscInitialize(). By default, PetscInitialize() sets the PETSc “world” communicator PETSC COMM WORLDto MPI COMM WORLD.For those not familiar with MPI, a communicator is a way of indicating a collection of processes that willbe involved together in a calculation or communication. Communicators have the variable type MPI Comm.In most cases users can employ the communicator PETSC COMM WORLD to indicate all processes in a givenrun and PETSC COMM SELF to indicate a single process.MPI provides routines for generating new communicators consisting of subsets of processors, though mostusers rarely need to use these. The book Using MPI, by Lusk, Gropp, and Skjellum [GLS94] provides anexcellent introduction to the concepts in MPI. See also the MPI homepage. Note that PETSc users neednot program much message passing directly with MPI, but they must be familiar with the basic concepts ofmessage passing and distributed memory computing.All PETSc routines return a PetscErrorCode, which is an integer indicating whether an error has occurredduring the call. The error code is set to be nonzero if an error has been detected; otherwise, it is zero. Forthe C/C interface, the error variable is the routine’s return value, while for the Fortran version, eachPETSc routine has as its final argument an integer error variable.All PETSc programs should call PetscFinalize() as their final (or nearly final) statement, as givenbelow in the C/C and Fortran formats, respectively:ierr PetscFinalize();return ierr;call PetscFinalize(ierr)1.2. Getting Started5
PETSc/TAO Users Manual, Release 3.17.2This routine handles options to be called at the conclusion of the program, and calls MPI Finalize()if PetscInitialize() began MPI. If MPI was initiated externally from PETSc (by either the user oranother software package), the user is responsible for calling MPI Finalize().1.2.4 Simple PETSc ExamplesTo help the user start using PETSc immediately, we begin with a simple uniprocessor example that solvesthe one-dime
PETSc/TAO Users Manual Revision 3.17 Mathematics and Computer Science Division. About Argonne National Laboratory Argonne is a U.S. Department of Energy laboratory managed by UChicago Argonne, LLC under contract DE -AC02 -06CH 11357. The Laboratory ¶s main facility is outside Chicago, at
