Read, Map And Extract MODIS Aerosol Data Using Python

Transcription

National Aeronautics and Space AdministrationRead, Map and Extract MODIS Aerosol Data UsingPython ScriptsAdvanced Webinar: Data Analysis Tools for High Resolution Air Quality Satellite DatasetsPawan Gupta & Melanie Follette-Cook, January 17-22, 2018

Computer and Python Requirements

Operating System Both session 2 & session 3 will cover python scripts to perform similar tasks. Session 2 is presented using a mac (apple) operating system. All the directory structure and python layout will be according to mac operatingsystem. Session 3 is presented using a windows machine. Showing both operating systems will allow end-users to get a feeling of the layoutof both operating systems. We expect all codes to run smoothly on mac, windows, and linux operatingsystems.NASA’s Applied Remote Sensing Training Program3

Computer Requirements Install Python 2.7 using Anaconda Install all required python packages– Package List (right) Test python and package installationsusing following python test code– test python.py Download MODIS Data and PythonCodes using following link– ARSET LINK ZIP FILE For more detail on the code, ’s Applied Remote Sensing Training Program Python package list:–– pyhdf–– numpy– sys–– mpl ormaph5pytimecalendar4

Python Test Open the spyder editorinside Anaconda Open test python.py Make sure the directoryhas the python codeand HDF file Open the ipythonconsole in the spyder Run the code using thegreen arrow on the top Output should be animage as shownNASA’s Applied Remote Sensing Training ProgramHDF fileoutput5

Know your Data

Understanding a MODIS File NameLevel 2, 10 km, Aerosol ProductProduct Name Terra: MOD04 Aqua: MYD04TimeFile processing informationMOD04 L2.A2001079.0255.006.2006289012028.hdfDate Year Julian DayCollectionHDFLook, Panoply, IDL, Python, Fortran, MatLab, and more can be used to read the dataNASA’s Applied Remote Sensing Training Program7

Understanding a MODIS File NameLevel 2, 3 km, Aerosol ProductProduct Name Terra: MOD04 Aqua: MYD04TimeFile processing informationMOD04 3K.A2001079.0255.006.2006289012028.hdfDate Year Julian DayCollectionHDFLook, Panoply, IDL, Python, Fortran, MatLab, and more can be used to read the dataNASA’s Applied Remote Sensing Training Program8

MODIS Aerosol Parameters (SDS) Optical Depth Land and Ocean– Retrieved using Dark Target Algorithm– Only high quality data Over land QA 3 Over ocean QA 1, 2, 3– 10 km and 3km Dark Target Deep Blue Optical Depth 550 Combined– Deep Blue & Dark Target Algorithm Merged Product– 10 km only Quality Assurance Land– Quality flag associated with DD productNASA’s Applied Remote Sensing Training Program9

Quality Assurance is Extremely ImportantQA indicates confidence in the quality of the retrievalQuality Assurance OceanQuality Assurance Land Scale is 0-3 Scale is 0-3 Recommended Ocean QA above 1,2, 3 Recommended Land QA of 3 Factors:– Number of pixels– Error fitting– How close to glintNASA’s Applied Remote Sensing Training Program Factors:– Number of pixels– Error fitting– Surface reflectance10

Getting Ready with Python

Anaconda & the Spyder EditorNASA’s Applied Remote Sensing Training Program12

Spyder Viewcode areaipython areaNASA’s Applied Remote Sensing Training Program13

Current Directory View & File List Create a list of HDF files ‘fileList.txt’ The directory should have– All the python codes– All the HDF data files– A list of HDF files named as ‘fileList.txt’NASA’s Applied Remote Sensing Training Program14

Read a MODIS Aerosol File (HDF)and Print SDS List

Print Scientific Data Sets (SDSs)read mod aerosol and list sds.py Purpose: read aMODIS aerosol level 2data file in HDF formatand print all theScientific Data Sets(SDS) The code works forboth 10 km and 3 kmproductsNASA’s Applied Remote Sensing Training Program16

Running and Output Click the greenarrow to run thecode The code willprocess all the filesin the fileList.txtone-by-one Follow theinstructions in theipython terminal(i.e. enter ‘Y’ or‘N’ whenprompted andhit enter)NASA’s Applied Remote Sensing Training Programoutput17

Editing the Codechange the nameof fileList.txt to anyname you’d likeThis code hasbeen tested for 3km and 10 kmMODIS aerosolLevel 2 data filesNASA’s Applied Remote Sensing Training Program18

Applications MODIS Level 2 aerosol data are provided in HDF files Each HDF file contains several geophysical parameters Special codes and tools are required to open HDF files This code helps users see the name of the available SDSs inside an HDF file forfurther analysisNASA’s Applied Remote Sensing Training Program19

Map Aerosol Optical Depth

Plot and save a map of MODIS AODread and map mod aerosol.pyNASA’s Applied Remote Sensing Training Program21

Running and OutputAOD statisticsOutput AODmapNASA’s Applied Remote Sensing Training Program22

Editing the CodeChange the Color ScaleChange the SDShttps://matplotlib.org/examples/color/colormaps reference.htmlNASA’s Applied Remote Sensing Training Program23

Applications This is a sample code to read and map the MODIS Level 2 aerosol data The code can be modified to address different mapping needs Users can create daily maps of AOD over certain regions and start analyzingchanges over time AOD maps can also help identify regions with high pollution levelsExample:High AOD values fromsmoke show goodagreement withsurface monitors(circles).NASA’s Applied Remote Sensing Training ProgramMay 14, 2007May 15, 2007May 16, 200724

Extract AOD at a Surface Station

Extract AOD Values at a given locationread mod aerosol at a location.py Purpose: read aMODIS aerosol level 2data file in HDF formatand extract AODvalues at a givenground location The code works forboth 10 km and 3 kmproductsNASA’s Applied Remote Sensing Training Program26

Running and OutputType “Y” to process file,“N” to skipSelect SDSLat & Lon of XXXXXXXXXXXXXNASA’s Applied Remote Sensing Training Program27

Editing the Code – Change the SDSNASA’s Applied Remote Sensing Training Program28

Editing the Code – Change the AOD CalculationsNASA’s Applied Remote Sensing Training Program29

ApplicationsSatellite AOD ValidationNASA’s Applied Remote Sensing Training ProgramAOD-PM2.5 RelationshipTime Series Analysis30

Output HDF variables to CSV

Output MODIS Aerosol Level 2 HDF variables to a CSV fileread mod aerosol and dump ascii.py Purpose: read aMODIS aerosol level 2data file in HDF formatand write certain SDSsinto a csv (text) file The code works forboth 10 km and 3 kmproductsNASA’s Applied Remote Sensing Training Program32

OutputThis code saves a .csvfile, which can beopened by excel, atext editor, or othercodes or softwareNASA’s Applied Remote Sensing Training Program33

Editing the CodeChange the listSDS to be writtenas an outputNASA’s Applied Remote Sensing Training Program34

Applications This is a sample code to read and extract the MODIS Level 2 aerosol data The code can be modified to extract multiple SDSs into a single .csv file The code be easily modified to extract data over a certain region The output file can be opened in excel, or any other data analysis toolNASA’s Applied Remote Sensing Training Program35

Create Air Quality Maps

Create an Air Quality Mapread aod and calculate pm25.py Purpose: read a MODISaerosol level 2 data file inHDF format and create aPM2.5 air quality categorymap using the relationshipbetween AOD and PM2.5 The code works for both 10km and 3 km productsoutputNASA’s Applied Remote Sensing Training Program37

Create an Air Quality Mapread aod and calculate pm25.pyDisclaimer: This is just anexample code. The defaultAOD-PM2.5 relationship usedhere is assumed relationshipover USA.The users of this code areresponsible for checking thevalidity of this relationshipand encouraged to use localrelationships for visualizing AQin different parts of the world.NASA’s Applied Remote Sensing Training Programoutput38

Editing the Code – Change the SDSThe user canchange theAOD SDS tobe used inPM2.5calculationNASA’s Applied Remote Sensing Training Program39

Editing the Code: Change the AOD-PM2.5 Relationship and AQIThe code usesPM2.5 Slope * AOD Interceptas the linear regression equationto calculate PM2.5 from AODThe code uses the U.S. EPAdefinition of air quality categoriesbased on PM2.5Change thedefault slope &interceptChange the airquality categoriesAQI Calculator:https://airnow.gov/index.cfm?action airnow.calculatorNASA’s Applied Remote Sensing Training Program40

Application: Convert AOD into PM2.5 & Air Quality MapsNASA’s Applied Remote Sensing Training Program41

Application – Compare Satellite with Surface MapsNASA’s Applied Remote Sensing Training Program42

Questions & Answers

Read, Map and Extract MODIS Aerosol Data Using Python Scripts Advanced Webinar: Data Analysis Tools for High Resolution Air Quality Satellite Datasets Pawan Gupta & Melanie Follette-Cook, January 17-22, 2018. Computer and Python Requi