Transcription
National Aeronautics and Space AdministrationRead, Map and Extract MODIS Aerosol Data UsingPython ScriptsAdvanced Webinar: Data Analysis Tools for High Resolution Air Quality Satellite DatasetsPawan Gupta & Melanie Follette-Cook, January 17-22, 2018
Computer and Python Requirements
Operating System Both session 2 & session 3 will cover python scripts to perform similar tasks. Session 2 is presented using a mac (apple) operating system. All the directory structure and python layout will be according to mac operatingsystem. Session 3 is presented using a windows machine. Showing both operating systems will allow end-users to get a feeling of the layoutof both operating systems. We expect all codes to run smoothly on mac, windows, and linux operatingsystems.NASA’s Applied Remote Sensing Training Program3
Computer Requirements Install Python 2.7 using Anaconda Install all required python packages– Package List (right) Test python and package installationsusing following python test code– test python.py Download MODIS Data and PythonCodes using following link– ARSET LINK ZIP FILE For more detail on the code, ’s Applied Remote Sensing Training Program Python package list:–– pyhdf–– numpy– sys–– mpl ormaph5pytimecalendar4
Python Test Open the spyder editorinside Anaconda Open test python.py Make sure the directoryhas the python codeand HDF file Open the ipythonconsole in the spyder Run the code using thegreen arrow on the top Output should be animage as shownNASA’s Applied Remote Sensing Training ProgramHDF fileoutput5
Know your Data
Understanding a MODIS File NameLevel 2, 10 km, Aerosol ProductProduct Name Terra: MOD04 Aqua: MYD04TimeFile processing informationMOD04 L2.A2001079.0255.006.2006289012028.hdfDate Year Julian DayCollectionHDFLook, Panoply, IDL, Python, Fortran, MatLab, and more can be used to read the dataNASA’s Applied Remote Sensing Training Program7
Understanding a MODIS File NameLevel 2, 3 km, Aerosol ProductProduct Name Terra: MOD04 Aqua: MYD04TimeFile processing informationMOD04 3K.A2001079.0255.006.2006289012028.hdfDate Year Julian DayCollectionHDFLook, Panoply, IDL, Python, Fortran, MatLab, and more can be used to read the dataNASA’s Applied Remote Sensing Training Program8
MODIS Aerosol Parameters (SDS) Optical Depth Land and Ocean– Retrieved using Dark Target Algorithm– Only high quality data Over land QA 3 Over ocean QA 1, 2, 3– 10 km and 3km Dark Target Deep Blue Optical Depth 550 Combined– Deep Blue & Dark Target Algorithm Merged Product– 10 km only Quality Assurance Land– Quality flag associated with DD productNASA’s Applied Remote Sensing Training Program9
Quality Assurance is Extremely ImportantQA indicates confidence in the quality of the retrievalQuality Assurance OceanQuality Assurance Land Scale is 0-3 Scale is 0-3 Recommended Ocean QA above 1,2, 3 Recommended Land QA of 3 Factors:– Number of pixels– Error fitting– How close to glintNASA’s Applied Remote Sensing Training Program Factors:– Number of pixels– Error fitting– Surface reflectance10
Getting Ready with Python
Anaconda & the Spyder EditorNASA’s Applied Remote Sensing Training Program12
Spyder Viewcode areaipython areaNASA’s Applied Remote Sensing Training Program13
Current Directory View & File List Create a list of HDF files ‘fileList.txt’ The directory should have– All the python codes– All the HDF data files– A list of HDF files named as ‘fileList.txt’NASA’s Applied Remote Sensing Training Program14
Read a MODIS Aerosol File (HDF)and Print SDS List
Print Scientific Data Sets (SDSs)read mod aerosol and list sds.py Purpose: read aMODIS aerosol level 2data file in HDF formatand print all theScientific Data Sets(SDS) The code works forboth 10 km and 3 kmproductsNASA’s Applied Remote Sensing Training Program16
Running and Output Click the greenarrow to run thecode The code willprocess all the filesin the fileList.txtone-by-one Follow theinstructions in theipython terminal(i.e. enter ‘Y’ or‘N’ whenprompted andhit enter)NASA’s Applied Remote Sensing Training Programoutput17
Editing the Codechange the nameof fileList.txt to anyname you’d likeThis code hasbeen tested for 3km and 10 kmMODIS aerosolLevel 2 data filesNASA’s Applied Remote Sensing Training Program18
Applications MODIS Level 2 aerosol data are provided in HDF files Each HDF file contains several geophysical parameters Special codes and tools are required to open HDF files This code helps users see the name of the available SDSs inside an HDF file forfurther analysisNASA’s Applied Remote Sensing Training Program19
Map Aerosol Optical Depth
Plot and save a map of MODIS AODread and map mod aerosol.pyNASA’s Applied Remote Sensing Training Program21
Running and OutputAOD statisticsOutput AODmapNASA’s Applied Remote Sensing Training Program22
Editing the CodeChange the Color ScaleChange the SDShttps://matplotlib.org/examples/color/colormaps reference.htmlNASA’s Applied Remote Sensing Training Program23
Applications This is a sample code to read and map the MODIS Level 2 aerosol data The code can be modified to address different mapping needs Users can create daily maps of AOD over certain regions and start analyzingchanges over time AOD maps can also help identify regions with high pollution levelsExample:High AOD values fromsmoke show goodagreement withsurface monitors(circles).NASA’s Applied Remote Sensing Training ProgramMay 14, 2007May 15, 2007May 16, 200724
Extract AOD at a Surface Station
Extract AOD Values at a given locationread mod aerosol at a location.py Purpose: read aMODIS aerosol level 2data file in HDF formatand extract AODvalues at a givenground location The code works forboth 10 km and 3 kmproductsNASA’s Applied Remote Sensing Training Program26
Running and OutputType “Y” to process file,“N” to skipSelect SDSLat & Lon of XXXXXXXXXXXXXNASA’s Applied Remote Sensing Training Program27
Editing the Code – Change the SDSNASA’s Applied Remote Sensing Training Program28
Editing the Code – Change the AOD CalculationsNASA’s Applied Remote Sensing Training Program29
ApplicationsSatellite AOD ValidationNASA’s Applied Remote Sensing Training ProgramAOD-PM2.5 RelationshipTime Series Analysis30
Output HDF variables to CSV
Output MODIS Aerosol Level 2 HDF variables to a CSV fileread mod aerosol and dump ascii.py Purpose: read aMODIS aerosol level 2data file in HDF formatand write certain SDSsinto a csv (text) file The code works forboth 10 km and 3 kmproductsNASA’s Applied Remote Sensing Training Program32
OutputThis code saves a .csvfile, which can beopened by excel, atext editor, or othercodes or softwareNASA’s Applied Remote Sensing Training Program33
Editing the CodeChange the listSDS to be writtenas an outputNASA’s Applied Remote Sensing Training Program34
Applications This is a sample code to read and extract the MODIS Level 2 aerosol data The code can be modified to extract multiple SDSs into a single .csv file The code be easily modified to extract data over a certain region The output file can be opened in excel, or any other data analysis toolNASA’s Applied Remote Sensing Training Program35
Create Air Quality Maps
Create an Air Quality Mapread aod and calculate pm25.py Purpose: read a MODISaerosol level 2 data file inHDF format and create aPM2.5 air quality categorymap using the relationshipbetween AOD and PM2.5 The code works for both 10km and 3 km productsoutputNASA’s Applied Remote Sensing Training Program37
Create an Air Quality Mapread aod and calculate pm25.pyDisclaimer: This is just anexample code. The defaultAOD-PM2.5 relationship usedhere is assumed relationshipover USA.The users of this code areresponsible for checking thevalidity of this relationshipand encouraged to use localrelationships for visualizing AQin different parts of the world.NASA’s Applied Remote Sensing Training Programoutput38
Editing the Code – Change the SDSThe user canchange theAOD SDS tobe used inPM2.5calculationNASA’s Applied Remote Sensing Training Program39
Editing the Code: Change the AOD-PM2.5 Relationship and AQIThe code usesPM2.5 Slope * AOD Interceptas the linear regression equationto calculate PM2.5 from AODThe code uses the U.S. EPAdefinition of air quality categoriesbased on PM2.5Change thedefault slope &interceptChange the airquality categoriesAQI Calculator:https://airnow.gov/index.cfm?action airnow.calculatorNASA’s Applied Remote Sensing Training Program40
Application: Convert AOD into PM2.5 & Air Quality MapsNASA’s Applied Remote Sensing Training Program41
Application – Compare Satellite with Surface MapsNASA’s Applied Remote Sensing Training Program42
Questions & Answers
Read, Map and Extract MODIS Aerosol Data Using Python Scripts Advanced Webinar: Data Analysis Tools for High Resolution Air Quality Satellite Datasets Pawan Gupta & Melanie Follette-Cook, January 17-22, 2018. Computer and Python Requi