Beach Monitoring Database User Guide (For Use With WQX)

Transcription

Beach Monitoring Database User Guide (For use with WQX)Contract Number GS-35F-4797HTask Order # 1518TDD 10.02 eBeachesVersion 3.0June 25, 2020

Table of ContentsTable of Contents . 21List of Exhibits . 42Revision Log . 43Introduction. 63.14Reference Materials . 6Before Submitting Data . 74.1NAAS Account . 74.2WQX Organization ID . 74.3Submission Method Selection . 74.4Table Details. 75Entering Data into the Database . 115.16Special Cases. 12Preparing Update/Insert XML Submissions . 166.1Process Overview. 166.2Excluding Data from Submissions . 167Preparing Delete XML Submissions . 208Loading XML . 219Submitting Files. 229.1Preparing the File . 229.2The First Submission . 229.3Submitting Files . 229.4Retrieving Submission Results Using Node Client Lite . 2310Appendix A—Microsoft Access . 2511Appendix B—Monitoring Database Change Log . 2712Appendix C—Table Relationships . 2913Appendix D—Data Elements . 2913.1Organization . 3013.2Project. 3313.3Monitoring Location . 3513.4Activity . 4013.5Result. 5014Appendix E—Data Entry Forms . 62Beach Monitoring Database User Guide July 21, 20102

14.1Forms Tab . 6214.2Return to Main Menu . 6514.3Generate Insert/Update XML . 671515.116Appendix F—Summary Reports . 68Reports Tab . 68Appendix G—CDX Messages . 7016.1WQX Submission Status . 7016.2Transaction History .71Beach Monitoring Database User Guide July 21, 20103

1List of ExhibitsExhibit 2-1 Other Naming ConventionsExhibit 4-1 Generate Insert/Update XML Form4Exhibit 4-3 Send to EPA Flag ManagerExhibit 4-4 Send To EPA Flag TablesExhibit 6-1 Load XML Form and Me List of ExhibitsExhibit 2-1 Other Naming Conventions 1Exhibit 4-1 Generate Insert/Update XML Form4Exhibit 4-3 Send to EPA Flag ManagerExhibit 4-4 Send To EPA Flag TablesExhibit 6-1 Load XML Form and MessageExhibit 8-1 Beach Monitoring DatabaseExhibit 8-2 ORG PHONE TableExhibit 10-1 Table RelationshipsExhibit 12-1 Microsoft Access 2000 Forms TabExhibit 12-2 Microsoft Access 2007 Forms TabExhibit 12-3 Microsoft Access 2007 Main MenuExhibit 12-4 Microsoft Access 2000 Project Data Entry ScreenExhibit 12-5 Microsoft Access 2007 Project Data Entry ScreenExhibit 12-6 Microsoft Access 2007 Generate Insert/Update XML FormExhibit 13-1 Microsoft Access 2000 Reports TabExhibit 13-2 Microsoft Access 2007 Reports TabExhibit 14-1 WQX Submission StatusExhibit 14-2 WQX Submission Status – FailedExhibit 14-3 Transaction History Message2Revision LogDateVersionNo.DescriptionAuthor10/1/072.0Final VersionAndrewHampton10/10/072.0.1Added relationship diagramAndrewHampton10/22/072.0.2Added ID information to the table in section2.4.3Added Change Log AppendixAndrewHampton11/7/072.0.3Updated Change Log AppendixAdded Analytical Method InformationSectionAndrewHampton11/8/072.0.4Updated Change Log AppendixAndrewHampton2/4/082.1.0Updated Change Log AppendixAdded Load XML SectionErik RichardsReviewerBeach Monitoring Database User Guide July 21, 2010ReviewDate4

2/25/082.1.1Updated Change Log AppendixErik Richards7/2/082.1.2Updated Change Log AppendixErik Richards12/17/082.1.3Updated Change Log AppendixErik Richards8/3/092.2.0Updated Change Log AppendixAdded Appendix EErik Richards7/21/102.2.1Updated Change Log AppendixUpdated Appendix E & FAdded Appendix GErik Richards6/12/20203.0Updated Change Log AppendixUpdated Appendix BUpdated Appendix DUpdated Appendix EChris StevensonBeach Monitoring Database User Guide July 21, 20105

3IntroductionThe purpose of this document is to give instruction on how the WQX Access Database can be usedto generate XML submission files for the WQX and how the file should be submitted. A workingknowledge of the WQX XML Schema is very helpful when working with WQX data, and links todocuments which detail the WQX XML are provided in the Reference Materials Section. If youare looking for more general information about WQX or the Beach Act Grant Program, please firstvisit the links in the Reference Materials section.3.1Reference MaterialsFor more information about the BEACH Act Grant Program or if you are new to the BEACH ActGrant Program, visit these links:General Beach Program Information – https://www.epa.gov/beachesSubmitting Beach Data to EPA - ta-epaNote particularly on that webpage: how to get credentials to submit beach data, how to use theExchange Network, and the sub-page on submitting Monitoring data.The WQX team within the EPA has written several documents which are very useful forunderstanding WQX data and the WQX data submission process. Though these documents aremore technical in nature, understanding their contents is very beneficial. These documents arelocated at the following URLs:Basic information about WQX - https://www.epa.gov/storet/wqx/index.htmlDocument Downloads - https://www.epa.gov/storet/wqx/wqx downloads.htmlInformation on WQX’s Exchange Network presence x.htmOne document that is particularly helpful is the WQX XML Training for Beach Monitoring 14-09/documents/wqx-xml-trainingmanual2013.pdf). The WQX XML Training for Beach Monitoring Data explains the details and businessrules about a submission file.Since all WQX submission files will be traveling over the Exchange Network, a basicunderstanding of the Exchange Network is also beneficial. The Network Basics section and FAQssection of https://www.exchangenetwork.net/ give a good overview of how the Exchange Networkworks and its purpose.The underlying technology of the Exchange Network and submission files is XML. To learn moreabout XML, visit http://www.w3schools.com/xml/xml syntax.asp.BeachMonitoring Database User GuideJune 17, 20206

4Before Submitting DataThese are the steps that must be taken before a submission can be made.4.1NAAS AccountA Network Authorization and Authentication Service, or NAAS (pronounced naz), account isrequired to submit files on the Exchange Network. To request a NAAS account, contact themanager of the eBeaches System, Bill Kramer at kramer.bill@epa.gov You may also need tocontact your jurisdiction’s Node Administrator1.4.2WQX Organization IDThe WQX Organization ID can be created by contacting the STORET help desk by email atstoret@epa.gov or by phone at 1-800-424-9067. Also, there is a process in place to migrate datafrom STORET to WQX which will enable states to keep the same Organization ID for WQXsubmissions. Contact the help desk for more information about this data migration process.4.3Submission Method SelectionThere are several options available for choosing a method to submit data to the WQX. None ofthese options affect how data is entered into the database or how the submission file is generated.The submission method chosen will determine what is done with the submission file after it isgenerated. These options are outlined in a document located ts/datausers/index.htm4.4Table DetailsThis section explains the business rules of the database and gives some helpful tips. For basicinformation about Microsoft Access, see Appendix A – Microsoft Access.2.4.1Foreign KeysThere are many columns that will be filled out that require a special value which is defined inanother table. The table and column that contains the special value are called the reference tableand reference key, respectively. The column that uses the reference key is called a foreign key.For example, in the ORG PHONE table, there is a foreign key column which stores the type ofphone number, REF PHONE TYPE NAME. Valid types of phone numbers are stored in theREF PHONE TYPE NAME column of the REF PHONE TYPE NAME table. At the time of1You can find your state’s Node Administrator’s contact information here: eachMonitoring Database User GuideJune 17, 20207

this writing, the valid types of phone numbers which are stored in the reference table are Fax,Home, Mobile, Office, and Pager.BeachMonitoring Database User GuideJune 17, 20208

2.4.1.1Foreign Key Naming ConventionsThere are two naming categories for foreign key columns: Ref columns and UID columns2.4.1.1.1REF ColumnsTables that begin with “REF” are reference tables and contain reference keys. The columns thatbegin with “REF” which are not in a “REF” table are foreign keys.Generally, each foreign key column whose name begins with “REF” references values which arestored in an identically named reference table which contains an identically named reference keycolumn. The phone type example followed this convention. Two notable exceptions to thisgeneral rule are time zone information and unit of measure information. Time zone reference keysare stored in the REF TIME ZONE table and time zone foreign keys have TMZONE in theirname. Unit of measure reference keys are stored in the REF MEASUREMENT UNIT table andunit of measure foreign keys have MSUNIT in their name.2.4.1.1.2UID ColumnsMost tables have a column which ends with “UID” which is automatically generated and uniquelyidentifies that row of data. For example, in the ACTIVITY table, each row has a unique numberin the ACT UID column. Tables may also have other columns which end in “UID” which relaterows of data from different tables. For example, the ACTIVITY table has a column namedORG UID which signifies which organization conducted the Activity. Also, the ACTIVITY tablehas a column named PRJ UID which signifies which project the activity is associated with. Tosummarize, columns which end with UID are either the unique identifier for the row or a foreignkey which relates rows from different tables.2.4.2Reference Data UpdatesThe date contained within the reference tables2 are subject to change based on requests foradditions made to the WQX team by the state users. The tables are currently based on the valuesfrom 9/28/06. Updates to the reference data will be published periodically.Using the Node Client Lite, the following steps can be followed to get the current valid domainvalues.1. Connect to a node in Node Client Lite2. Under “Things I can do”, click “Get Data”3. For the Data Flow, select WQX2Reference tables are tables which begin with “REF”BeachMonitoring Database User GuideJune 17, 20209

4. For the Service, select “WQX.GetDomainValueByElementName v1.0”5. Enter the Element Namea. A valid list of elements names is available on page 15 of the WQX XML TrainingManual. See the Reference Materials for the location of the WQX XML TrainingManual6. Click Submit7. A results screen will come up and show you where the Result File is stored8. Navigate to the result file, and open the result file with Internet Explorer2.4.3Other Naming ConventionsExhibit 2-1 Other Naming ConventionsAbbreviationCDDescriptionCD stands for Code. This typically means there is a specific abbreviation(code) that is needed for the column. For example, in the ORG ADDRESStable, there is a REF STATE CD column. In this case, valid values for thiscolumn are the two digit state codes (AK, AL, etc.) which are located in theREF STATE CD column of the REF STATE CD table.IDAll columns that end with ID (e.g. ACT ID, MLOC ID, etc) must be unique.Whenever a new set of data is submitted with a previously existing ID, the olddata is overwritten. For example, if an Activity in the 2004 data with anACT ID of CO123456-01 has been submitted, and a new Activity in the 2005data has the same ACT ID, the activity from the 2004 data will be lost. Acommon naming practice is to combine multiple pieces of data to for an ID.For example, the combination of the Station ID, the date, and time of anActivity would be a good ACT ID.BeachMonitoring Database User GuideJune 17, 202010

5Entering Data into the DatabaseThe order data should be entered into the database is not strictly enforced, but a basicunderstanding of how the XML is generated will help explain how data should be entered. TheXML generation starts in the ORGANIZATION table and works its way through the tables in thisorder:ORGANIZATIONORG ELECTRONIC ADDRESSORG PHONEORG ADDRESSPROJECT3ATTACHED OBJECTMONITORING LOCATIONMONITORING LOCATION ALTERNATEATTACHED OBJECTACTIVITYACTIVITY PROJECTACTIVITY CONDUCTING ORGATTACHED OBJECT RESULT RESULT DETECT QUANT LIMIT RESULT LAB SAMPLE PREP ATTACHED OBJECTACTIVITY GROUPACTIVITY GROUP DETAILThe most important thing to note about this is that data in “child tables” will be ignored unless itrelates to a row in its “parent table”. For example, any data in the RESULT table will be ignoredunless it relates to a row in its parent table, ACTIVITY. Also, the row in the ACTIVITY tablewill be ignored unless it relates to a row in its parent table, ORGANIZATION.3Methods of creating Project IDs before WQX resulted in multiple beach names assigned to a single Project ID. To avoid that error using thisdatabase, verify that all values in the PRJ ID column of the PROJECT table are unique.BeachMonitoring Database User GuideJune 17, 202011

5.1Special CasesMost of the tables and columns are fairly straightforward; however there are some that can beconfusing. The following sections are descriptions of how to handle the data which is atypical.3.1.1Attached Binary ObjectsSince there can be multiple Attached Binary Objects associated with Results, Projects, MonitoringLocations, and Activities, there is a separate table to identify the relationships. To add anAttached Binary Object to the database, follow these steps:1. Open the ATTACHED OBJECT table in the database2. In the FILE NAME column, enter the file name of the object including the extension3. In the FILE TYPE column, enter the file extension of the file you are attaching. Forexample, if you are attaching lab results in an Excel file, you would enter “xls”.4. In the appropriate column, enter the UID you wish to associate the file with5. Verify that the SEND TO EPA column is checked6. After creating the XML submission file, add the submission file and all Attached Objectsto a zip file7. Submit the zip file3.1.2Activity Project IDsSince there can be multiple Project IDs4 associated with an Activity, there is a separate table toidentify the relationships. The ACTIVITY PROJECT table is used to associate Projects andActivities. To add associations, follow these steps:1. Open the ACTIVITY PROJECT table2. Enter the data from the ACTIVITY.ACT UID column into theACTIVITY PROJECT.ACT UID column3. Enter the data from the PROJECT.PRJ UID column into theACTIVITY PROJECT.PRJ UID column4. Close the ACTIVITY PROJECT table3.1.3Activity Conducting OrganizationSince there can be multiple Organizations involved in conducting an Activity, there is a separatetable to identify the relationship. The ACTIVITY CONDUCTING ORG table is used toassociate Activities and their Conducting Organizations. To add associations, follow these steps:4Project IDs and Beach IDs are the same thing. Monitoring data uses the term Project ID, and Notification Data uses the term Beach ID.BeachMonitoring Database User GuideJune 17, 202012

1. Open the ACTIVITY CONDUCTING ORG table2. Enter the data from the ACTIVITY.ACT UID column into theACTIVITY CONDUCTING ORG.ACT UID column3. Enter the name of the activity’s conducting organization in theACTIVITY CONDUCTING ORG.ACORG NAME column4. Close the ACTIVITY CONDUCTING ORG table3.1.4Activity Group Activity IdentifiersSince there can be multiple Activity Identifiers associated with an Activity Group, there is aseparate table to id

Monitoring Database User Guide June 17, 2020 7 4 Before Submitting Data These are the steps that must be taken before a submission can be made. 4.1 NAAS Account A Network Authorization and Authentication Service, or NAAS (pronounced naz), account is required to submit files on the Exchange