Site Remediation News June 1999 (Vol 11 N0 1) Article 01

Updates To Electronic Data Submission Requirements

By: David Haymes
Division of Publicly Funded Site Remediation
Bureau of Planning and Systems

The Site Remediation Program has updated the Site Remediation Program Electronic Data Interchange (SRP-EDI) manual, and is in the process of updating the Electronic Data Submission Application (EDSA checker), and the Hazsite application.

Updates to the Site Remediation Program — Electronic Data Interchange (SRP-EDI) Manual

Updates to the Site Remediation Program Electronic Data Interchange (SRP-EDI) manual include providing expanded explanations of the various means of creating and submitting the dataset files to the SRP, as well as correcting typographical errors. A section has been added explaining the “hybrid” option, in which the responsible party/consultant creates the DTST and HZSAMPLE files, and then incorporates the HZRESULT file created by the laboratory. A section has also been added for saving the dataset files in the ASCII format. In addition, several of the definitions included in the “Discussion of Fields” section have been expanded. Lastly, the list of “valid values” for SAMPLE TYPE has been updated to be consistent with values allowed by the Hazsite and EDSA applications.

The updated SRP-EDI manual was released in mid-April, 1999. The SRP appreciates suggestions for improving the manual; recommendations should be forwarded to the author. Please refer to the end of this article for phone numbers/email addresses for contacting the author.

Submittal of Data in Text (ASCII) Format

The SRP is currently accepting electronic data submissions created in the Text (ASCII, “TXT”) format, as well as “WK1” and “DBF” formats, and datasets created using the Hazsite application. Refer to the SRP-EDI Manual for directions for saving files in the text format. Responsible parties shall be aware that by the end of this year, the SRP intends to only accept dataset files submitted in the Text-Tab Delimited (“TXT”) format, and will no longer accept dataset files submitted in the WK1 or DBF formats. The dataset files can still be created in either the WK1 or DBF format, and the format of the WK1 or DBF files can be checked using EDSA, but the files need to be translated into the TXT format prior to submittal to the SRP. If needed, the author can be contacted for assistance regarding this matter.

As noted below, the Hazsite application needs to be updated to save files in the Text-Tab Delimited (“TXT”) format. Until these updates are completed, datasets created using the Hazsite application can be submitted in the DBF format.

EDSA Checker Application

The SRP has made available the EDSA data checker application, which verifies whether the files in the dataset are properly formatted, as detailed in the SRP-EDI manual. “Formatted properly” means that the table columns are in the correct order, the number of characters for a given column entry do not exceed the specified maximum length, that specific entries match between the tables, etc.

By default, datasets created using the Hazsite application are in the correct format. However, datasets created using spreadsheet or database applications may not be in the correct format, and therefore should be evaluated using the EDSA check program to ensure that they are formatted properly. Users should be aware that the version of the EDSA checker program currently available to the public is capable of evaluating datasets created in the WK1 and DBF formats, but cannot evaluate datasets created in the ASCII format.

The SRP is in the process of updating EDSA to be able to evaluate datasets created in the ASCII format. The updated version of the application will also allow users to view error reports on the screen (rather than automatically printing the reports). Also, users will be able to view the HZSAMPLE and HZRESULT files on screen, to assist in evaluating any errors that are reported. An “evaluation” version of the updated EDSA program was released in early May 1999; this version was released on a limited basis, for the purpose of testing the program and determining what programming bugs are present and removing them. A final release to the regulated community is anticipated to be June 1999.

New Mandatory Fields

The following fields are now considered mandatory, and must be completed in order for the dataset to be accepted by the data checker:

DESC (Description) field in the DTST file. The reason for this change is as follows: at this time, the fields SRPID, DIRECTORY, and SUBMITDATE are used to identify a given dataset. The SRP has determined that additional information is needed to further distinguish datasets. The DESC field is adequate for this purpose.

CONSULTANT (Consultant) field in the DTST file. The reason for this change is to assist in associating historical dataset files with recent datasets. If questions arise between recent and historical datasets, it is difficult for the SRP to know whom to contact if the CONSULTANT field is not completed.

DUPSAMP (Duplicate) field in the HZSAMPLE file. The reason for this change is to avoid situations in which duplicates are run, but the field is inadvertently omitted from the HZSAMPLE file. Without knowing that duplicate samples were analyzed, the SRP would assume that multiple samples were collected from a given sampling location. LAT/LONG and State Plane Coordinates fields in the HZSAMPLE file. Inclusion of location information for each sample, either as Latitude and Longitude OR State Plane Coordinates, is already required. However, the SRP prefers that the location information be provided in State Plane Coordinates. In general, State Plane Coordinates are more accurate than Latitude and Longitude readings. If data have already been collected in Latitude and Longitude format, programs are available for converting Latitude and Longitude to State Plane Coordinates, several of which are available through the SRP website.

CAS number field in the HZRESULT file. The reason for this change is as follows: contaminants may have more than one name (i.e., trichloroethene, trichloroethylene), but will only have one CAS number. Therefore, the EDSA program evaluates the CAS number for a result entry, and then compares the CAS number to a list of acceptable contaminant names. The SRP intends to release the list of CAS numbers and acceptable contaminant names in the near future (the release will be as a downloadable file from the SRP Electronic Data Submission-Hazsite webpage).

Hazsite Application

Changes to the Hazsite program will include the addition of new fields to better describe the collection of ground water samples. In addition, copying and saving records will be made more user-friendly, files will be saved in the Text-Tab Delimited (*.TXT) format, and various programming bugs will also be corrected. Lastly, a data “import” function may also be added to the application. Release of this updated version of Hazsite is expected to be Fall 1999. Additional information regarding this update will be available in future SRP Newsletters, as well as through the SRP internet webpage.

Future Webpage Enhancements

SRP intends to improve the functionality of the SRP EDI Webpage. This will include allowing for improved interaction between the SRP and the regulated community, including formalization of the following processes: distribution of program upgrades and tools, user manuals and other literature, a user-friendly interface for information requests, and a “bulletin board system” accessible to everyone, on which the regulated community can post questions and SRP can post answers, and the regulated community can then search for similar questions/problems and solutions.

Resources

Additional information regarding electronic data submittal, as well as downloads for the Hazsite application and the EDSA checker application, can be obtained from the SRP webpage. In addition, notifications regarding future updates to the Hazsite application and the EDSA program will be made on the Electronic Data Submission-Hazsite page of the SRP website, as well as through SRP Newsletter articles. The SRP intends to develop both a postal mailing list and an email mailing list specifically for notifying parties of updates to the Hazsite application and the EDSA checker application, as well as other aspects of electronic data submissions.

The author can be contacted either by phone (609-633-1380) or email (dhaymes@dep.state.nj.us). Please contact him if you have suggestions for improving the SRP-EDI manual, or if you wish to be added to any postal/electronic mailing lists pertaining to submitting electronic data to the SRP. If you have questions regarding use of the EDSA application, please contact the assigned case manager prior to contacting the author.

The internet address for the Electronic Data Submission-Hazsite webpage is: http://www.state.nj.us/dep/srp/regs/hazsite/hazsite.htm.

 

No Prev

Table of Contents

Next