I started with the NEI2019 data and the assigns script as I provided and got the HVAL error message. I then made two smaller input data sets (the first 13 or so lines of emissions data for both the annual and monthly) and tried this but got the header error. I deleted the extraneous “,”s and was able recreate the HVAL error. I then remade the input files more carefully so the emissions were only for Washington State and made sure the year was set to 1980 rather than 2019, in all instances in the emissions input. The program now seemed to be able to get past the HVAL error but now produced the CEMS error.
The Jan input file is about 30 MB so I am not able to upload it.
We are not sure why SMOKE is looking for the CEMSUM file in this situation and would like to reproduce this issue. Could you try subsetting your annual and hourly inventories to include only 1 county and see if it still asks for the file? If so, those files should be small enough to upload to the forum so that we can test the case here.
I have created small annual c1c2 file and monthly files for jan and feb that contain emissions data for one county. These are attached. When I try to run these the program is now stopping when trying to write the inv2 report. I have attached this log file (renamed as a text file) showing the error as well. Sigh!
In regards to the CEMS problem I have also attached the pthour…lst file (also changed to a text file). The header indicates it is CEMS data so this is where the program is being told to expect CEMS data.
We note that your run script appears to be from our 2017gb package. We recommend that you use one of our newer platforms, such as the 2019ge package at https://gaftp.epa.gov/Air/emismod/2019/smoke_2019_platform_core_17mar2023.zip. Since you are using 2019-like CMV inventories (as suggested by the filenames) you may be better off using the 2019ge package as the starting point. For instance, this Smkreport error is basically because the REPCONFIG configuration in the 2017gb CMV scripts isn’t compatible with the 2019ge CMV inventories. It’s possible switching to the 2019ge platform could fix other issues also.
Re: “The header indicates it is CEMS data so this is where the program is being told to expect CEMS data.” – this is not correct from our experience. Our PTHOUR .lst files for CMV say “#LIST CEM” at the top (that’s just what our PTHOUR .lst file builder puts there), but despite that, Smkinven still does not look for a CEMSUM file in our CMV runs. So something else is triggering that CEMSUM issue.
I found the issue with CEMSUM file that you are having caused by some garbage characters in the header line of the *hourly.csv files where file format is indicated. If you do
cat -v cmv_C1C2_01_cmv_c1c2_2019_12US1_2019_US_hourly.csv | less
then you would see something like this:
M-oM-;M-?#FORMAT=FF10_HOURLY_POINT^M
These are mix of garbage characters and carriage return due to wrong encoding. I assumed these garbage characters came from processing larger csv files to these smaller csv files.
With these garbage characters, smkinven would failed to identify the format of the *hourly.csv file (FF10) and resorted to the format identified in the header of the list file (#LIST CEM) generated by helper scripts. As smkinven incorrectly identified these *hourly.csv inputs files as CEM data format, it then triggered the CEMSUM file requirement.
You can fix the encoding issue this by using something like dos2unix command:
Many thanks for the heads up about this issue. I have been looking the emission files using OpenOffice and I didn’t notice the problem with trailing “,” on the header but the issue you pointed out would be hard to detect. I will keep working on preparing a suitable inventory set to test for the HVAL problem.