Hello,
I am trying to set up emissions for a 4k CONUS run and I am running into this problem when running ptegu script. I’m using the SMOKE.EMF 2016_beta platform to do this, and I downloaded 4K conus/canada/mexico files from the epa ftp site.
Log file:
***testing*** SRGPRO_PATH set by SRGPRO input: /projects/b1045/SMOKE.EMF/2016_beta/ge_dat/gridding/CONUS_CANADA_MEXICO_4K_beta/
Linux2_x86_64ifort
NOTE: Default MONTH setting to 'jan'
/bin/ls: No match.
/bin/ls: No match.
/bin/ls: No match.
/bin/ls: No match.
SCRIPT NOTE: Setting months with no partial months
SCRIPT NOTE: Scanning PTREF for duplicate records
SCRIPT NOTE: No duplicates found in /projects/b1045/SMOKE.EMF/2016_beta/ge_dat/temporal/noncem_2016beta_tref_winter_30nov2018_v0
***testing*** SRGPRO_PATH set by SRGPRO input: /projects/b1045/SMOKE.EMF/2016_beta/ge_dat/gridding/CONUS_CANADA_MEXICO_4K_beta/
Linux2_x86_64ifort
***testing*** SRGPRO_PATH set by SRGPRO input: /projects/b1045/SMOKE.EMF/2016_beta/ge_dat/gridding/CONUS_CANADA_MEXICO_4K_beta/
Linux2_x86_64ifort
Not running daily inventory for this sector (DAY_SPECIFIC_YN not set to Y).
Creating dataset /projects/b1045/SMOKE.EMF/2016_beta/2016ff_16j/inputs/ptegu/pthour_ptegu_dec_2016ff_16j.lst
using script combine_data.csh
Processing environment variables EMISHOUR_MULTI_A
nameprefix = /projects/b1045/SMOKE.EMF/2016_beta/2016emissions/2016ff_16j/inputs/cem/2016/HOUR_UNIT
month = 12
monname = dec
/bin/ls: No match.
SCRIPT ERROR: Some file patterns not matched for style Hourly
According to MULTIFILE_NAMEBREAK, the file prefixes are:
/projects/b1045/SMOKE.EMF/2016_beta/2016emissions/2016ff_16j/inputs/cem/2016/HOUR_UNIT
Check file names and rerun
ERROR: Could not run combine_data.csh for hourly list file in Annual_area
The multifile_namebreak follows the correct convention as I’m using 2016 emissions which were already packaged with the 2016_beta system.
I also notice that within the logfile, it says it’s creating the /projects/b1045/SMOKE.EMF/2016_beta/2016ff_16j/inputs/ptegu/pthour_ptegu_dec_2016ff_16j.lst file, which does not exist, so I’m not sure if it’s being created and deleted when the script fails, or if this is my error. I can’t identify where in the script it is supposed to be creating this file.
I’ve also adjusted the start/end date and run months to see if this error is addressed in january, but I get the same error out.
If anyone has any insight to this, it would be much appreciated, thank you!
Anastasia
Hi, we believe the issue is that SMOKE is not finding the CEMS data in the location it is looking.
Please review where the contents of the CEM package (HOUR_UNIT files) were unzipped.
If the hourly CEMs were downloaded from our FTP package for beta platform (2016ff_inventory_cem_17dec2018.zip), we expect that location to be:
/projects/b1045/SMOKE.EMF/2016_beta/2016ff_16j/inputs/cem/
It looks like the script is expecting the HOUR_UNIT files to be located in:
/projects/b1045/SMOKE.EMF/2016_beta/2016emissions/2016ff_16j/inputs/cem/2016/
This is based on the EMISHOUR_MULTI_A parameter in the run script.
First, confirm that EMISHOUR_MULTI_A is pointing to the correct directory for the hourly CEMs
If the EMISHOUR_MULTI_A location is correct, are you using hour CEMs from our FTP package, or from somewhere else?
We using the ones we have posted with the platform.
If the EMISHOUR_MULTI_A location is correct and you are using our hourly CEMs, then we’ll need to follow up with some more information.
Also, note that we’ve never run a 4km on the full CONUS – only on sub-parts of the CONUS for specific regions, so you may encounter issues in terms of memory requirements if you are trying to run the entire CONUS at 4km, but that is not the specific problem here.
Hello,
Yes I’ve downloaded these files from the FTP package for beta platform and created the 2016 directory because I have other directory years in this CEM directory as well. So EMISHOUR_MULTI_A is pointing to the correct directory, and I am using the CEMs from the beta platform.
And yes, I have run into memory issues with a 4 km CONUS run, but I have managed to solve these memory issues by using an updated netcdf4 package because netcdf3 has a memory limit of 4 gb. So far I’ve only managed to run WRF at 4 km on our univeristy HPC cluster and this fix has worked, but I imagine I will continue to run into memory issues in the future.
We noticed that your stdout references December. Are you running with the spinnup duration > 0?
If so, perhaps it is the December 2015 CEMS data that are not being found?
Yes, there is a 2015-12 CEMS file in the 2016ff_inventory_cem_17dec2018.zip, and I’m doing 10 days spin up in December 2015 for a January 2016 - onwards run.
We think that you need to set the spinup_duration parameter to 0 because it’s not finding CEMS data for the first half of December.
To cover your spinup period emissions, you can plan to use emissions from the latter part of December, 2016.
Does it run if you set spinup_duration to 0?
It does not, sorry, I actually have already set spinup_duration = 0 this whole time.
I have this defined:
setenv RUN_MONTHS "12"
setenv SPINUP_DURATION "0"
setenv SPINUP_MONTH_END "Y"
## Emissions modeling year
# (i.e. meteorological year, not necessarily the inventory year"
setenv BASE_YEAR "2015"
setenv EPI_STDATE_TIME "${BASE_YEAR}-12-23 00:00:00.0"
setenv EPI_ENDATE_TIME "${BASE_YEAR}-12-31 23:59:00.0"
I have also tried to download 2015 CEM data from the Air Markets program in hopes to address the december issue and this has not addressed the problem.
Try this:
Set BASE_YEAR to 2016. Note that the EPI_STDATE_TIME and EPI_ENDATE_TIME settings aren’t designed to be used for partial months and should be left alone:
setenv EPI_STDATE_TIME “${BASE_YEAR}-01-01 00:00:00.0”
setenv EPI_ENDATE_TIME “${BASE_YEAR}-12-31 23:59:00.0”
If you only want to run the last 10 days of December, the easiest thing to do is to set RUN_MONTHS = 12 (as you have already done) and then just let SMOKE create emissions for the full month of December. (The run settings file is the way to have SMOKE skip the days she doesn’t want, but I don’t think it’s worth getting into that…)
Our scripts aren’t meant to run for partial months.
I suggest also seeing if you can get January to run first before going back to December.
Thank you for all your patience with this – January 2016 does run further than the December run, however there is an issue in /projects/b1045/SMOKE.EMF/2016_beta/2016ff_16j/intermed/ptegu/logs/smkinven_ptegu_jan_2016ff_16j.log where
WARNING in subroutine CHKFIL3 <<<
Inconsistent file attribute NROWS for file PHOUR
Value from file: 5642
Value from caller: 5897
Could not open file “PHOUR”.
*** ERROR ABORT in subroutine OPENPDOUT
Could not open file “PHOUR”.
Which ultimately makes the January run fail.
That error usually means there is a leftover PHOUR file from a previous run. Deleting the PHOUR file from the intermed/ptegu directory will hopefully fix that.
1 Like
Thank you that was the problem in the January 2016 run! So January 2016 does work – oddly, December 2015 doesn’t work. Perhaps something is wrong with the hourly CEM in December 2015, when I redownload from the EPA the file sizes are larger than the files from the 2016_beta platform though the same error is thrown.
You can’t run December 2015 with our package – you need to run December 2016, and you can reuse the same files for the spinup in Dec 2015.
Oh okay great – thank you!