My test CMAQv5.3.1 simulation of 36km CONUS domain stopped in 30 minutes with this error:
Could you post your run script and the main log file?
Hi @cgnolte, thanks for showing concern, and sorry for the late reply. I was on a different machine and did not have the old machine’s error I mentioned in this topic, but even on the new machine, the same error persisted. It turns out the issue was due to a bug when using representative dates for emissions, and I just used bug-fix at github to recompile CMAQ. That problem is now gone, but a new problem probably unrelated to the one in this thread came up - I have started a new thread for that new issue at Longjmp causes uninitialized stack frame.
I managed to get other errors away with LTNG and WB_DUST set to off, but the error Could not extract GR_EMIS_001 persisted (Run starts at 20140102:030000 and goes smoothly till the end of 20140102 and then exits when simulation for 20140103 starts), so I am revisiting the thread! The standard output shows this error towards this end:
The CTM_LOG_015* file has this:
I have checked my emission file for that day, and it exists and has the required timestamp.
slurm-5749119_4294967294.out.txt (244.1 KB) run_cctm_2016_12US1_stamp2_12US2_testrun_myfix_22may2020.csh (46.2 KB) CTM_LOG_015.v531_intel_2016_CONUS_20140102.txt (746.6 KB)
You are starting the run at 2014002:030000, and attempting to run for 24 hours.
Your GR_EMIS_001 emissions file begins at 2014002:000000, and contains 25 1-h time steps:
"GR_EMIS_001" opened as OLD:READ-ONLY
File name "/home1/05774/tg850734/scratch/cmaq_run_18may2020/cmaq_in/emis/2d_emis/ag/m3tshift_to_20140102_on_18may2020_m3wndw_to_12US2_on_18may2020_emis_mole_ag_20160102_12US1_cmaq_cb6_2016fh_16j.ncf"
File type GRDDED3
Execution ID "????????????????"
Grid name "12US2"
Dimensions: 246 rows, 396 cols, 1 lays, 29 vbles
NetCDF ID: 524288 opened as READONLY
Starting date and time 2014002:000000 (0:00:00 Jan. 2, 2014)
Timestep 010000 (1:00:00 hh:mm:ss)
Maximum current record number 25
Thus, the last time step is at 2014003:000000, and when the model looks for 2014003:010000, that data cannot be found, and the model aborts with an appropriate error message “Time step not available.”
Where you say “I have checked the emission file for that day, and it exists and has the required timestamp”, the log file indicates that is not the case. How did you check the time stamps? One way to do so is ncdump -vTFLAG <filename>.
Thanks a lot - I had been thinking that STTIME was only for the first day of the simulation, so I started from 2014002:030000 hours and set TSTEPS to 240000 hoping that from the second day onwards, the simulation would start at 2014003:000000 and end at 2014003:230000. My chemical BC files start at 030000 hrs and not at 000000 hrs, so if I set STTIME to 030000, START_DATE to 2014002 and TSTEPS to 210000, would CMAQ still run in 21 hours chunks for all days of the simulation period with this setting?
It might be helpful if you stated as clearly as you can what it is you are trying to do, rather than simply what error you are encountering. If you already said this in a previous thread, then I may have missed it or forgotten.
What is the overall time period you want to model? One day, or multiple days? You can run in 21-h run segments, followed by a 3-h run segment, and then repeat as many times as you like. I’m not clear why you would want to do so.
Your gridded emissions files start at 0Z and contain 25 time steps each. Do you have separate emissions data files for each day?
Your met files start at 040000 on 1 Jan and contain data for 741 hourly time steps, so presumably you want to run for about a month.
Your BNDY_CONC file starts at 030000 on 2 Jan and contains data for 5 6-h time steps.
If you have multiple BNDY_CONC files, then why not put them together into a single file, analogous to what you have done for the met data? You can use m3xtract for this, appending to the existing file as you go, until you have a month-long file.
Then you can start at 0Z on Jan 3, and run for 24-h periods. If you really want to start at 030000 on Jan 2, then first do a 21-h run segment, and pick up the run from there.
Thanks for the good suggestions - I was going for long simulations like you said, but hadn’t thought of these tricks.
No one seems to be using the file-set lists capabilities (requires I/O API 3.2; there were bugs in 3.1’s implementation of file-set lists). For example:
setenv GR_EMIS_001 "E1,E2,E3,E4,E5
setenv E1 <path for first-day emissions file>
setenv E2 <path for second-day emissions file>
setenv E3 <path for third-day emissions file>
setenv E4 <path for fourth-day emissions file>
setenv E5 <path for fifth-day emissions file>
...
[and similarly for other time stepped input files]
time CCTM.exe !! five-day continuous run
set foo = ${status}
if ( ${foo} != 0 ) then
echo "*** ERROR ${foo} in CCTM.exe "
exit ( ${foo} )
endif
[Note that the script checks the error-status for CCTM.exe and exits (with a message and passing that status along as the status of the script) in case of failure…]
Hi, I also encountered this problem. I have double checked the Strattime of GR_EMIS_001, INIT_CONC_1, MET_BDY_3D, and BNDY_CONC_1. They all keep start from 2019243 00:00:00. I don’t how to solve this.
"GR_EMIS_001" opened as OLD:READ-ONLY
File name "/public/home/hujun/programs/CMAQ_Project/data/2019FJ/emis/d01/emis.apb.CB05.PRD.d01.2019243.ncf"
File type GRDDED3
Execution ID "????????????????"
Grid name "LAM_28N114E"
Dimensions: 163 rows, 157 cols, 34 lays, 32 vbles
NetCDF ID: 458752 opened as READONLY
Starting date and time 2019243:000000 (0:00:00 Aug. 31, 2019)
Timestep 010000 (1:00:00 hh:mm:ss)
Maximum current record number 25
"INIT_CONC_1" opened as OLD:READ-ONLY
File name "/public/home/hujun/programs/CMAQ_Project/data/2019FJ/icbc/ICON_v54_20190831_profile_20190831"
File type GRDDED3
Execution ID "ICON_v54.exe"
Grid name "2019FJ_d01"
Dimensions: 163 rows, 157 cols, 47 lays, 248 vbles
NetCDF ID: 524288 opened as READONLY
Time-independent data.
Number of Emissions Layers: 34
out of total Number of Model Layers: 47
"MET_BDY_3D" opened as OLD:READ-ONLY
File name "/public/home/hujun/programs/CMAQ_Project/data/2019FJ/met/mcip/d01/METBDY3D_190831.nc"
File type BNDARY3
Execution ID "mcip"
Grid name "2019FJ_d01_CROSS"
Dimensions: 163 rows, 157 cols, 47 lays, 16 vbles, 1 cells thick
NetCDF ID: 589824 opened as READONLY
Starting date and time 2019243:000000 (0:00:00 Aug. 31, 2019)
Timestep 010000 (1:00:00 hh:mm:ss)
Maximum current record number 25
"BNDY_CONC_1" opened as OLD:READ-ONLY
File name "/public/home/hujun/programs/CMAQ_Project/data/2019FJ/icbc/BCON_v54_20190831_profile_20190831"
File type BNDARY3
Execution ID "BCON_v54.exe"
Grid name "2019FJ_d01"
Dimensions: 163 rows, 157 cols, 47 lays, 248 vbles, 1 cells thick
NetCDF ID: 655360 opened as READONLY
Time-independent data.
Number of Total Point Sources on this sub-domain equals 0
Note: Large numbers of point sources (e.g. > 100,000 on any sub-domain
processor) will cause noticeable runtime penalties. Users are advised to
limit the maximum number of point sources on any sub-domain to below this
threshold.
CTM_OCEAN_CHEM set to FALSE. Open ocean and surf zone
fractions will be set to 0. There will be no oceanic
halogen-mediated loss of ozone, dms chemistry, or sea spray aerosol emissions.
>>--->> WARNING in subroutine XTRACT3
Requested variable not available in file GR_EMIS_001which contains variables:
M3WARN: DTBUF 0:00:00 Aug. 31, 2019 (2019243:000000)
ALD2
ALDX
CO
CO2
ETH
ETHA
ETOH
FORM
HONO
IOLE
ISOP
MEOH
NH3
NO
NO2
NOX
NVOL
OLE
PAR
PEC
PM2_5
PMC
PMFINE
PNO3
POC
PSO4
SO2
TERP
TOL
UNR
VOC
XYL
*** ERROR ABORT in subroutine retrieve_time_de on PE 005
Could not extract GR_EMIS_001 file
PM3EXIT: DTBUF 0:00:00 Aug. 31, 2019
Date and time 0:00:00 Aug. 31, 2019 (2019243:000000)
Hello Jason,
could you please share which model version and chemical mechanism you are using?
Looking at this message
Requested variable not available in file GR_EMIS_001
the error likely isn’t caused by any issues with time steps in your input files (which appear to be fine as you noted), but rather with the contents of your emission file, possibly pointing to an inconsistency between what you provide in terms of primary PM species and what is required by the mechanism.
It seems that in addition to PSO4, PNO3, PEC, POC and PM2_5, the file contains a bulk unspeciated “PMFINE” species (rather than PFE, PAL, POTHR, etc.) which might not be the level of specificity required by the CMAQ version and mechanism / aerosol module you are using. However, I’m not sure if this is what’s causing the problem, @Ben_Murphy would have a better idea once you specify which model version and mechanism you are using.
Hi Hogrefe,
I use CMAQv5.4 and choose cb6r5_ae7_aq chemical mechanism. The emission file was used for running CMAQv5.0.2 with cb05tucl_ae5_aq chemical mechanism. Do these old emission species match with new version CMAQ or new chemical mechanism? If so, how could I use these old emission species. Besides, I didn’t find the cb05tucl_ae5_aq chemical mechanism in the CMAQv5.4.
Hi Jason,
At least one issue you are facing is the mapping of the older AE5 speciation to the new AE7 mechanism you are using. To address this, I recommend removing the following lines in the file CMAQ_Control_DESID_cb6r5_ae7_aq.nml:
‘EVERYWHERE’, ‘ALL’ ,‘SULF’ ,‘SULF’ ,‘GAS’ ,0. ,‘UNIT’,‘a’,
‘EVERYWHERE’, ‘ALL’ ,‘DMS’ ,‘DMS’ ,‘GAS’ ,1. ,‘UNIT’,‘a’,
‘EVERYWHERE’, ‘ALL’ ,‘SULF’ ,‘ASO4’ ,‘FINE’ ,1. ,‘MASS’,‘a’,
‘EVERYWHERE’, ‘ALL’ ,‘PNH4’ ,‘ANH4’ ,‘FINE’ ,1. ,‘UNIT’,‘a’,
‘EVERYWHERE’, ‘ALL’ ,‘PCL’ ,‘ACL’ ,‘FINE’ ,1. ,‘UNIT’,‘a’,
‘EVERYWHERE’, ‘ALL’ ,‘PNA’ ,‘ANA’ ,‘FINE’ ,1. ,‘UNIT’,‘a’,
‘EVERYWHERE’, ‘ALL’ ,‘PMOTHR’ ,‘AOTHR’ ,‘FINE’ ,1. ,‘UNIT’,‘a’,
‘EVERYWHERE’, ‘ALL’ ,‘PFE’ ,‘AFE’ ,‘FINE’ ,1. ,‘UNIT’,‘a’,
‘EVERYWHERE’, ‘ALL’ ,‘PAL’ ,‘AAL’ ,‘FINE’ ,1. ,‘UNIT’,‘a’,
‘EVERYWHERE’, ‘ALL’ ,‘PSI’ ,‘ASI’ ,‘FINE’ ,1. ,‘UNIT’,‘a’,
‘EVERYWHERE’, ‘ALL’ ,‘PTI’ ,‘ATI’ ,‘FINE’ ,1. ,‘UNIT’,‘a’,
‘EVERYWHERE’, ‘ALL’ ,‘PCA’ ,‘ACA’ ,‘FINE’ ,1. ,‘UNIT’,‘a’,
‘EVERYWHERE’, ‘ALL’ ,‘PMG’ ,‘AMG’ ,‘FINE’ ,1. ,‘UNIT’,‘a’,
‘EVERYWHERE’, ‘ALL’ ,‘PK’ ,‘AK’ ,‘FINE’ ,1. ,‘UNIT’,‘a’,
‘EVERYWHERE’, ‘ALL’ ,‘PMN’ ,‘AMN’ ,‘FINE’ ,1. ,‘UNIT’,‘a’,
‘EVERYWHERE’, ‘ALL’ ,‘PH2O’ ,‘AH2O’ ,‘FINE’ ,1. ,‘UNIT’,‘a’,
You may opt to replace those lines with alternatives that scale emissions for those species to the PMFINE variable you have on the file. For example,
‘EVERYWHERE’, ‘ALL’ ,‘PMFINE’ ,‘AFE’ ,‘FINE’ ,X.Y ,‘UNIT’,‘a’,
where X.Y is the real number scale factor you would like to apply for particulate iron (AFE) in this case. You can check Reff et al. (2009) https://pubs.acs.org/doi/full/10.1021/es802930x for some representative source-resolved scale factors for metal species that are often applied in the SPECIATE database. I’m not sure how these would apply to your total merged emissions though.
For organics (POC and PNCOM), I recommend modifying all of the lines with APNCOM from, for example:
‘EVERYWHERE’, ‘ALL’ ,‘PNCOM’ ,‘VSVPO1’ ,‘GAS’ ,0.045,‘MASS’,‘a’,
to:
‘EVERYWHERE’, ‘ALL’ ,‘POC’ ,‘VSVPO1’ ,‘GAS’ ,0.018,‘MASS’,‘a’,
Here, I have changed the emission file variable to POC (which exists) and multiplied the scale factor by 0.4, which is the default assumption employed by Simon and Bhave (2012) https://pubs.acs.org/doi/full/10.1021/es202361w for ‘other’ sources. If you have access to specific vehicle or biomass burning emissions inputs, then you can scale them with 0.25 and 0.7 respectively. This can be done by using the ‘source stream’ field, which is currently set to ‘ALL’. For example, if your vehicle emission stream is named ‘VEHICLES’ and biomass burning emissions are named ‘BIOMASS’, then the following would work:
‘EVERYWHERE’, ‘ALL’ ,‘POC’ ,‘VSVPO1’ ,‘GAS’ ,0.018,‘MASS’,‘a’,
‘EVERYWHERE’, ‘VEHICLES’ ,‘POC’ ,‘VSVPO1’ ,‘GAS’ ,0.01125,‘MASS’,‘o’,
‘EVERYWHERE’, ‘BIOMASS’ ,‘POC’ ,‘VSVPO1’ ,‘GAS’ ,0.0315,‘MASS’,‘o’,
The final ‘o’ indicates that the original scale factor should be overwritten for the emissions from the stream identified on that line. If you have several files that make up the vehicle or biomass burning emissions, you can combine them using the Stream Family feature in the CMAQ_Control_DESID.nml file.
Finally, you may also have issues with gas emissions. If that’s the case, we may be able to help further with mapping those species.
Good luck!
Ben
Thanks a lot. It’s pretty complicate to map of the older AE5 speciation to the new AE7 mechanism. Is it possible that I use cb05tucl_ae5_aq chemical mechanism with CMAQv5.4?
Hi Jason,
Yes, it is possible to bring in a mechanism consistent with cb05tucl_ae5 and run it in v5.4. You will need to have:
Then follow the instructions here: CMAQ/CMAQ_UG_tutorial_chemicalmechanism.md at main · USEPA/CMAQ · GitHub
Cheers,
Ben