Could not open CTM_CONC_1 error

CMAQ
1

I am running a two-way coupled WRF-CMAQ (WRFv4.5.1, CMAQv5.5) simulation. The model completes a 10-day spin-up in December successfully but fails at the beginning of January 1. I suspect an input or restart alignment issue but have not identified the cause. The simulation fails after ~60 seconds.

I have verified that the following input files exist, align with the simulation dates/timing, and do not include erroneous nans: wrfbdy, wrffdda, wrfsdda, wrfinput, wrflowinp, ndown. I’ve also looked at all emission inputs (inln, stackgroups, emis_mole, merged_dates) for the new year, verifying the files are correctly linked, reflect the new year, and do not include erroneous values.

Due to permission restrictions, I have upload the rsl.out file, rsl.error file, module lists, and my submission scripts to my Github linked here. I have included my grep -i error rsl.out.0000 output below, however, I will note that the grep errors occur during successful runs and am not convinced they are the root of the problem here.

If anyone has suggestions on additional troubleshooting steps, I would appreciate any guidance you can offer.

-VL

rsl.out.0000 tail

 CONCENTRATION FILE   ---<|
=================================================

CTM_AELMO_1 :
/projects/b1045/rcs_support/wrf-cmaq/WRF_DIR/work_dir/
WRFv451_CMAQv55/CCTM/scripts/WRFCMAQ-BASE_202301_D02-sw_feedback/
CCTM_AELMO_4.5.155_BASE_202301_D02_20230101.nc

>>--->> WARNING in subroutine OPEN3
File not available.

Value for IOAPI_CHECK_HEADERS:  N returning FALSE
Value for IOAPI_OFFSET_64:  YES returning TRUE

"CTM_AELMO_1" opened as NEW(READ-WRITE )

Starting date and time  2023001:000000
Timestep 010000

CTM_CONC_1 :
/projects/b1045/rcs_support/wrf-cmaq/WRF_DIR/work_dir/
WRFv451_CMAQv55/CCTM/scripts/WRFCMAQ-BASE_202301_D02-sw_feedback/
CCTM_CONC_4.5.155_BASE_202301_D02_20230101.nc

>>--->> WARNING in subroutine OPEN3
File not available.

Could not open CTM_CONC_1 for update - try to open new

rsl.error.0000 tail

d01 2023-01-01_00:01:40 --> TOP OF DIAGNOSTICS PACKAGE
d01 2023-01-01_00:01:40 --> CALL DIAGNOSTICS PACKAGE: ACCUMULATED AND BUCKET DIAGNOSTICS
d01 2023-01-01_00:01:40  call HALO_RK_E
d01 2023-01-01_00:01:40 calling inc/HALO_EM_E_5_inline.inc
d01 2023-01-01_00:01:40  call HALO_RK_MOIST
d01 2023-01-01_00:01:40 calling inc/HALO_EM_MOIST_E_5_inline.inc
d01 2023-01-01_00:01:40  call HALO_RK_SCALAR
d01 2023-01-01_00:01:40 calling inc/HALO_EM_SCALAR_E_5_inline.inc
d01 2023-01-01_00:01:40  call end of solve_em

grep -i error rsl.out.0000

d01 2023-01-01_00:00:00  NetCDF error: NetCDF: Attribute not found
d01 2023-01-01_00:00:00  NetCDF error in ext_ncd_get_dom_ti.code REAL, line          83  Element GMT
d01 2023-01-01_00:00:00  NetCDF error: NetCDF: Attribute not found
d01 2023-01-01_00:00:00  NetCDF error in ext_ncd_get_dom_ti.code INTEGER, line          83  Element JULYR
d01 2023-01-01_00:00:00  NetCDF error: NetCDF: Attribute not found
d01 2023-01-01_00:00:00  NetCDF error in ext_ncd_get_dom_ti.code INTEGER, line          83  Element JULDAY
Binary file rsl.out.0000 matches
2

I am unsure, but noticed warnings about FDDA files not being of the correct date.

**WARNING** Time in input file not equal to time on domain **WARNING**
 **WARNING** Trying next time in file wrfsfdda_d02 ...

I wonder if this environment variable cont_from_spinup_run should be set to true instead of false in your run script.

Change:

set cont_from_spinup_run =        F   # indicates whether a wrf spinup run prior to the twoway model run

to:

set cont_from_spinup_run =        T   # indicates whether a wrf spinup run prior to the twoway model run
3

Hi Vlang,

I couldn’t find the mistake in your files. In my own work, I run simulations from December to January without issue—by keeping them continuous.. Try running one continuous simulation from December through January without stopping. Since part1.csh runs successfully, you can simply extend it to cover the full period.

Xu