Model hanging at CTM_CONC_1 open/create step

Hello all,

I am running CMAQv5.4 using EQUATES data. Occasionally, when I’m running the model restarting from the previous day, the simulation will hang at the following step:

     =================================================
     |>---   OPEN OR CREATE CONCENTRATION FILE   ---<|
     =================================================
     CTM_CONC_1      :/scratch/gap6/CMAQ_sepNO_12US1/data/output_CCTM_v54_intel_M3DRY_EQUATES_08/CCTM_CONC_v54_intel_M3DRY_EQUATES_20170802.nc
     
     >>--->> WARNING in subroutine OPEN3
     File not available.
     Could not open CTM_CONC_1 for update - try to open new

See the attached log file for an example. Also attached is my runscript. Once it reaches this step, nothing changes for any of the processors, and it just hangs until timeout.
CTM_LOG_040.v54_intel_M3DRY_EQUATES_20170802.txt (43.2 KB)

run_cctm_MEGAN_0802-05.csh (40.8 KB)

There is no such CONC file at the specified path location, so the problem isn’t that it already exists. I have simulated data all the way from 02/2017, and this problem has been inconsistently occurring in the simulations since I started simulating June. I had the same issue in a previous simulation, but disabling a DESID rule seemed to fix it. This has not fixed the problem this time. I also tried increasing the memory from 4gb to 5gb and that didn’t work.

I have modified the MEGAN soil NOx subroutines, specifically to output NO emissions from soils separated by agricultural and non-agricultural sources using techniques similar to the B3GTS code in BEIS. I’m not sure if this is presenting a file I/O problem. In the original B3GTS code, it called “SUBST_BARRIER” in a few locations, but I couldn’t figure out how to do this in the MEGAN code (it wouldn’t compile). I don’t know what calling that does or if it’s relevant to this issue. I can share this code privately if it might be relevant.

Best,
Grant

That warning just means that the model is creating a new file instead of writing to an existing one.

The hanging behavior is probably related to the modifications you made in the MEGAN code. You can send them to me at willison.jeff@epa.gov and I’ll take a look at them tomorrow.

1 Like