CMAQ_5.3.1 Benchmark ISAM

Hello,
I am running CMAQ_5.3.1 benchmark case, the base case runs successful but ISAM, it runs for sometime but terminates with these messages on the screen. I am also attaching log file if that helps. Thanks for the support.
CTM_LOG_000.v531_gcc_Bench_2016_12SE1_20160701.txt (224.6 KB)

===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= PID 330 RUNNING AT prdcmaqmstv01
= EXIT CODE: 9
= CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES

YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Killed (signal 9)
This typically refers to a problem with your application.
Please see the FAQ page for debugging suggestions
Command exited with non-zero status 9
real 85.39
user 72.85
sys 3.14


** Runscript Detected an Error: CGRID file was not written. **
** This indicates that CMAQ was interrupted or an issue **
** exists with writing output. The runscript will now **
** abort rather than proceeding to subsequent days. **


  1. I can’t tell how many processors this was run on. If run on more than 1 pe, there will be multiple CTM_LOG files.
  2. To check to see if there is an error message in one of the other CTM_LOG* files, please use the following command, after changing directories to the location of the log files.

grep -i error CTM_LOG*

This case was run with single processor

I would try it using at least 4 processors
in your run script set:

 set PROC      = mpi               #> serial or mpi

and

   @ NPCOL  =  2; @ NPROW =  2

Also verify that you have the following command in your .cshrc

limit stacksize unlimited

Thanks @lizadams . I ran with mpi and also added ‘limit stacksize unlimited’ in csh.cshrc file (i assume this is what you suggested). It stops at the same point it seems. This time I am attaching all the log files. I ran the command grep -i error CTM_LOG*, it return nothing. Thanks a lot.

CTM_LOG_003.v531_gcc_Bench_2016_12SE1_20160701.txt (205.0 KB) CTM_LOG_000.v531_gcc_Bench_2016_12SE1_20160701.txt (224.6 KB) CTM_LOG_001.v531_gcc_Bench_2016_12SE1_20160701.txt (205.0 KB) CTM_LOG_002.v531_gcc_Bench_2016_12SE1_20160701.txt (205.0 KB)