I have been customizing the CMAQ v5.3 run script to run for SOAS 2013 case and i see there are some differences in emissions (more Sector-wise classifications in v5.3 compared to v5.2.1. inputs we have) and meteorological inputs, since we are using the v5.2.1 inputs from the SOAS 2013 Google drive.
Just wanted to double check that we are not missing anything by trying to run CMAQ v5.3 with these SOAS 2013 inputs configured with v5.2.1.
Is there SOAS 2013 Input data available configured for CMAQ v5.3 run?
I have condensed the differences i noticed with questions into this table where highlighted inputs are those that v5.3 has that v.5.2.1 does not:
Hi Quazi,
Thank you for your questions regarding the differences between CMAQv5.3 and CMAQv5.2.1.
The answer is yes for all of them, except for the question about whether the LUFRAC_CRO variable exists in the v5.2.1 inputs. That variable is not available with the CMAQv5.2.1 release.
The LUFRAC_CRO variable is a new variable with the new MCIPv5.0 release, which is distributed as part of the CMAQv5.3 release.
I did tried to run v5.3 with v5.2 SOAS 2013 inputs, i am getting stuck at this point:
*** ERROR ABORT in subroutine depv_data_setup on PE 615
Could not read “L1_SW” from file “E2C_CHEM”
PM3EXIT: date&time specified as 0
Date&time specified as 0
I think this has to do with newer FEST-C v1.4 having Soil moisture inputs (L1_SW and L2_SW) , which is not there in older FEST-C v1.1 or 1.2 which was used in v5.2 SOAS 2013 inputs, what can i substitute for that in centralized_io_module.F (From where CCTM is calling all the EPIC/FEST-C variables). Also, FEST-C L1_SW and L2_SW is in mm where as , SOIM1 and SOIM2 in METCRO2D are in m3/m3
I tried to re-run for SOAS 2013 v5.2 inputs with CMAQ v5.3 , by switching OFF (N) the CTM_ABFLUX (bi-directional EPIC NH3) option as it was causing issues (as stated in my previous Post in this thread), but still i see it is able to run only till 5 minutes time step & i don’t see any specific issues with any inputs. Any idea why it is so (I am attaching the CTM_LOG file)CTM_LOG_615_record.v53_intel17.2_soas_2016_cb6_20130521.txt (135.3 KB) ?
tail CTM_LOG_615.v53_intel17.2_soas_2016_cb6_20130521
Processing completed... 2.9 seconds
Processing Day/Time [YYYYDDD:HHMMSS]: 2013141:000500
Which is Equivalent to (UTC): 0:05:00 Tuesday, May 21, 2013
Time-Step Length (HHMMSS): 000500
Temporal BEIS at time 0:07:30
After VDIFF : Gas 2.231E-01 | Aer 3.634E+09 | Non 1.735E-06
After COUPLE : Gas 2.109E+03 | Aer 6.230E+13 | Non 1.621E-02Quazi, there is no error in the log file you uploaded (for processor 615). Is there anything in any of the other log files (try grepping for ABORT) or the overall log file? If none of those have any information then I’d try recompiling in debug mode (your problem occurs almost instantly, so performance is not an issue), and then if that doesn’t illuminate things, then try running under a debugger.
I didn’t find any ABORT step in the overall log file (i have attached it as well). I thought i did compiled my CCTM executable with DEBUG mode as TRUE (Makefile is attached, correct me if i am wrong and need to add anything for compiling CCTM with debug mode ON?).
I was able to run the Benchmark case successfully before so not been able to pin point what’s causing this termination.Makefile.txt (29.4 KB) Makefile.txt (29.4 KB) Makefile.txt (29.4 KB) slurm_soas2013_ros315isop_log.txt (320.8 KB)
From your slurm log file, you have a lot of processes (616) running on what looks to me like 14 nodes. How many processors per node are there on your system? Also, there are lots of warnings about stack size unlimited causing performance problems. Maybe try increasing the number of nodes and/or decreasing the number of processes.
I am using 44 procs-per-node with 14 nodes, will try that.
I am also re-compiling by setting the Debug_CCTM option in bldit_cctm.csh (which i think was commented earlier)
After running on DEBUG mode, i get this ABORT message and the terminates right at the start:
==================================
|>— CONFIGURE SCENARIO —<|
==================================
*** ERROR ABORT in subroutine SA_IOLIST on PE 615
Error Opening SA_IO_LIST file
PM3EXIT: date&time specified as 0
Date&time specified as 0
Have you tried running again using the optimized compiled version? I have had trouble running in debug mode on dogwood.
Sorry i misunderstood your question earlier, I did try running with executable compiled with DEBUG=TRUE(optimized flag) on Dogwood, that is what you were asking right?
I also am not able to run it, i get the following error message in the slurm.out file (also lot of core. files as well, CTM_LOG files are not created):*
Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that process rank 60 with PID 28213 on node c-208-24 exited on signal 8 (Floating point exception).
And the same script runs successfully with CCTM executable created without DEBUG flags compilation
Hi @lizadams, I was wondering how one can run the ‘optimized compiled version’ of CMAQv5.3.2 if the debug mode does not run?