CMAQ v 54 stops running without error in the middle of simulation

Hi,
I am running CMAQ v54 and it stops in the middle of simulation.
I’ve tried to locate the error in in CTM_LOG*, but there are no error messages.
Tried to decrease setenv CTM_MAXSYNC 720 ( default) to up to 60, but CMAQ stopped at the same hour.
Wondering what is the reason and how to fix it

Thanks a lot,

Andrei

Please review the tips to help us answer your question: PLEASE READ before posting
Did you run the benchmark tutorial and using inputs provided? CMAQ/DOCS/Users_Guide/Tutorials/CMAQ_UG_tutorial_benchmark.md at main · USEPA/CMAQ · GitHub
Do you obtain the error “CMAQv54 stops in the middle of the simulation” when running this benchmark tutorial?

If not, please specify more about your CMAQv54 simulation:
Model Domain: e.g. 12US1 or 12NE3 or please specify the grid resolution, region and size of domain.
Compiler and version: e.g. gcc or intel
Library Installation version: netCDF-4 compressed or netCDF classic
Length of model simulation: e.g. 1 day, 1 month, 1 year (difficult to tell what stops in the middle of the simulation means)
What is the size of your compute server that you are running CMAQ?
1. Do you have enough disk space for the model inputs and outputs?
2. Do you have enough memory / core?
3. Are you using a job scheduler such as slurm?

Thanks, Liz, for quick respond.

Hi Liz,
I think I’ve solved the problem.
I run CMAQ on the system with 256G RAM, with 192 threads.
Domain dimensions are 35 layers x 196 col x 169 rows.

I used all threads in configuration 16x12 and had the above issue.
I tried configuration 12x12 threads and run was successful.

Am I right, that my system does not have enough resources to run CMAQ using all threads ?
Could you pl. recommend some rules on relations between RAM, domain size and number of threads to run CMAQ ?

Thank you very much,

Andrei