Infinite loop problem in CMAQ5.3.3

CMAQ
1

Hi everyone,

We are using CMAQv5.3.3 to do the air quality forecast (every day we do a 3-day forecast since half a year ago) but suffer from the infinite loop problem sometimes.

This time it stopped at Beijing time 21:42 Aug. 24, 2022, and we thought it might be caused by the typhoon Ma-On.

We used the NVIDIA HPC SDK (Originally PGI) compiler Version 21.5 to compile all libraries and CMAQ.

I posted some information below. Could you please help to see if I set something wrong or what I could improve to avoid the infinite loop problem? Thanks a lot for your time.

CMAQ log files:

image
image480×851 182 KB

2

CMAQ compile_script and run_script setting:

image
image672×602 180 KB

3

Domain setting

image
image453×579 101 KB

(blue-WRF, red-CMAQ)

4

@yhuangci,

To get a better sense of the problem could you please attach your entire 0th processor ancillary log file and master log file.

While I have a couple of ideas, understanding the specifics of your scenario may help.

5

Please check the log files below. Thanks.

CTM_LOG_000

6

@yhuangci,

Thanks for providing the ancillary log file. It’s quite interesting that the ancillary log file did not get as far as the master log file. If you could attach any (or all if possible) other ancillary log files to see if other processors got any further that would help.

In the mean time, you could also try adjusting the CTM_MINSYNC to be 10 and CTM_MAXSYNC to be 60.

7

Thanks a lot for your suggestion. I uploaded all the relevant files at CMAQ_infinite - Google Drive

Also I would like to ask why sometimes the CMAQ got stuck in an infinite loop but this problem can be solved by restarting the model without any adjustment. Thanks.