Warning in subroutine Read BCs - CMAQ 5.2.1 using EDGAR

CMAQ Run Time Errors and Issues
1

Hi all,

I have been trying to run CMAQ using four nested domains in Brazil, and the multi-day simulations have been displaying a Warning message on Read BC subroutine:

>>—>> WARNING in subroutine Read BCs on PE 000
Applying fix to aerosol Boundary Conditions for aerosol modes. Turn on verbose_rdbcon preprocessor flag to learnmore.

Even with the warnings, coarser domains (d01 and d02) simulations have completed succesfully, The following domain (d03) returns a MPI Abort and forrtl: error (78): process killed (SIGTERM) message in the second day of the simulation:

Maximum eddy diffusivity of: 366.54 (m**2/sec)
at col, row, layer: 22, 12, 14
corresponding to a free tropospheric wind shear of: 1.46485E-03 (/sec),
a bulk Richardson Number of: -101.27 ,
and pot. temps. in layer and layer+1: 311.70 311.95
after VDIFF G 1.2549490E-01 A 1.1550044E+09 N 3.2895958E-05
after COUPLE_WR G 1.1501271E+03 A 1.5365843E+13 N 3.0120060E-01
>>—>> WARNING in subroutine Read BCs on PE 000
Applying fix to aerosol Boundary Conditions for aerosol modes. Turn on verbose_rdbcon preprocessor flag to learnmore.
M3WARN: DTBUF 23:02:30 Jan. 10, 2015 (2015010:230230)
after HADV G 1.1510116E+03 A 1.5298298E+13 N 3.0081290E-01

MPI_ABORT was invoked on rank 1 in communicator MPI_COMM_WORLD
with errorcode 1074146265.
NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.

forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
CCTM_v521.exe 0000000000C85A0A Unknown Unknown Unknown
libpthread-2.12.s 0000003165E0F710 Unknown Unknown Unknown
CCTM_v521.exe 00000000005A138C Unknown Unknown Unknown
CCTM_v521.exe 00000000005A48CE Unknown Unknown Unknown
CCTM_v521.exe 0000000000572513 Unknown Unknown Unknown
CCTM_v521.exe 000000000056C5CC Unknown Unknown Unknown
CCTM_v521.exe 0000000000409EDE Unknown Unknown Unknown
libc-2.12.so 000000316521ED5D __libc_start_main Unknown Unknown
CCTM_v521.exe 0000000000409DE9 Unknown Unknown Unknown

Could you help me understand what is causing that warning in the Read BCs subroutine and how can I address it? Additionally, is it reasonable to assume that this warning is the cause for the process killing?

The CTM_LOG* from the d03 domain finishes with the following error:

>>—>> WARNING in subroutine Read BCs on PE 001
Applying fix to aerosol Boundary Conditions for aerosol modes. Turn on verbose_rdbcon preprocessor flag to learnmore.
M3WARN: DTBUF 23:02:30 Jan. 10, 2015 (2015010:230230)
after HADV G 1.3393640E+03 A 1.5577415E+13 N 5.9261298E-01
*** ERROR ABORT in subroutine VPPM on PE 001
max iterations exceeded in vppm at 77
PM3EXIT: date&time specified as 0
Date&time specified as 0

I would greatly appreciate any insight into how I should treat this occurence.

Thank you!

2

The crash is almost certainly unrelated to the warning about aerosol boundary conditions.
However, are you using your coarse domain CMAQ simulations to provide boundary conditions for the nested domain? If so, then I think you should not be getting those warnings.

The error you are getting (as indicated by the log) is in the advection routine VPPM. These are difficult to debug. I would first try reducing the time step for this day. From the log file, see what synchronization time step it is using. It looks like it might be 5 minutes. You can then try specifying in the run script:
setenv CTM_MAXSYNC 150
to set the maximum synchronization time step to 150 seconds (2.5 minutes). Sometimes using a shorter time step allows the model to get past the point that is causing problems.

2 Likes
3

Hi fammaciel,

Have you solve the problem? I have the same error when using the CMAQ 5.2.

4

Hi Leonard,

Please consider reviewing the following recommendations and submit information about your model run to a new issue.

Thank you,
Liz