CCTM 5.3.2 Program received signal SIGBUS: Access to an undefined portion of a memory object

CMAQ Run Time Errors and Issues
#1

Hi, there
I have tested CMAQ 5.3.2 these days. MCIP/ICON/BCON worked well. But there was an error ( upoaded in the attached file ). when more than 25 procs were runnning CCTM( 45 procs in the log file). It could be fixed using less than 25 procs like 25 or 20. Any one have an idea about how to fix it. Thanks very much.

Errors:
Program received signal SIGBUS: Access to an undefined portion of a memory object.

Backtrace for this error:

Program received signal SIGBUS: Access to an undefined portion of a memory object.

Program received signal SIGBUS: Access to an undefined portion of a memory object.

Program received signal SIGBUS: Access to an undefined portion of a memory object.

Backtrace for this error:
#9 0x422338 in se_comm_module_MOD_se_pe_comm3s
#10 0x68DFA1 in x_ppm
#11 0x67F101 in hadv
#12 0x661672 in sciproc_
#13 0x65531D in cmaq_driver_
#14 0x654175 in MAIN__ at cmaq_main.F:?
#0 0x2B3EB9404697
#1 0x2B3EB9404CDE
#2 0x2B3EBA2D93FF
#3 0x2B3EBA3F942A
#4 0x2B3EB90CA5E4
#5 0x2B3EB90B9F8B
#6 0x2B3EB90C1807
#7 0x2B3EB902EB11
#8 0x2B3EB8D5E9D7log.txt (42.7 KB)

#2

Hi,
Thank you for reporting this issue. I am working to see if I can repeat the error on our system. First, we have a few questions:

  1. Did you compile in debug mode? I see a backtrace, but it doesn’t give detailed info such as line numbers…, so I’m not sure?
  2. Can you reproduce the error using the benchmark dataset, aka is there a domain decomposition on the benchmark set that this happens at for your system?
  3. Can you try upgrading to a more recent version of the gcc compiler?
    Thanks, Liz
#3

Thanks, Liz.
1. I didn’t change the compilation option in config_cmaq.csh. "setenv myDBG “-Wall -O0 -g -fcheck=all -ffpe-trap=invalid,zero,overflow -fbacktrace”.
2. Yes, i will have a try with the benchmark dataset.
3. Since I install CMAQ in docker mode, upgrading agcc will increase substantially the size of my docker image. It my not the best choce for me. But i will have a try. Do you have any recommended version of GCC, like gcc 7.1.0 or something else?

#4

@lizadams Hi, Recently, I have tried different compilers(Intel 2015 and gcc 9.2.0) and benchmark dataset, the error still exist.

#5

I also tried the same compilers without docker and error disappeared. It may the be the problem of docker memory limitations.