MPI and CCTM error

CMAQ Run Time Errors and Issues
1

Dear CMAQ forum

I am running the CMAQ simulation test with openmpi v.3.1.1. At the end a get the next error:

mié oct 17 09:57:03 CDT 2018
set MPI = /user/local/bin
set MPIRUN = /user/local/bin/mpirun
mpirun -r ssh -np 12 /home/maria/Escritorio/CMAQ_REPO/CCTM/scripts/BLD_CCTM_v52_gcc/CCTM_v52.exe
[mpiexec@maria-HP-Z820-Workstation] match_arg (utils/args/args.c:159): unrecognized argument r
[mpiexec@maria-HP-Z820-Workstation] HYDU_parse_array (utils/args/args.c:174): argument matching returned error
[mpiexec@maria-HP-Z820-Workstation] parse_args (ui/mpich/utils.c:1596): error parsing input array
[mpiexec@maria-HP-Z820-Workstation] HYD_uii_mpx_get_parameters (ui/mpich/utils.c:1648): unable to parse user arguments
[mpiexec@maria-HP-Z820-Workstation] main (ui/mpich/mpiexec.c:149): error parsing parameters

Any of you know how can I fix it?
Appreciate your help.

María

2

the error is telling you that it doesn’t recognize the -r parameter in
mpirun -r ssh -np 12

would double check if that’s the correct syntax

3

try removing -r ssh from the command.
Use this instead:

mpirun -np 12
4

Thank you for your suggestion, it worked. However, I am getting a new error:

set MPI = /usr/local/bin
set MPIRUN = /usr/local/bin/mpirun
mpirun -np 12 /home/maria/Escritorio/CMAQ_REPO/CCTM/scripts/BLD_CCTM_v52_gcc/CCTM_v52.exe

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0 0x14b5f1de516a in ???
#1 0x14b5f1de4393 in ???
#2 0x14b5f127d13f in ???
#3 0x14b5f2214c67 in ???
#4 0x14b5f24ead8a in ???
#5 0x562d8af8a8f5 in ???
#6 0x562d8af91897 in ???
#7 0x562d8adfdc8e in ???
#8 0x14b5f12671c0 in ???
#9 0x562d8adfdcc9 in ???
#10 0xffffffffffffffff in ???

I was looking for a solution, but I did not get a possible answer. Do you have any idea?

Thank you for your help.
María

5

Try changing the stacksize limit using the command in your shell before you run cmaq, or you can add it to your .cshrc file in your home directory

limit stacksize unlimited
6

I applied the command that you suggested, however it did not work. I was trying to find a solution but finally I decided to recompile all. At this point, I am getting a new error while I am running the CCTM build script:

distr_env.o: En la función distr_env_': distr_env.c:(.text+0xd6): unreference toompi_mpi_comm_world’
distr_env.c:(.text+0xdd): unreference toompi_mpi_int' distr_env.c:(.text+0xf4): unreference toompi_mpi_comm_world’
distr_env.c:(.text+0xfb): unreference to ompi_mpi_char' distr_env.c:(.text+0x10c): unreference toompi_mpi_comm_world’
distr_env.c:(.text+0x148): unreference to ompi_mpi_comm_world' distr_env.c:(.text+0x14f): unreference toompi_mpi_int’
distr_env.c:(.text+0x16e): unreference to ompi_mpi_comm_world' distr_env.c:(.text+0x175): unreference toompi_mpi_char’
collect2: error: ld returned 1 exit status
Makefile:409: fallo en las instrucciones para el objetivo ‘CCTM_v521.exe’
make: *** [CCTM_v521.exe] Error 1
ERROR while running make command

Do you have any idea to solve this error?
Appreciate your help.

María

7

Did someone do something to the MPI installation in the mean-time?

This would seem to be a MPI-library issue, although that doubled “MPI_” in “undefined reference to mpi_mpi_int” (etc.) is truly strange…

8

Hi Maria,

As a bandage type of solution, you can remove distr_env in the Makefile. Carlie has mentioned "it is strange" and I agree with hime. So if you want to pursue further down about your error, please contact me directly, wong.david-c@epa.gov.

Cheers,
David

9

Hi Maria, I am also getting the same errors though I am using mpich (mvapich2) instead openmpi. How did solve your problem?

10

Uzzal

I decided to use openmpi and those errors disappear.
Maria