# Compilation flags for CMAQ using gcc 10.2.0 and openMPI **URL:** \ **Category:** CMAQ\ **Created:** [August 8, 2023, 11:29pm UTC](https://forum.cmascenter.org/t/compilation-flags-for-cmaq-using-gcc-10-2-0-and-openmpi/4295 "2023-08-08T23:29:57Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![ferrada](https://avatars.discourse-cdn.com/v4/letter/f/898d66/32.png) [@ferrada](https://forum.cmascenter.org/u/ferrada)\ **Post date:** [August 8, 2023, 11:29pm UTC](https://forum.cmascenter.org/t/compilation-flags-for-cmaq-using-gcc-10-2-0-and-openmpi/4295/1 "2023-08-08T23:29:57Z")
Hello, I am trying to compile CCTM from CMAQ v5.3.2.1. For this version, I want to read emissions compressed in netCDF-4 (.nc4) format. I am using precompiled libraries already available on my HPC system. These are: gcc 10.2.0, openmpi 4.1.3, zlib 1.2.11, hdf5 1.10.7, netcdf-c 4.8.1 and netcdf-fortran 4.5.3. All of them where built with the same gcc compiler. With them, I have been able to compile I/O API 3.2 (special version 20200828) with sucess - or I least I believe so since after adding some flags, such as -DIOAPI\_NCF4=1 and -fallow-argument-mismatch, I was able to compile without errors. The problem now appears when compiling CCTM. Sadly, as a new user I am not allowed to upload the log file. I have properly defined the directories to my openMPI, however, the error shows that the mpi.h file does not exist: ```auto distr_env.c:36:10: fatal error: mpi.h: No such file or directory 36 | #include "mpi.h" | ^ ~~~~~~ compilation terminated. ``` On my CMAQ\_HOME/lib/gcc/mpi the file mpi.h indeed does not exist. Checking the config\_cmaq.csh I see that the code links the openmpi/lib to my CMAQ/lib/gcc/mpi. However, after inspecting I see that the mpi.h file exist on the openmpi directory, but under openmpi/include rather than openmpi/lib. Should I manually change this? I dug into several threads on this forum and I could not find someone with a similar error. How can I solve this? Also, I am a bit confused since I have seen some threads recommending to use the “-fopenmp” flag but at the same time, the config\_cmaq.csh file expresses that it should not be used (?): `setenv myLINK_FLAG # "-fopenmp" openMP not supported w/ CMAQ` Any feedback would be much appreciated. Thanks. =================================== build\_cctm.log: (I removed all the intro to be able to post this topic). ```auto Compiler is set to gcc *****(many lines here were removed)***** USE/MODULE dependencies defined Makefile generated mpifort -c -ffixed-form -ffixed-line-length-132 -funroll-loops -finit-character=32 -fallow-argument-mismatch -fallow-argument-mismatch -O3 -funroll-loops -finit-character=32 -Wtabs -Wsurprising -march=native -ftree-vectorize -ftree-loop-if-convert -finline-limit=512 -I /lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/lib/x86_64/gcc/ioapi/lib -I /lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/lib/x86_64/gcc/ioapi/include_files -I /lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/lib/x86_64/gcc/mpi -I. -fallow-argument-mismatch -Dparallel -Dstage_opt -DSUBST_BARRIER=SE_BARRIER -DSUBST_GLOBAL_MAX=SE_GLOBAL_MAX -DSUBST_GLOBAL_MIN=SE_GLOBAL_MIN -DSUBST_GLOBAL_MIN_DATA=SE_GLOBAL_MIN_DATA -DSUBST_GLOBAL_TO_LOCAL_COORD=SE_GLOBAL_TO_LOCAL_COORD -DSUBST_GLOBAL_SUM=SE_GLOBAL_SUM -DSUBST_GLOBAL_LOGICAL=SE_GLOBAL_LOGICAL -DSUBST_LOOP_INDEX=SE_LOOP_INDEX -DSUBST_SUBGRID_INDEX=SE_SUBGRID_INDEX -DSUBST_HI_LO_BND_PE=SE_HI_LO_BND_PE -DSUBST_SUM_CHK=SE_SUM_CHK -DSUBST_SE_INIT=SE_INIT -DSUBST_INIT_ARRAY=SE_INIT_ARRAY -DSUBST_COMM=SE_COMM -DSUBST_MY_REGION=SE_MY_REGION -DSUBST_SLICE=SE_SLICE -DSUBST_GATHER=SE_GATHER -DSUBST_DATA_COPY=SE_DATA_COPY -DSUBST_IN_SYN=SE_IN_SYN -DSUBST_PE_COMM=\"./PE_COMM.EXT\" -DSUBST_CONST=\"./CONST.EXT\" -DSUBST_FILES_ID=\"./FILES_CTM.EXT\" -DSUBST_EMISPRM=\"./EMISPRM.EXT\" -DSUBST_MPI=\"/lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/lib/x86_64/gcc/mpi/include/mpif.h\" BEIS_DEFN.F mpifort -c -ffixed-form -ffixed-line-length-132 -funroll-loops -finit-character=32 -fallow-argument-mismatch -fallow-argument-mismatch -O3 -funroll-loops -finit-character=32 -Wtabs -Wsurprising -march=native -ftree-vectorize -ftree-loop-if-convert -finline-limit=512 -I /lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/lib/x86_64/gcc/ioapi/lib -I /lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/lib/x86_64/gcc/ioapi/include_files -I /lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/lib/x86_64/gcc/mpi -I. -fallow-argument-mismatch -Dparallel -Dstage_opt -DSUBST_BARRIER=SE_BARRIER -DSUBST_GLOBAL_MAX=SE_GLOBAL_MAX -DSUBST_GLOBAL_MIN=SE_GLOBAL_MIN -DSUBST_GLOBAL_MIN_DATA=SE_GLOBAL_MIN_DATA -DSUBST_GLOBAL_TO_LOCAL_COORD=SE_GLOBAL_TO_LOCAL_COORD -DSUBST_GLOBAL_SUM=SE_GLOBAL_SUM -DSUBST_GLOBAL_LOGICAL=SE_GLOBAL_LOGICAL -DSUBST_LOOP_INDEX=SE_LOOP_INDEX -DSUBST_SUBGRID_INDEX=SE_SUBGRID_INDEX -DSUBST_HI_LO_BND_PE=SE_HI_LO_BND_PE -DSUBST_SUM_CHK=SE_SUM_CHK -DSUBST_SE_INIT=SE_INIT -DSUBST_INIT_ARRAY=SE_INIT_ARRAY -DSUBST_COMM=SE_COMM -DSUBST_MY_REGION=SE_MY_REGION -DSUBST_SLICE=SE_SLICE -DSUBST_GATHER=SE_GATHER -DSUBST_DATA_COPY=SE_DATA_COPY -DSUBST_IN_SYN=SE_IN_SYN -DSUBST_PE_COMM=\"./PE_COMM.EXT\" -DSUBST_CONST=\"./CONST.EXT\" -DSUBST_FILES_ID=\"./FILES_CTM.EXT\" -DSUBST_EMISPRM=\"./EMISPRM.EXT\" -DSUBST_MPI=\"/lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/lib/x86_64/gcc/mpi/include/mpif.h\" centralized_io_module.F mpifort -c -ffixed-form -ffixed-line-length-132 -funroll-loops -finit-character=32 -fallow-argument-mismatch -fallow-argument-mismatch -O3 -funroll-loops -finit-character=32 -Wtabs -Wsurprising -march=native -ftree-vectorize -ftree-loop-if-convert -finline-limit=512 -I /lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/lib/x86_64/gcc/ioapi/lib -I /lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/lib/x86_64/gcc/ioapi/include_files -I /lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/lib/x86_64/gcc/mpi -I. -fallow-argument-mismatch -Dparallel -Dstage_opt -DSUBST_BARRIER=SE_BARRIER -DSUBST_GLOBAL_MAX=SE_GLOBAL_MAX -DSUBST_GLOBAL_MIN=SE_GLOBAL_MIN -DSUBST_GLOBAL_MIN_DATA=SE_GLOBAL_MIN_DATA -DSUBST_GLOBAL_TO_LOCAL_COORD=SE_GLOBAL_TO_LOCAL_COORD -DSUBST_GLOBAL_SUM=SE_GLOBAL_SUM -DSUBST_GLOBAL_LOGICAL=SE_GLOBAL_LOGICAL -DSUBST_LOOP_INDEX=SE_LOOP_INDEX -DSUBST_SUBGRID_INDEX=SE_SUBGRID_INDEX -DSUBST_HI_LO_BND_PE=SE_HI_LO_BND_PE -DSUBST_SUM_CHK=SE_SUM_CHK -DSUBST_SE_INIT=SE_INIT -DSUBST_INIT_ARRAY=SE_INIT_ARRAY -DSUBST_COMM=SE_COMM -DSUBST_MY_REGION=SE_MY_REGION -DSUBST_SLICE=SE_SLICE -DSUBST_GATHER=SE_GATHER -DSUBST_DATA_COPY=SE_DATA_COPY -DSUBST_IN_SYN=SE_IN_SYN -DSUBST_PE_COMM=\"./PE_COMM.EXT\" -DSUBST_CONST=\"./CONST.EXT\" -DSUBST_FILES_ID=\"./FILES_CTM.EXT\" -DSUBST_EMISPRM=\"./EMISPRM.EXT\" -DSUBST_MPI=\"/lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/lib/x86_64/gcc/mpi/include/mpif.h\" PHOT_MOD.F gcc -c -O2 -fallow-argument-mismatch -DFLDMN -I/lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/lib/x86_64/gcc/mpi/include -I. distr_env.c mpifort -c -ffixed-form -ffixed-line-length-132 -funroll-loops -finit-character=32 -fallow-argument-mismatch -fallow-argument-mismatch -O3 -funroll-loops -finit-character=32 -Wtabs -Wsurprising -march=native -ftree-vectorize -ftree-loop-if-convert -finline-limit=512 -I /lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/lib/x86_64/gcc/ioapi/lib -I /lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/lib/x86_64/gcc/ioapi/include_files -I /lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/lib/x86_64/gcc/mpi -I. -fallow-argument-mismatch -Dparallel -Dstage_opt -DSUBST_BARRIER=SE_BARRIER -DSUBST_GLOBAL_MAX=SE_GLOBAL_MAX -DSUBST_GLOBAL_MIN=SE_GLOBAL_MIN -DSUBST_GLOBAL_MIN_DATA=SE_GLOBAL_MIN_DATA -DSUBST_GLOBAL_TO_LOCAL_COORD=SE_GLOBAL_TO_LOCAL_COORD -DSUBST_GLOBAL_SUM=SE_GLOBAL_SUM -DSUBST_GLOBAL_LOGICAL=SE_GLOBAL_LOGICAL -DSUBST_LOOP_INDEX=SE_LOOP_INDEX -DSUBST_SUBGRID_INDEX=SE_SUBGRID_INDEX -DSUBST_HI_LO_BND_PE=SE_HI_LO_BND_PE -DSUBST_SUM_CHK=SE_SUM_CHK -DSUBST_SE_INIT=SE_INIT -DSUBST_INIT_ARRAY=SE_INIT_ARRAY -DSUBST_COMM=SE_COMM -DSUBST_MY_REGION=SE_MY_REGION -DSUBST_SLICE=SE_SLICE -DSUBST_GATHER=SE_GATHER -DSUBST_DATA_COPY=SE_DATA_COPY -DSUBST_IN_SYN=SE_IN_SYN -DSUBST_PE_COMM=\"./PE_COMM.EXT\" -DSUBST_CONST=\"./CONST.EXT\" -DSUBST_FILES_ID=\"./FILES_CTM.EXT\" -DSUBST_EMISPRM=\"./EMISPRM.EXT\" -DSUBST_MPI=\"/lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/lib/x86_64/gcc/mpi/include/mpif.h\" hrsolver.F cc1: warning: command-line option ‘-fallow-argument-mismatch’ is valid for Fortran but not for C distr_env.c:36:10: fatal error: mpi.h: No such file or directory 36 | #include "mpi.h" | ^ ~~~~~~ compilation terminated. make: *** [Makefile:462: distr_env.o] Error 1 make: *** Waiting for unfinished jobs.... centralized_io_module.F:1882:0: 1882 | #endif ! end if stage option | Warning: extra tokens at end of #endif directive centralized_io_module.F:1883:0: 1883 | #endif ! end if m3dry option | Warning: extra tokens at end of #endif directive centralized_io_module.F:3784:51: 3784 | WRITE( LOGDEV, "(5x,A,/,5x,A,/,5x,A)"), | 1 Warning: Legacy Extension: Comma before i/o item list at (1) centralized_io_module.F:3803:37: 3803 | WRITE( XMSG, '(A,A,A)' ),'ERROR: Could not read ', | 1 Warning: Legacy Extension: Comma before i/o item list at (1) centralized_io_module.F:3812:59: 3812 | WRITE( LOGDEV, "(5x,A,/,5x,A,/,5x,A,/,5x,A)" ), | 1 Warning: Legacy Extension: Comma before i/o item list at (1) centralized_io_module.F:3828:59: 3828 | WRITE( LOGDEV, "(5x,A,/,5x,A,/,5x,A,/,5x,A)" ), | 1 Warning: Legacy Extension: Comma before i/o item list at (1) centralized_io_module.F:3850:59: 3850 | WRITE( LOGDEV, "(5x,A,/,5x,A,/,5x,A,/,5x,A)" ), | 1 Warning: Legacy Extension: Comma before i/o item list at (1) centralized_io_module.F:3872:59: 3872 | WRITE( LOGDEV, "(5x,A,/,5x,A,/,5x,A,/,5x,A)" ), | 1 Warning: Legacy Extension: Comma before i/o item list at (1) centralized_io_module.F:3887:59: 3887 | WRITE( LOGDEV, "(5x,A,/,5x,A,/,5x,A,/,5x,A)" ), | 1 Warning: Legacy Extension: Comma before i/o item list at (1) centralized_io_module.F:3901:59: 3901 | WRITE( LOGDEV, "(5x,A,/,5x,A,/,5x,A,/,5x,A)" ), | 1 Warning: Legacy Extension: Comma before i/o item list at (1) centralized_io_module.F:3915:59: 3915 | WRITE( LOGDEV, "(5x,A,/,5x,A,/,5x,A,/,5x,A)" ), | 1 Warning: Legacy Extension: Comma before i/o item list at (1) centralized_io_module.F:2361:53: 2361 | WRITE( LOGDEV, '(/,5x,A,/,5x,A,/5x,A)' ), | 1 Warning: Legacy Extension: Comma before i/o item list at (1) **ERROR** while running make command mv Makefile Makefile.gcc mv: 'Makefile' and 'Makefile.gcc' are the same file if ( -e Makefile.gcc && -e Makefile ) rm Makefile rm Makefile ln -s Makefile.gcc Makefile if ( 0 != 0 ) then if ( -e /lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/CCTM/scripts/BLD_CCTM_v5321_gcc_STAGE/CCTM_v5321.cfg ) then echo >>> previous CCTM_v5321.cfg exists, re-naming to CCTM_v5321.cfg.old <<< >>> previous CCTM_v5321.cfg exists, re-naming to CCTM_v5321.cfg.old <<< mv /lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/CCTM/scripts/BLD_CCTM_v5321_gcc_STAGE/CCTM_v5321.cfg /lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/CCTM/scripts/BLD_CCTM_v5321_gcc_STAGE/CCTM_v5321.cfg.old endif mv CCTM_v5321.cfg.bld /lustre/isaac/proj/UTK0221/NIH/CONUS/CMAQ/5.3.2.1-gcc/CCTM/scripts/BLD_CCTM_v5321_gcc_STAGE/CCTM_v5321.cfg exit ``` =================================== My config\_cmaq.csh: (I kept only the settings relevant to gcc) ```auto #!/bin/csh -f # ================= CMAQv5.3.X Configuration Script ================= # # Requirements: I/O API & netCDF libraries # # PGI, Intel, or Gnu Fortran compiler # # MPICH for multiprocessor computing # # Optional: Git for GitHub source code repository # # # # Note that this script was configured/tested on Red Hat Linux O/S # # # # To report problems or request help with this script/program: # # http://www.cmascenter.org/help-desk.cfm # # =================================================================== # #> Critical Folder Locations # CMAQ_HOME - this is where the config_cmaq.csh script is located. It # is also the root directory for all the executables. It may include # the repository if the user is building CMAQ inside the repository. It # may, on the other hand, be outside the repository if the user has # created a separate project directory where they wish to put build- # and run-scripts as well as executables. setenv CMAQ_HOME $cwd # CMAQ_REPO - this is always the location of the CMAQ repository that # the user will pull from to create exectuables. If the user is building # CMAQ inside the repository then it will be equal to CMAQ_HOME. If not, # the user must supply an alternative folder locaiton. setenv CMAQ_REPO /lustre/isaac/proj/UTK0221/andrew/CMAQ/5.3.2.1 # CMAQ_DATA - this may be where the input data are located. It may be a # symbolic link to another location on the system, but it should be # provided here setenv CMAQ_DATA $CMAQ_HOME/data if ( ! -d $CMAQ_DATA ) mkdir -p $CMAQ_DATA cd $CMAQ_HOME #=============================================================================== #> architecture & compiler specific settings #=============================================================================== #> Set the compiler option if ( $#argv == 1 ) then #> Use the user's input to set the compiler parameter setenv compiler $1 setenv compilerVrsn Empty else if ( $#argv == 2 ) then #> Compiler Name and Version have been provided setenv compiler $1 setenv compilerVrsn $2 else if ( $#argv == 0 ) then #> If config.cmaq is called from Bldit.cctm or run.cctm, then this #> variable is already defined if ( ! $?compiler ) then echo "Error: 'compiler' should be set either in the" echo " environment or as input to config.cmaq" echo " Example:> ./config.cmaq [compiler]" echo " Options: intel | gcc | pgi" exit else if ( ! $?compilerVrsn ) then setenv compilerVrsn Empty endif else #> More than two inputs were given. Exit this script just to #> be on the safe side. echo "Error: Too many inputs to config.cmaq. This script" echo " is expecting one input (the name of the" echo " desired compiler. In some installations, you " echo " may also be able to specify the compiler version " echo " as the second input, but this is not by default." exit endif echo "Compiler is set to $compiler" # GAF: setenv NCDIR /spack/spack-0.17.2/apps/linux-rhel8-cascadelake/gcc-10.2.0/netcdf-c-4.8.1-7dwpqiw3lhygretuczntsfnug27ark67 setenv NFDIR /spack/spack-0.17.2/apps/linux-rhel8-cascadelake/gcc-10.2.0/netcdf-fortran-4.5.3-slv6qm3skovi7vcwvnveuvliajniqovh setenv IOAPI_BLD /lustre/isaac/proj/UTK0221/andrew/LIBRARIES/ioapi-3.2/Linux2_x86_64gfort_openmpi_4.1.3_gcc_10.2.0 setenv IOAPI_SRC /lustre/isaac/proj/UTK0221/andrew/LIBRARIES/ioapi-3.2 setenv OPENMPIDIR /sw/isaac/applications/openmpi/4.1.3/rhel8.7-gcc10.2.0/openmpi-4.1.3 #> Compiler flags and settings switch ( $compiler ) #> gfortran compiler............................................................ case gcc: #> I/O API, netCDF, and MPI library locations setenv IOAPI_INCL_DIR ${IOAPI_SRC}/ioapi/fixed_src #> I/O API include header files setenv IOAPI_LIB_DIR ${IOAPI_BLD} #> I/O API libraries setenv NETCDF_LIB_DIR ${NCDIR}/lib #> netCDF C directory path setenv NETCDF_INCL_DIR ${NCDIR}/include #> netCDF C directory path setenv NETCDFF_LIB_DIR ${NFDIR}/lib #> netCDF Fortran directory path setenv NETCDFF_INCL_DIR ${NFDIR}/include #> netCDF Fortran directory path setenv MPI_LIB_DIR ${OPENMPIDIR}/lib #> MPI directory path setenv MPI_INCL_DIR ${OPENMPIDIR}/include #> Compiler Aliases and Flags #> set the compiler flag -fopt-info-missed to generate a missed optimization report in the bldit logfile setenv myFC mpifort setenv myCC gcc setenv myFSTD "-O3 -funroll-loops -finit-character=32 -Wtabs -Wsurprising -march=native -ftree-vectorize -ftree-loop-if-convert -finline-limit=512" setenv myDBG "-Wall -O0 -g -fcheck=all -ffpe-trap=invalid,zero,overflow -fbacktrace" setenv myFFLAGS "-ffixed-form -ffixed-line-length-132 -funroll-loops -finit-character=32 -fallow-argument-mismatch" setenv myFRFLAGS "-ffree-form -ffree-line-length-none -funroll-loops -finit-character=32" setenv myCFLAGS "-O2 -fallow-argument-mismatch" setenv myLINK_FLAG # "-fopenmp" openMP not supported w/ CMAQ setenv extra_lib "" #setenv mpi_lib "-lmpi_mpifh" #> -lmpich for mvapich or -lmpi for openmpi setenv mpi_lib "" #> -lmpich for mvapich or -lmpi for openmpi breaksw default: echo "*** Compiler $compiler not found" exit(2) breaksw endsw #> Apply Specific Module and Library Location Settings for those working inside EPA # source /work/MOD3DEV/cmaq_common/cmaq_env.csh #>>> UNCOMMENT if at EPA #> Add The Complier Version Number to the Compiler String if it's not empty setenv compilerString ${compiler} if ( $compilerVrsn != "Empty" ) then setenv compilerString ${compiler}${compilerVrsn} endif #=============================================================================== #> I/O API, netCDF, and MPI libraries setenv netcdf_lib "-lnetcdf" #> -lnetcdff -lnetcdf for netCDF v4.2.0 and later setenv netcdff_lib "-lnetcdff" setenv ioapi_lib "-lioapi" setenv pnetcdf_lib "-lpnetcdf" #> Query System Info and Current Working Directory setenv system "`uname -m`" setenv bld_os "`uname -s``uname -r | cut -d. -f1`" setenv lib_basedir $CMAQ_HOME/lib #> Generate Library Locations setenv CMAQ_LIB ${lib_basedir}/${system}/${compilerString} setenv MPI_DIR $CMAQ_LIB/mpi setenv NETCDF_DIR $CMAQ_LIB/netcdf setenv NETCDFF_DIR $CMAQ_LIB/netcdff setenv PNETCDF_DIR $CMAQ_LIB/pnetcdf setenv IOAPI_DIR $CMAQ_LIB/ioapi #> Create Symbolic Links to Libraries if ( ! -d $CMAQ_LIB ) mkdir -p $CMAQ_LIB if ( -e $MPI_DIR ) rm -rf $MPI_DIR # ln -s $MPI_LIB_DIR $MPI_DIR ln -s $MPI_INCL_DIR $MPI_DIR if ( ! -d $NETCDF_DIR ) mkdir $NETCDF_DIR if ( ! -e $NETCDF_DIR/lib ) ln -sfn $NETCDF_LIB_DIR $NETCDF_DIR/lib if ( ! -e $NETCDF_DIR/include ) ln -sfn $NETCDF_INCL_DIR $NETCDF_DIR/include if ( ! -d $NETCDFF_DIR ) mkdir $NETCDFF_DIR if ( ! -e $NETCDFF_DIR/lib ) ln -sfn $NETCDFF_LIB_DIR $NETCDFF_DIR/lib if ( ! -e $NETCDFF_DIR/include ) ln -sfn $NETCDFF_INCL_DIR $NETCDFF_DIR/include if ( ! -d $IOAPI_DIR ) then mkdir $IOAPI_DIR ln -sfn $IOAPI_INCL_DIR $IOAPI_DIR/include_files ln -sfn $IOAPI_LIB_DIR $IOAPI_DIR/lib endif #> Check for netcdf and I/O API libs/includes, error if they don't exist if ( ! -e $NETCDF_DIR/lib/libnetcdf.a ) then echo "ERROR: $NETCDF_DIR/lib/libnetcdf.a does not exist in your CMAQ_LIB directory!!! Check your installation before proceeding with CMAQ build." exit endif if ( ! -e $NETCDFF_DIR/lib/libnetcdff.a ) then echo "ERROR: $NETCDFF_DIR/lib/libnetcdff.a does not exist in your CMAQ_LIB directory!!! Check your installation before proceeding with CMAQ build." exit endif if ( ! -e $IOAPI_DIR/lib/libioapi.a ) then echo "ERROR: $IOAPI_DIR/lib/libioapi.a does not exist in your CMAQ_LIB directory!!! Check your installation before proceeding with CMAQ build." exit endif if ( ! -e $IOAPI_DIR/lib/m3utilio.mod ) then echo "ERROR: $IOAPI_MOD_DIR/m3utilio.mod does not exist in your CMAQ_LIB directory!!! Check your installation before proceeding with CMAQ build." exit endif #> Set executable id setenv EXEC_ID ${bld_os}_${system}${compilerString} ``` ---
**Author:** ![cjcoats](https://avatars.discourse-cdn.com/v4/letter/c/ac8455/32.png) [@cjcoats](https://forum.cmascenter.org/u/cjcoats)\ **Post date:** [August 9, 2023, 10:59am UTC](https://forum.cmascenter.org/t/compilation-flags-for-cmaq-using-gcc-10-2-0-and-openmpi/4295/2 "2023-08-09T10:59:00Z")
The standard UNIX/Linux conventions are that _mpi.h_ should be in a _something/include_ directory; it is an artifact of the way that CMAQ build system attempts to support multiple compilers and MPI systems that you need to sym-link it to (CMAQ’s notion of version-specific) _CMAQ/lib/gcc/mpi._ Your experience with `-fallow-argument-mismatch` is one that shows their methods are inadequate, and don’t allow for Fortran-compiler specifics the way I/O API binary-types such as `Linux2_x86_64gfort10` and `Linux2_x86pg_gcc_mp` do. The _config\_cmaq.csh_ file claim that `-fopenmp` not be be used because CMAQ does not use OpenMP is based upon a failure to understand that including OpenMP support in the compile-and-link does not cost anything: including this flag does not affect the performance of CMAQ in any way. Many other programs do use OpenMP and want this support; if the I/O API library is compiled to support OpenMP, then the CMAQ build-process needs this flag. ---
**Author:** ![ferrada](https://avatars.discourse-cdn.com/v4/letter/f/898d66/32.png) [@ferrada](https://forum.cmascenter.org/u/ferrada)\ **Post date:** [August 10, 2023, 8:29pm UTC](https://forum.cmascenter.org/t/compilation-flags-for-cmaq-using-gcc-10-2-0-and-openmpi/4295/3 "2023-08-10T20:29:53Z")
Thanks for the feedback. Although I am not completely sure what is the solution or workaround to this. Let me clarify, for I/O API I used the `Makeinclude.Linux2_x86_64gfort10` as base. I had to include the `--fallow-argument-mismatch` flag when building it since after a Google search, I found that adding it will remove the errors I was getting. The errors were the same as the ones shown in [this other CMAS thread](https://forum.cmascenter.org/t/rank-mismatch-error-while-installing-ioapi-3-2/1631). However, the solution given there did not solve my problem. But adding this flag did. My guess that this issue was caused by the newer gcc version I am using. Now, I made a test without using this flag when building CCTM and also linking the openmpi/include directory as CMAQ\_HOME/lib/gcc/mpi. Therefore, the mpi.h file is present there now. However, I get the same error as before. The code complains that it cannot find that file. What should I do now? Snapshot of the error: ```auto distr_env.c:36:10: fatal error: mpi.h: No such file or directory 36 | #include "mpi.h" | ^ ~~~~~~ compilation terminated. make: *** [Makefile:461: distr_env.o] Error 1 make: *** Waiting for unfinished jobs.... ``` Note 1: I am not using the `-fopenmp` flag when building CCTM, since I also did not use it for I/O API. However, I am not sure whether I should specify something in other flags in my config\_cmaq.csh, such as extra\_lib, mpi\_lib or myLINK\_FLAG. Note 2: I noticed that the Makefile created defines: ```auto FC = mpifort CC = gcc ``` Is this correct? Should not it be `FC = gfortran` or something similar? Before running the bldit\_cctm.csh script I have set `setenv FC gfortran` but then it is overwritten in the Makefile. ---
**Author:** ![cjcoats](https://avatars.discourse-cdn.com/v4/letter/c/ac8455/32.png) [@cjcoats](https://forum.cmascenter.org/u/cjcoats)\ **Post date:** [August 10, 2023, 8:43pm UTC](https://forum.cmascenter.org/t/compilation-flags-for-cmaq-using-gcc-10-2-0-and-openmpi/4295/4 "2023-08-10T20:43:56Z")
_mpifort_ is actually a script (of sorts) that is built specifically for some C-compiler/Fortran-compiler set (e.g., _gcc_ and _gfortran_); it “knows” from the MPI install-and-build-process how to invoke the Fortran compiler in such a way as to “find” its own include-files and library-files. The way the CMAQ build-process overrides that internal “knowledge” about a particular MPI installation actually reflects a rather serious misunderstanding about the entire MPI-system characteristics ;-(