CCTM compilation

lizadams · October 4, 2019, 2:43pm

The log file contains the following error which has prevented the bldmake executable to be created.

mpifort: Command not found.

Please try to use the following command

which mpifort

output on my system is:

/nas/longleaf/apps-dogwood/mpi/gcc_9.1.0/openmpi_4.0.1/bin/mpifort

This will determine if your system has mpifort available on your system

If it is available use the following command to determine what compiler it is referencing

 mpifort --version

output on my system is:

GNU Fortran (GCC) 9.1.0
Copyright (C) 2019 Free Software Foundation, Inc.

If it isn’t found, then you may need to load a module on your system.
Use the command

module list

to determine what modules are available

If you don’t have modules available, then please ask your system administrator to install openmpi or, you may need to install it yourself.

zhuli · October 6, 2019, 6:57am

Hi~lizadams

Thanks for your reply.

I’ve used the command ‘which mpifort’, output on my system is:

/usr/bin/which: no mpifort in (/usr/local/bin/bin:/home/cmaq/cmaq_zl/installsf/netcdf_install/hdf5/bin:/usr/local/bin/bin:/home/cmaq/cmaq_zl/installsf/netcdf_install/hdf5/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/home/cmaq/.local/bin:/home/cmaq/bin:/usr/local/git/bin:/home/cmaq/cmaq_zl/installsf/mpich-install/bin:/usr/local/netcdf4.1.3/bin:/usr/local/netcdf4.1.3/include:/usr/local/netcdf4.1.3/lib:/usr/local/git/bin:/home/cmaq/cmaq_zl/installsf/mpich-install/bin:/usr/local/netcdf4.1.3/bin:/usr/local/netcdf4.1.3/include:/usr/local/netcdf4.1.3/lib)

And then I check the version of my complier using ‘gcc -v’&’ gfortran -v’ :

COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/local/libexec/gcc/x86_64-pc-linux-gnu/6.4.0/lto-wrapper

COLLECT_GCC=gfortran
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/4.8.5/lto-wrapper

So my questions are:

**Do I have to upgrade gcc or/and gfortran to the latest versions? **
Do I need to reinstall the mpich-3.2.1 or install openmpi?
Which libraries need to be reinstalled or compiled?

Kindly help, please.

zhuli · October 8, 2019, 1:33am

Hi~ Catalyst2682

I got a similar error as you when compling CCTM:

Error: Syntax error in COMMON statement at (1)
STATE3.EXT:191:27:
Error: Syntax error in COMMON statement at (1)
make: *** [pshut3.o] Error 1
ERROR while running make command

How did you solve this problem?
Kindly help me, please~

Catalyst2682 · October 8, 2019, 5:26am

Hello @zhuli,

I have sent you a message regarding this.

Regards,

Catalyst

lizadams · November 7, 2019, 4:23pm

You will need to download and install openmpi if you don’t have it installed already on your system.
https://www.open-mpi.org/

zhuli · November 11, 2019, 1:58pm

Hi,lizadams
I’m trying to compile CCTM with PGI but got an error as follows:

mpifort -c -Mfixed -Mextend -mcmodel=medium -tp px -O3 -I /home/cmaq/MOD/CMAQv5.3/zlpgi_CMAQ/lib/x86_64/pgi/ioapi/lib -I /home/cmaq/MOD/CMAQv5.3/zlpgi_CMAQ/lib/x86_64/pgi/ioapi/include_files -I /home/cmaq/MOD/CMAQv5.3/zlpgi_CMAQ/lib/x86_64/pgi/mpi -I. pio_init.f
PGF90-F-0004-Corrupt or Old Module file /home/cmaq/MOD/CMAQv5.3/zlpgi_CMAQ/lib/x86_64/pgi/ioapi/lib/m3utilio.mod (pio_init.f: 151)
PGF90/x86-64 Linux 19.4-0: compilation aborted
make: *** [pio_init.o] error 2

Could you give me some help?
Thanks a lot.

zhuli

lizadams · November 11, 2019, 11:33pm

Hi Zhuli,

I have pgi available on our servers, I will install openmpi to try and reproduce the issue that you are seeing.
The error message states that you are using a Corrupt or Old Module file: ioapi/lib/m3utilio.mod.

Can you report the versions of underlying software that you are using to compile CMAQ and confirm that you have used the same pgf90 compiler to build openmpi, netCDF, and I/O API following instructions similar to the following draft tutorial

github.com

lizadams/CMAQ/blob/master/DOCS/Users_Guide/Tutorials/CMAQ_UG_tutorial_build_library_pgi.md

## Load PGI compiler

```
module load pgi
```

1. Check version number

```
pgf90 --version
```


## Install openmpi 
tips on how to do this using pgi compiler found here: https://www.pgroup.com/userforum/viewtopic.php?t=5413
I also had to download and install the latest UCX library following these instructions:
https://github.com/open-mpi/ompi/issues/4441
https://github.com/openucx/ucx/wiki/OpenMPI-and-OpenSHMEM-installation-with-UCX

1. Download openmp from the following website https://www.open-mpi.org/software/ompi/v4.0/

This file has been truncated. show original

Also, be sure you are following these instructions once the openmpi, netcdf, netcdff and ioapi libraries are built:

github.com

USEPA/CMAQ/blob/master/DOCS/Users_Guide/Tutorials/CMAQ_UG_tutorial_benchmark.md

# CMAQ Installation & Benchmarking Tutorial

Purpose: This guide describes how to install and run the CMAQ test case, which serves two different purposes. The first being to familiarize the user with the CMAQ suite of programs and how they work together, and secondly to verify the installation of the software on your system via benchmarking. 

Benchmarking refers to a simulation that is used to verify that the software is installed correctly.  Benchmarking CMAQ is recommended in the following circumstances:
- Installation by a new user
- Installation on a new server     
- Following kernel upgrades
- Following Fortran/C compiler upgrades
- Following netCDF or I/O API library upgrades

## System Checks 

The following support software are required for compiling and running CMAQ.  

1. Fortran and C compilers, e.g., [Intel](https://software.intel.com/en-us/fortran-compilers), [Portland Group](http://www.pgroup.com), [Gnu](https://gcc.gnu.org/wiki/GFortran)
2. [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
2. Message Passing Interface (MPI), e.g., [OpenMPI](https://www.open-mpi.org) or [MVAPICH2](http://www.mcs.anl.gov/research/projects/mpich2).
3. Latest release of [netCDF-C](https://www.unidata.ucar.edu/software/netcdf/docs/getting_and_building_netcdf.html) and [netCDF-Fortran](https://www.unidata.ucar.edu/software/netcdf/docs/building_netcdf_fortran.html) **built without netCDF4, HDF5, HDF4, DAP client, PnetCDF, or zlib support** 
4. [I/O API](http://www.cmascenter.org/ioapi) version 3.2 or later

This file has been truncated. show original

pgf90 --version
mpif90 --version

Also be sure that the paths to the ioapi library are correct under the CMAQv5.3 directory
Here is what I have:
/proj/ie/proj/CMAS/CMAQ/CMAQv5.3_branch_pgi/pgi_19.7_openmpi_4.0.2/lib/x86_64/pgi/ioapi

lib -> /proj/ie/proj/CMAS/CMAQ/CMAQv5.3_branch_pgi/pgi_19.7_openmpi_4.0.2/LIB/ioapi-3.2/Linux2_x86_64_openmpi_4.0.2_pgi_19.7

include_files -> /proj/ie/proj/CMAS/CMAQ/CMAQv5.3_branch_pgi/pgi_19.7_openmpi_4.0.2/LIB/ioapi-3.2/ioapi/fixed_src

zhuli · November 12, 2019, 1:38am

Thank you so much, lizadams.

I’ve installed GUN9.1.0 and PGI19.4 on my workstation, and builded netcdf-c-4.7.0, netcdf-f-4.4.5, ioapi3.2 with GUN. Maybe I used PGI because when I “make check” in installation of netcdf-f, I got an error: Make [2]: pgf77: command not found. So I have to install PGI7.1.6 and then “ln -s pgf77 /PGI19.4install/bin”.

I’ve checked the version of OPENMPI which was installed with PGI19.4 is openmpi-3.1.3.

So my questions are:

Do I have to compile CMAQ with GUN only?
Do I need to install OPENMPI with GUN?
What should I do next?

Appreciate your help.
Zhuli

lizadams · November 12, 2019, 3:17pm

Hi Zhuli,

In general, it is best not to mix compilers. If you are using GNU 9.1.0, then stick with that the whole way, to build everything.
For the make check step, you can set the F77 compiler to use the F90 compiler.

Or, if you have the PGI compiler, then use that for building all of the libraries.

If it is available on your server, use the module load command to set up your environment for the compiler, as the paths are automatically set up, along with the LD_LIBRARY_PATH for that compiler. If you don’t have the module load command option for both the compiler, and the openmpi layer, you may need to edit your .cshrc to add the paths so that when you run the configure command that it picks up the correct compiler.

For instance, the command ‘module show’ allows you to see all of the environment variables that are set by default when you load that module.
module show openmpi_4.0.1/gcc_9.1.0

/nas/longleaf/apps-dogwood/lmod/modulefiles/Core/openmpi_4.0.1/gcc_9.1.0.lua:

whatis(“Set up environment for OPENMPI 4.0.1 and GCC 9.1.0 compiler “)
family(“mpi”)
load(“gcc/9.1.0”)
setenv(“MVAPICH_HOME”,”/nas/longleaf/apps-dogwood/mpi/gcc_9.1.0/openmpi_4.0.1”)
setenv(“MPICH_HOME”,"/nas/longleaf/apps-dogwood/mpi/gcc_9.1.0/openmpi_4.0.1")
setenv(“MPI_HOME”,"/nas/longleaf/apps-dogwood/mpi/gcc_9.1.0/openmpi_4.0.1")
prepend_path(“PATH”,"/nas/longleaf/apps-dogwood/mpi/gcc_9.1.0/openmpi_4.0.1/bin")
prepend_path(“PATH”,"/nas/longleaf/apps/gcc/9.1.0/bin")
prepend_path(“LD_LIBRARY_PATH”,"/nas/longleaf/apps-dogwood/mpi/gcc_9.1.0/openmpi_4.0.1/lib")
prepend_path(“LD_LIBRARY_PATH”,"/nas/longleaf/apps/gcc/9.1.0/lib64")
prepend_path(“LD_LIBRARY_PATH”,"/nas/longleaf/apps/gcc/9.1.0/lib")
prepend_path(“MANPATH”,"/nas/longleaf/apps-dogwood/mpi/gcc_9.1.0/openmpi_4.0.1/share/man")
prepend_path(“INCLUDE”,"/nas/longleaf/apps-dogwood/mpi/gcc_9.1.0/openmpi_4.0.1/include")
prepend_path(“CPATH”,"/nas/longleaf/apps-dogwood/mpi/gcc_9.1.0/openmpi_4.0.1/include")
help([[ openmpi_gcc/9.1.0 module
****************************************************

      This module sets up the following environment

]])

johnsmith101 · January 9, 2020, 1:36pm

I’m not sure about ioapi, because I used make build, but other libraries and WRF installed with the same compilers.

Allen · March 8, 2020, 7:10am

Hello @Catalyst2682,
Could you please send me the same message?
I have a same problem like:
STATE3.EXT:174.27:
Included at pshut3.F:53:

    COMMON  / BSTATE3 /                                             &
                       1

Error: Syntax error in COMMON statement at (1)
STATE3.EXT:191.27:
Included at pshut3.F:53:

    COMMON  / CSTATE3 /                                             &
                       1

Error: Syntax error in COMMON statement at (1)
make: *** [pshut3.o] Error 1
ERROR while running make command

Could you give some help?
Thanks a lot!
Allen

cjcoats · March 8, 2020, 10:15pm

See this section of the I/O API installation guide: https://cjcoats.github.io/ioapi/AVAIL.html#cmaq:

NOTE for users of CMAQ and SMOKE

According to the ISO Fortran Standard, there are two “source formats” for Fortran source code – the older ( f77 ) “fixed” format with code in columns 7-72 and continuation markers in column 6, and the ( f90 ) “free” format, with source in columns 1-132 and ampersand (“&”) for trailing continuation markers, as well as a specification for code which is acceptable to both forms.

Since it is used by various models that use “free” format, the I/O API carefully follows the Standard in this regard, and uses the ISO-app[roved “both-form” format for its Fortran INCLUDE files.

Some of the models—in particular, CMAQ and SMOKE however, do not—instead using a “fixed-132” format. The I/O API Makefile has a way of dealing with this; the commands

cd /ioapi
make fixed_src
will create a new directory ioapi/fixed_src and populate it with copies of the INCLUDE files that have been edited to work with the non-Standard “fixed-132” format. You will need to use these INCLUDE files with CMAQ and SMOKE.

Allen · March 9, 2020, 9:44am

Thank you for reply, but It didn’t succeed.
After I make fixed_src, I go back to compile CCTM. It still have the same error message.
Could you please give some advise for this situation?
Thank a lot!
Allen

cjcoats · March 9, 2020, 3:11pm

You have to use those INCLUDE-files in your CMAQ build.

Allen · March 10, 2020, 9:12am

I copy the .EXT files to INCLUDE-files location, and I passed the error.
Successfully compiling CCTM.
Thanks a lot!

Allen

Catalyst2682 · March 11, 2020, 9:09am

@Allen
I am sorry for the late response, I just got your message on this thread.
I am glad you’ve compiled it successfully.

Cheers and regards,

Catalyst

johnsmith101 · April 11, 2020, 11:34am

Thanks man issue solved tubemate mobdro

Rypo · March 6, 2021, 12:52am

Have you figure out this problem? I meet the same problem.
distr_env.c(36): catastrophic error: cannot open source file “mpi.h”
#include “mpi.h”
^

compilation aborted for distr_env.c (code 4)
make: *** [distr_env.o] Error 4
ERROR while running make command

Thank you

cjcoats · March 6, 2021, 12:33pm

[General rule: include the full text of the compile-line at which the error occurred.]

Is the compiler mpicc?

If not, it should be.

If so, it indicates a mis-installation of your MPI – mpicc and mpif90 are really wrappers around other “conventional” compilers, that know about MPI include-files, libraries, etc., and use that knowledge to set up and then execute the conventional compilers, and this “cannot open” indicates that either MPI is not correctly installed or else that something is mucking around with the compile-flags to prevent mpicc from working correctly.

And without the text of the compile-line, I don’t have a clue which.

Topic		Replies	Views
I made an error when Compiling CMAQ20200828 for CCTM Compiling	3	948	October 8, 2020
CMAQ5.2 CCTM compile error Compiling	1	655	September 24, 2019
Trying to compile CCTM, but I get an error message relating to the Makefile Compiling	5	647	October 18, 2023
When compiling bldit_cctm.csh for CMAQ v5.4, errors related to vppm.F and twoway_rrtmg_aero_optics.F90 were encountered CMAQ cmaq	1	27	July 31, 2024
Error in compile CCTM Compiling	18	3107	April 19, 2020

CCTM compilation

For instance, the command ‘module show’ allows you to see all of the environment variables that are set by default when you load that module. module show openmpi_4.0.1/gcc_9.1.0

/nas/longleaf/apps-dogwood/lmod/modulefiles/Core/openmpi_4.0.1/gcc_9.1.0.lua:

NOTE for users of CMAQ and SMOKE

Related topics

For instance, the command ‘module show’ allows you to see all of the environment variables that are set by default when you load that module.
module show openmpi_4.0.1/gcc_9.1.0