NaN or Infinity detected in HDIFF CMAQ

Hello everyone,

I am trying to run CMAQ for a gridded emission input. I encounter the following error while doing it:

Compiler is set to gcc

Working Directory is /home/ghajari/Build_CMAQ/CMAQ_Project/CCTM/scripts
Build Directory is /home/ghajari/Build_CMAQ/CMAQ_Project/CCTM/scripts/BLD_CCTM_v532_gcc
Output Directory is /home/ghajari/Build_CMAQ/CMAQ_Project/data/output_CCTM_v532_gcc_Iran_10km
Log Directory is /home/ghajari/Build_CMAQ/CMAQ_Project/data/output_CCTM_v532_gcc_Iran_10km/LOGS
Executable Name is CCTM_v532.exe

---CMAQ EXECUTION ID: CMAQ_CCTMv532_ghajari_20210731_115742_526348971 ---

Set up input and output files for Day 2020-12-24.
CMAQ Processing of Day 20201224 Began at ‫شنبه ۳۱ جولای ۲۱، ساعات ۱۶:۲۷:۴۲ (+0430)‬
        CTM_APPL  |  v532_gcc_Iran_10km_20201224
================================================================================
|                                                                              |
|               The Community Multiscale Air Quality (CMAQ) Model              |
|                                   Version 5.3.2                              |
|                                                                              |
|                          Built and Maintained by the                         |
|                        Office of Research and Development                    |
|                   United States Environmental Protection Agency              |
|                                                                              |
|                            https://www.epa.gov/cmaq                          |
|                                                                              |
|       Source Code:   https://www.github.com/USEPA/cmaq/tree/master           |
|       Documentation: https://www.github.com/USEPA/cmaq/tree/master/DOCS      |
|                                                                              |
|         The CMAQ Model is tested and released with cooperation from          |
|         the Community Modeling and Analysis System (CMAS) Center via         |
|         contract support. CMAS is managed by the Institute for the           |
|         Environment, University of North Carolina at Chapel Hill.            |
|         CMAS URL: (https://www.cmascenter.org)                               |
|                                                                              |
================================================================================

     This program uses the EPA-AREAL/MCNC-EnvPgms/BAMS Models-3
     I/O Applications Programming Interface, [I/O API] which is
     built on top of the netCDF I/O library (Copyright 1993, 1996
     University Corporation for Atmospheric Research/Unidata
     Program) and the PVM parallel-programming library (from
     Oak Ridge National Laboratory).
     Copyright (C) 1992-2002 MCNC,
     (C) 1992-2018 Carlie J. Coats, Jr.,
     (C) 2003-2012 Baron Advanced Meteorological Systems, LLC, and
     (C) 2014-2021 UNC Institute for the Environment.
     Released under the GNU LGPL  License, version 2.1.  See URL

         https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html

     for conditions of use.

     ioapi-3.2: $Id: init3.F90 200 2021-05-10 14:06:20Z coats $
     netCDF version 4.7.4 of Jul 11 2021 01:42:43 $


     ===========================================
     |>---   ENVIRONMENT VARIABLE REPORT   ---<|
     ===========================================

     |> Grid and High-Level Model Parameters:
     +=========================================
      --Env Variable-- | --Value--
      --------------------------------------------------------------------------------
             BLD  |             (default)
          OUTDIR  |  /home/ghajari/Build_CMAQ/CMAQ_Project/data/output_CCTM_v532_gcc_Iran_10km
       NEW_START  |          T
  ISAM_NEW_START  |  Y (default)
       GRID_NAME  |  CMAQ-Iran
       CTM_TSTEP  |       10000
      CTM_RUNLEN  |      430000
    CTM_PROGNAME  |  DRIVER (default)
      CTM_STDATE  |     2020359
      CTM_STTIME  |       60000
     CTM_MAXSYNC  |         300
     CTM_MINSYNC  |          60

     |> Multiprocess control, output and error checking:
     +====================================================
      --Env Variable-- | --Value--
      --------------------------------------------------------------------------------
 PRINT_PROC_TIME  |          T
     FL_ERR_STOP  |          F
       CTM_CKSUM  |          T
AVG_FILE_ENDTIME  |          F
   AVG_CONC_SPCS  |  ALL
       CONC_SPCS  |  ALL
 ACONC_BLEV_ELEV  |   1 1
  CONC_BLEV_ELEV  |   1 1
 IOAPI_LOG_WRITE  |          F
         VERTEXT  |          F
VERTEXT_COORD_PA  |  /home/ghajari/Build_CMAQ/CMAQ_Project/CCTM/scripts/lonlat.csv
   gc_matrix_nml  |  /home/ghajari/Build_CMAQ/CMAQ_Project/CCTM/scripts/BLD_CCTM_v532_gcc/GC_cb6r3_ae7_aq.nml
   ae_matrix_nml  |  /home/ghajari/Build_CMAQ/CMAQ_Project/CCTM/scripts/BLD_CCTM_v532_gcc/AE_cb6r3_ae7_aq.nml
   nr_matrix_nml  |  /home/ghajari/Build_CMAQ/CMAQ_Project/CCTM/scripts/BLD_CCTM_v532_gcc/NR_cb6r3_ae7_aq.nml
   tr_matrix_nml  |  /home/ghajari/Build_CMAQ/CMAQ_Project/CCTM/scripts/BLD_CCTM_v532_gcc/Species_Table_TR_0.nml

     |> Chemistry and Photolysis:
     +=============================
      --Env Variable-- | --Value--
      --------------------------------------------------------------------------------
    CTM_PHOTDIAG  |          F
CORE_SHELL_OPTIC  |          F (default)
 OPTICS_MIE_CALC  |          F (default)
       GEAR_ATOL  |   0.100E-08 (default)
       GEAR_RTOL  |   0.100E-02 (default)
         RB_RTOL  |   0.100E-02 (default)
         RB_ATOL  |   0.100E-06 (default)

     |> Aerosols:
     +=============
      --Env Variable-- | --Value--
      --------------------------------------------------------------------------------
    CTM_AVISDIAG  |          F (default)
  CTM_ZERO_PCSOA  |          F (default)
         CTM_AOD  |          F (default)
      CTM_PMDIAG  |          F
     CTM_APMDIAG  |          T
APMDIAG_BLEV_ELE  |  1 1
 AVG_PMDIAG_SPCS  |             (default)
    STM_SO4TRACK  |          F

     |> Cloud Processes:
     +====================
      --Env Variable-- | --Value--
      --------------------------------------------------------------------------------
        CLD_DIAG  |          F

     |> Air-Surface Exchange Processes:
     +===================================
      --Env Variable-- | --Value--
      --------------------------------------------------------------------------------
      PX_VERSION  |          F
      CTM_ABFLUX  |          F
      CTM_MOSAIC  |          F
    CTM_SFC_HONO  |          T
     CLM_VERSION  |          F
    NOAH_VERSION  |          T
       CTM_STAGE  |          F (default)
   CTM_DEPV_FILE  |          F
      CTM_HGBIDI  |          F
CTM_BIDI_FERT_NH  |          T
 CTM_WBDUST_BELD  |  BELD3

     |> Transport Processes:
     +========================
      --Env Variable-- | --Value--
      --------------------------------------------------------------------------------
CTM_VDIFF_DIAG_F  |          F (default)
  SIGMA_SYNC_TOP  |   0.700E+00
    ADV_HDIV_LIM  |   0.900E+00 (default)
     CTM_ADV_CFL  |   0.950E+00
           KZMIN  |          T
        CTM_WVEL  |          T
   CTM_GRAV_SETL  |          T

     |> Emissions Parameters:
     +=========================
      --Env Variable-- | --Value--
      --------------------------------------------------------------------------------
   EMISSCTRL_NML  |  /home/ghajari/Build_CMAQ/CMAQ_Project/CCTM/scripts/BLD_CCTM_v532_gcc/EmissCtrl_cb6r3_ae7_aq.nml
      CTM_EMLAYS  |           0 (default)
       N_EMIS_GR  |           1
       N_EMIS_TR  |           0 (default)
     CTM_EMISCHK  |          F
    CTM_BIOGEMIS  |          F
       BIOG_SPRO  |  DEFAULT (default)
        BIOSW_YN  |          F (default)
       SUMMER_YN  |          T (default)
      B3GTS_DIAG  |          F
    CTM_MGEMDIAG  |          F (default)
  CTM_OCEAN_CHEM  |          T
     CTM_WB_DUST  |          F
 CTM_DUSTEM_DIAG  |          F
    CTM_SSEMDIAG  |          F
     CTM_LTNG_NO  |          F
   LTNG_ASSIM_DT  |           0 (default)
          LTNGNO  |  InLine (default)
        USE_NLDN  |          F (default)
        LTNGDIAG  |          F
         MOLSNCG  |   0.350E+03 (default)
         MOLSNIC  |   0.350E+03 (default)
       N_EMIS_PT  |           0
          IPVERT  |           0 (default)
        EMISDIAG  |  F
   EMIS_SYM_DATE  |          F (default)

     |> Process Analysis Parameters:
     +================================
      --Env Variable-- | --Value--
      --------------------------------------------------------------------------------
      CTM_PROCAN  |          F
    PA_BCOL_ECOL  |             (default)
    PA_BROW_EROW  |             (default)
    PA_BLEV_ELEV  |             (default)
       MET_TSTEP  |       10000 (default)

     MET data determined based on WRF ARW version 4.1.1


          -=-  MPP Processor-to-Subdomain Map  -=-
                 Number of Processors =   1
    ____________________________________________________
    |                                                  |
    |  PE    #Cols    Col_Range     #Rows    Row_Range |
    |__________________________________________________|
    |                                                  |
    |  0      229      1: 229        187      1: 187   |
    |__________________________________________________|

^C
ghajari@davood-ProLiant-DL380p-Gen8:~/Build_CMAQ/CMAQ_Project/CCTM/scripts$ ./run_cctm_edgar_2.csh
Start Model Run At  ‫شنبه ۳۱ جولای ۲۱، ساعات ۱۶:۲۸:۲۱ (+0430)‬
Compiler is set to gcc

Working Directory is /home/ghajari/Build_CMAQ/CMAQ_Project/CCTM/scripts
Build Directory is /home/ghajari/Build_CMAQ/CMAQ_Project/CCTM/scripts/BLD_CCTM_v532_gcc
Output Directory is /home/ghajari/Build_CMAQ/CMAQ_Project/data/output_CCTM_v532_gcc_Iran_10km
Log Directory is /home/ghajari/Build_CMAQ/CMAQ_Project/data/output_CCTM_v532_gcc_Iran_10km/LOGS
Executable Name is CCTM_v532.exe

---CMAQ EXECUTION ID: CMAQ_CCTMv532_ghajari_20210731_115821_375304495 ---

Set up input and output files for Day 2020-12-24.
CMAQ Processing of Day 20201224 Began at ‫شنبه ۳۱ جولای ۲۱، ساعات ۱۶:۲۸:۲۱ (+0430)‬
        CTM_APPL  |  v532_gcc_Iran_10km_20201224
================================================================================
|                                                                              |
|               The Community Multiscale Air Quality (CMAQ) Model              |
|                                   Version 5.3.2                              |
|                                                                              |
|                          Built and Maintained by the                         |
|                        Office of Research and Development                    |
|                   United States Environmental Protection Agency              |
|                                                                              |
|                            https://www.epa.gov/cmaq                          |
|                                                                              |
|       Source Code:   https://www.github.com/USEPA/cmaq/tree/master           |
|       Documentation: https://www.github.com/USEPA/cmaq/tree/master/DOCS      |
|                                                                              |
|         The CMAQ Model is tested and released with cooperation from          |
|         the Community Modeling and Analysis System (CMAS) Center via         |
|         contract support. CMAS is managed by the Institute for the           |
|         Environment, University of North Carolina at Chapel Hill.            |
|         CMAS URL: (https://www.cmascenter.org)                               |
|                                                                              |
================================================================================

     This program uses the EPA-AREAL/MCNC-EnvPgms/BAMS Models-3
     I/O Applications Programming Interface, [I/O API] which is
     built on top of the netCDF I/O library (Copyright 1993, 1996
     University Corporation for Atmospheric Research/Unidata
     Program) and the PVM parallel-programming library (from
     Oak Ridge National Laboratory).
     Copyright (C) 1992-2002 MCNC,
     (C) 1992-2018 Carlie J. Coats, Jr.,
     (C) 2003-2012 Baron Advanced Meteorological Systems, LLC, and
     (C) 2014-2021 UNC Institute for the Environment.
     Released under the GNU LGPL  License, version 2.1.  See URL

         https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html

     for conditions of use.

     ioapi-3.2: $Id: init3.F90 200 2021-05-10 14:06:20Z coats $
     netCDF version 4.7.4 of Jul 11 2021 01:42:43 $


     ===========================================
     |>---   ENVIRONMENT VARIABLE REPORT   ---<|
     ===========================================

     |> Grid and High-Level Model Parameters:
     +=========================================
      --Env Variable-- | --Value--
      --------------------------------------------------------------------------------
             BLD  |             (default)
          OUTDIR  |  /home/ghajari/Build_CMAQ/CMAQ_Project/data/output_CCTM_v532_gcc_Iran_10km
       NEW_START  |          T
  ISAM_NEW_START  |  Y (default)
       GRID_NAME  |  CMAQ-Iran
       CTM_TSTEP  |       10000
      CTM_RUNLEN  |      430000
    CTM_PROGNAME  |  DRIVER (default)
      CTM_STDATE  |     2020359
      CTM_STTIME  |       60000
     CTM_MAXSYNC  |         300
     CTM_MINSYNC  |          60

     |> Multiprocess control, output and error checking:
     +====================================================
      --Env Variable-- | --Value--
      --------------------------------------------------------------------------------
 PRINT_PROC_TIME  |          T
     FL_ERR_STOP  |          F
       CTM_CKSUM  |          T
AVG_FILE_ENDTIME  |          F
   AVG_CONC_SPCS  |  ALL
       CONC_SPCS  |  ALL
 ACONC_BLEV_ELEV  |   1 1
  CONC_BLEV_ELEV  |   1 1
 IOAPI_LOG_WRITE  |          F
         VERTEXT  |          F
VERTEXT_COORD_PA  |  /home/ghajari/Build_CMAQ/CMAQ_Project/CCTM/scripts/lonlat.csv
   gc_matrix_nml  |  /home/ghajari/Build_CMAQ/CMAQ_Project/CCTM/scripts/BLD_CCTM_v532_gcc/GC_cb6r3_ae7_aq.nml
   ae_matrix_nml  |  /home/ghajari/Build_CMAQ/CMAQ_Project/CCTM/scripts/BLD_CCTM_v532_gcc/AE_cb6r3_ae7_aq.nml
   nr_matrix_nml  |  /home/ghajari/Build_CMAQ/CMAQ_Project/CCTM/scripts/BLD_CCTM_v532_gcc/NR_cb6r3_ae7_aq.nml
   tr_matrix_nml  |  /home/ghajari/Build_CMAQ/CMAQ_Project/CCTM/scripts/BLD_CCTM_v532_gcc/Species_Table_TR_0.nml

     |> Chemistry and Photolysis:
     +=============================
      --Env Variable-- | --Value--
      --------------------------------------------------------------------------------
    CTM_PHOTDIAG  |          F
CORE_SHELL_OPTIC  |          F (default)
 OPTICS_MIE_CALC  |          F (default)
       GEAR_ATOL  |   0.100E-08 (default)
       GEAR_RTOL  |   0.100E-02 (default)
         RB_RTOL  |   0.100E-02 (default)
         RB_ATOL  |   0.100E-06 (default)

     |> Aerosols:
     +=============
      --Env Variable-- | --Value--
      --------------------------------------------------------------------------------
    CTM_AVISDIAG  |          F (default)
  CTM_ZERO_PCSOA  |          F (default)
         CTM_AOD  |          F (default)
      CTM_PMDIAG  |          F
     CTM_APMDIAG  |          T
APMDIAG_BLEV_ELE  |  1 1
 AVG_PMDIAG_SPCS  |             (default)
    STM_SO4TRACK  |          F

     |> Cloud Processes:
     +====================
      --Env Variable-- | --Value--
      --------------------------------------------------------------------------------
        CLD_DIAG  |          F

     |> Air-Surface Exchange Processes:
     +===================================
      --Env Variable-- | --Value--
      --------------------------------------------------------------------------------
      PX_VERSION  |          F
      CTM_ABFLUX  |          F
      CTM_MOSAIC  |          F
    CTM_SFC_HONO  |          T
     CLM_VERSION  |          F
    NOAH_VERSION  |          T
       CTM_STAGE  |          F (default)
   CTM_DEPV_FILE  |          F
      CTM_HGBIDI  |          F
CTM_BIDI_FERT_NH  |          T
 CTM_WBDUST_BELD  |  BELD3

     |> Transport Processes:
     +========================
      --Env Variable-- | --Value--
      --------------------------------------------------------------------------------
CTM_VDIFF_DIAG_F  |          F (default)
  SIGMA_SYNC_TOP  |   0.700E+00
    ADV_HDIV_LIM  |   0.900E+00 (default)
     CTM_ADV_CFL  |   0.950E+00
           KZMIN  |          T
        CTM_WVEL  |          T
   CTM_GRAV_SETL  |          T

     |> Emissions Parameters:
     +=========================
      --Env Variable-- | --Value--
      --------------------------------------------------------------------------------
   EMISSCTRL_NML  |  /home/ghajari/Build_CMAQ/CMAQ_Project/CCTM/scripts/BLD_CCTM_v532_gcc/EmissCtrl_cb6r3_ae7_aq.nml
      CTM_EMLAYS  |           0 (default)
       N_EMIS_GR  |           1
       N_EMIS_TR  |           0 (default)
     CTM_EMISCHK  |          F
    CTM_BIOGEMIS  |          F
       BIOG_SPRO  |  DEFAULT (default)
        BIOSW_YN  |          F (default)
       SUMMER_YN  |          T (default)
      B3GTS_DIAG  |          F
    CTM_MGEMDIAG  |          F (default)
  CTM_OCEAN_CHEM  |          T
     CTM_WB_DUST  |          F
 CTM_DUSTEM_DIAG  |          F
    CTM_SSEMDIAG  |          F
     CTM_LTNG_NO  |          F
   LTNG_ASSIM_DT  |           0 (default)
          LTNGNO  |  InLine (default)
        USE_NLDN  |          F (default)
        LTNGDIAG  |          F
         MOLSNCG  |   0.350E+03 (default)
         MOLSNIC  |   0.350E+03 (default)
       N_EMIS_PT  |           0
          IPVERT  |           0 (default)
        EMISDIAG  |  F
   EMIS_SYM_DATE  |          F (default)

     |> Process Analysis Parameters:
     +================================
      --Env Variable-- | --Value--
      --------------------------------------------------------------------------------
      CTM_PROCAN  |          F
    PA_BCOL_ECOL  |             (default)
    PA_BROW_EROW  |             (default)
    PA_BLEV_ELEV  |             (default)
       MET_TSTEP  |       10000 (default)

     MET data determined based on WRF ARW version 4.1.1


          -=-  MPP Processor-to-Subdomain Map  -=-
                 Number of Processors =   1
    ____________________________________________________
    |                                                  |
    |  PE    #Cols    Col_Range     #Rows    Row_Range |
    |__________________________________________________|
    |                                                  |
    |  0      229      1: 229        187      1: 187   |
    |__________________________________________________|



     ================================
     |>---   TIME INTEGRATION   ---<|
     ================================

     Processing Day/Time [YYYYDDD:HHMMSS]: 2020359:060000
       Which is Equivalent to (UTC): 6:00:00  Thursday,  Dec. 24, 2020
       Time-Step Length (HHMMSS): 000500
                 VDIFF completed...   37.3 seconds
                COUPLE completed...    1.0 seconds
                  HADV completed...   20.9 seconds
                  ZADV completed...   17.0 seconds
       NaN or Infinity detected on processor 000
       During        HDIFF :  Gas        NaN  | Aer        NaN  | Non        NaN
Command exited with non-zero status 1
real 86.30
user 81.49
sys 4.79

**************************************************************
** Runscript Detected an Error: CGRID file was not written. **
**   This indicates that CMAQ was interrupted or an issue   **
**   exists with writing output. The runscript will now     **
**   abort rather than proceeding to subsequent days.       **
**************************************************************
==================================
  ***** CMAQ TIMING REPORT *****
==================================
Start Day: 2020-12-24
End Day:   2020-12-26
Number of Simulation Days: 1
Domain Name:               CMAQ-Iran
Number of Grid Cells:      1284690  (ROW x COL x LAY)
Number of Layers:          30
Number of Processes:       1
   All times are in seconds.
Num  Day        Wall Time
01   2020-12-24   86.30
     Total Time = 86.30
      Avg. Time = 86.30

I am attaching the log file in which it indicates error as:
*** ERROR ABORT in subroutine CKSUMMER
NaN or Infinity detected in HDIFF

Is there anything that could be done to overcome this?
Thanks in advance.
Regards,
Amir
CTM_LOG_000.v532_gcc_Iran_10km_20201224.txt (84.5 KB)

I set the CTM_CKSUM to N and now the model is working, however I don’t know how it affects the results!!
I must add that the model works with CTM_CKSUM set to Y on a different system.
I am just puzzled :thinking:

Setting CTM_CKSUM to N just means that the model does not detect the error and abort. It does not mean the model is necessarily working correctly.

You could recompile the model in debug mode and rerun. The model should still crash, but provide a stack trace that gives where the error occurs.