MCIP Nested Domains

lawless · November 24, 2020, 6:01pm

Hello,

I am trying to set up a CMAQ run. I am currently on the PREP step trying to process my WRF output files. I set up a nested domain in WRF with 2 domains.

I use the following command ./run_mcip.csh gcc |& tee mcip.log but only get

Compiler is set to gcc
Tue Nov 24 10:58:04 MST 2020
if: Expression Syntax.

Attached it my runscript. run_mcip.csh (18.9 KB)

Also is there a tutorial on how to run CMAQ with nested domains? I havent really found anything although I am sure I just missed something.

Thanks,
Zach

lizadams · November 25, 2020, 8:01pm

Hi Zach,

To prepare MCIP inputs for a nested domain, it is my understanding that you would run the run_mcip.csh script twice, once using the inputs from wrf for the outer domain, and second time using the inputs from wrf for the inner domain.

The if: Expression Syntax error you are seeing is due to a typo.

I found this by using the sdiff command to display a side by side difference between your run script and the one provided. sdiff shows any differences between the two files by placing a pipe or | in the middle between the two files, where there is a line that contains differences.

sdiff run_mcip.lawless.csh run_mcip.csh

Example output with the problematic if statement.

#------------------------------------------------------------	#------------------------------------------------------------
# Make sure directories exist.					# Make sure directories exist.
#------------------------------------------------------------	#------------------------------------------------------------

if ( ! -d $InMetDir ) then					if ( ! -d $InMetDir ) then
  echo "No such input directory $InMetDir"			  echo "No such input directory $InMetDir"
  exit 1If						      |	  exit 1
endif								endif

Fix line number 282,
change

exit 1If

to

exit 1

lawless · December 4, 2020, 4:22pm

Thanks for finding that! Totally works now.

I was able to successfully run the outer and inner domains for all the PREP scripts but now I am confused as how to set up the nested domain in the CCTM runscripts. It doesn’t look there is a place in the run scripts to incorporate both?

Zach

lizadams · December 4, 2020, 5:18pm

For CMAQ, you would run the outer domain first, and then re-run BCON to use the outer domain CMAQ output as input to generate the Boundary Conditions for the inner domain.

lawless · December 21, 2020, 4:18pm

Liz,

I am trying to do the ICON and BCON for the fine domain using the coarse domain data but I keep running into problems with ICON and BCON. I get the following outputs

BCON:
~/53_CMAQ ~/53_CMAQ/PREP/bcon/scripts
Compiler is set to gcc
~/53_CMAQ/PREP/bcon/scripts

Input data path, CMAQ_DATA set to /home/pkasper/53_CMAQ/data

model BCON_v531.exe;

repo /home/pkasper/CMAQ_REPO/PREP/bcon/src;

lib_base /home/pkasper/53_CMAQ/lib/x86_64/gcc;

lib_1 ioapi/lib;

lib_2 ioapi/include_files;

lib_4 ioapi/lib;

cpp_flags “”;

f_compiler mpifort;

fstd “-O3 -funroll-loops -finit-character=32 -Wtabs -Wsurprising -march=native -ftree-vectorize -ftree-loop-if-convert -finline-limit=512”;

dbg “-Wall -O0 -g -fcheck=all -ffpe-trap=invalid,zero,overflow -fbacktrace”;

f_flags “-ffixed-form -ffixed-line-length-132 -funroll-loops -finit-character=32”;

f90_flags “-ffree-form -ffree-line-length-none -funroll-loops -finit-character=32”;

link_flags “-fopenmp”;

ioapi “-lioapi”;

netcdf "-lnetcdff -lnetcdf ";

netcdff “-lnetcdff”;

// project repository location: /home/pkasper/CMAQ_REPO/PREP/bcon/src

// required common
Module common;

// options are m3conc
Module m3conc;

// options are profile
Module profile;

setenv GRID_NAME April_2km_Nest_2
setenv GRIDDESC /home/pkasper/53_CMAQ/data/mcip/April_2km_Nest_2/GRIDDESC
setenv IOAPI_ISPH 20
setenv IOAPI_LOG_WRITE T
setenv IOAPI_OFFSET_64 YES
setenv EXECUTION_ID BCON_v531.exe
setenv BCON_TYPE echo $BCTYPE | tr "[A-Z]" "[a-z]"
tr [A-Z] [a-z]
echo regrid
set OUTDIR = /home/pkasper/53_CMAQ/data/bcon
set DATE = 2017-03-10
set YYYYJJJ = date -ud "${DATE}" +%Y%j
date -ud 2017-03-10 +%Y%j
set YYMMDD = date -ud "${DATE}" +%y%m%d
date -ud 2017-03-10 +%y%m%d
set YYYYMMDD = date -ud "${DATE}" +%Y%m%d
date -ud 2017-03-10 +%Y%m%d
if ( regrid == regrid ) then
setenv CTM_CONC_1 /home/pkasper/53_CMAQ/data/CMAQ_Output/output_CCTM_v531_gcc_April_2km_Nest_1/CCTM_CONC_v531_gcc_April_2km_Nest_1_20170310.nc
setenv MET_CRO_3D_CRS /home/pkasper/53_CMAQ/data/mcip/April_2km_Nest_1/METCRO3D_180613.nc
setenv MET_BDY_3D_FIN /home/pkasper/53_CMAQ/data/mcip/April_2km_Nest_2/METCRO3D_180613.nc
setenv BNDY_CONC_1 /home/pkasper/53_CMAQ/data/bcon/BCON_v531_180613_regrid_20170310 -v
endif
if ( regrid == profile ) then
if ( ! -d /home/pkasper/53_CMAQ/data/bcon ) mkdir -p /home/pkasper/53_CMAQ/data/bcon
ls -l /home/pkasper/53_CMAQ/PREP/bcon/scripts/BLD_BCON_v531_gcc/BCON_v531.exe
-rwxrwxr-x 1 pkasper pkasper 17047936 Mar 12 2020 /home/pkasper/53_CMAQ/PREP/bcon/scripts/BLD_BCON_v531_gcc/BCON_v531.exe
size /home/pkasper/53_CMAQ/PREP/bcon/scripts/BLD_BCON_v531_gcc/BCON_v531.exe
text data bss dec hex filename
1663077 14907216 19049888 35620181 21f8555 /home/pkasper/53_CMAQ/PREP/bcon/scripts/BLD_BCON_v531_gcc/BCON_v531.exe
unlimit
limit
cputime unlimited
filesize unlimited
datasize unlimited
stacksize unlimited
coredumpsize unlimited
memoryuse unlimited
vmemoryuse unlimited
descriptors 1048576
memorylocked 64 kbytes
maxproc 379890
maxlocks unlimited
maxsignal 379890
maxmessage 819200
maxnice 0
maxrtprio 0
maxrttime unlimited

/home/pkasper/53_CMAQ/PREP/bcon/scripts/BLD_BCON_v531_gcc/BCON_v531.exe

 This program uses the EPA-AREAL/MCNC-EnvPgms/BAMS Models-3      
 I/O Applications Programming Interface, [I/O API] which is      
 built on top of the netCDF I/O library (Copyright 1993, 1996    
 University Corporation for Atmospheric Research/Unidata         
 Program) and the PVM parallel-programming library (from         
 Oak Ridge National Laboratory).                                 
 Copyright (C) 1992-2002 MCNC,                                   
 (C) 1992-2018 Carlie J. Coats, Jr.,                             
 (C) 2003-2012 Baron Advanced Meteorological Systems, LLC, and   
 (C) 2014-2019 UNC Institute for the Environment.                
 Released under the GNU LGPL  License, version 2.1.  See URL     
                                                                 
     https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html     
                                                                 
 for conditions of use.                                          
                                                                 
 ioapi-3.2: $Id: init3.F90 136 2019-10-16 13:57:49Z coats $
 Version with PARMS3.EXT/PARAMETER::MXVARS3= 2048
 netCDF version 4.7.0 of Oct 30 2019 12:37:17 $
  
  
 EXECUTION_ID: BCON_v531.exe
 Value for BCON_TYPE:  'regrid'
 Value for BCON_TYPE:  'regrid'
 Value for GRID_NAME:  'April_2km_Nest_2'
 Value for GRID_NAME:  'April_2km_Nest_2'

 File "GRIDDESC" opened for input on unit:  99
 /home/pkasper/53_CMAQ/data/mcip/April_2km_Nest_2/GRIDDESC

 Value for IOAPI_CHECK_HEADERS not defined;returning default:   FALSE
  
 "MET_BDY_3D_FIN" opened as OLD:READ-ONLY   
 File name "/home/pkasper/53_CMAQ/data/mcip/April_2km_Nest_2/METCRO3D_180613.nc"
 File type GRDDED3 
 Execution ID "mcip"
 Grid name "April_2km_Nest_2"
 Dimensions: 188 rows, 108 cols, 32 lays, 18 vbles
 NetCDF ID:     65536  opened as READONLY            
 Starting date and time  2017069:120000 (12:00:00  March 10, 2017)
 Timestep                          010000 (1:00:00 hh:mm:ss)
 Maximum current record number      1459

###############################################################################
 #  Input section 
 ###############################################################################

 Boundary Concentrations from a CMAQ CTM concentration file.
 Value for SDATE not defined; returning default:  0
 Value for STIME not defined; returning default:  0
 Value for RUNLEN not defined; returning default:  0


 Input Models3 CTM file names: 
 Value for CTM_CONC_1:  '/home/pkasper/53_CMAQ/data/CMAQ_Output/output_CCTM_v531_gcc_April_2km_Nest_1/CCTM_CONC_v531_gcc_April_2km_Nest_1_20170310.nc'
 Value for CTM_CONC_1:  '/home/pkasper/53_CMAQ/data/CMAQ_Output/output_CCTM_v531_gcc_April_2km_Nest_1/CCTM_CONC_v531_gcc_April_2km_Nest_1_20170310.nc'
 Value for CTM_CONC_2 not defined; returning defaultval ':  '                '
 Value for CTM_CONC_2 not defined; returning defaultval ':  '                '
 Value for CTM_CONC_3 not defined; returning defaultval ':  '                '
 Value for CTM_CONC_3 not defined; returning defaultval ':  '                '
  
 "CTM_CONC_1" opened as OLD:READ-ONLY   
 File name "/home/pkasper/53_CMAQ/data/CMAQ_Output/output_CCTM_v531_gcc_April_2km_Nest_1/CCTM_CONC_v531_gcc_April_2km_Nest_1_20170310.nc"
 File type GRDDED3 
 Execution ID "CMAQ_CCTMv531_pkasper_20201215_220558_272598291"
 Grid name "April_2km_Nest_1"
 Dimensions: 103 rows, 103 cols, 1 lays, 13 vbles
 NetCDF ID:    131072  opened as READONLY            
 Starting date and time  2017069:120000 (12:00:00  March 10, 2017)
 Timestep                          010000 (1:00:00 hh:mm:ss)
 Maximum current record number        25
 Input SDATE equals zero; SDATE reset to MET_BDY_3D_FIN file start date
 Input STIME equals zero; STIME reset to MET_BDY_3D_FIN file start time

 Input RUNLEN not set or equal to zero.                                          
 Resetting RUNLEN to correspond to MET_BDY_3D_FIN file ending date & time.       

 NCOLS_IN:          103
 NROWS_IN:          103
 NLAYS_IN:            1
 NSPCS_IN:           13
 NBNDY:             596
 NLAYS:              32

 Value for IOAPI_ISPH:  '20'
 Value for IOAPI_ISPH:  '20'
 INITSPHERES:  input sphere Normal Sphere (MM5 & WRF-ARW) R=6370000

###############################################################################

Vertical Interpolation Section

###############################################################################

 The COORD.EXT and CTM vertical grid types are the same, but the resolution is different.
 Vertical interpolation using VGLVS (listed below).

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0 0x2B7DBAC2FE08
#1 0x2B7DBAC2EF90
#2 0x2B7DBB6BC4BF
#3 0x2B7DBB7D50C0
#4 0x2B7DBAD01354
#5 0x4148F4 in m3_vinterp_
#6 0x409CAA in m3_bcout_
#7 0x4130EF in m3_inbndy_
#8 0x4041BC in MAIN__ at bcon.F:?
Segmentation fault (core dumped)
0.044u 0.024s 0:00.20 30.0% 0+0k 32320+0io 18pf+0w
exit ( )

ICON:
~/53_CMAQ ~/53_CMAQ/PREP/icon/scripts
Compiler is set to gcc
~/53_CMAQ/PREP/icon/scripts

Input data path, CMAQ_DATA set to /home/pkasper/53_CMAQ/data

model ICON_v531.exe;

repo /home/pkasper/CMAQ_REPO/PREP/icon/src;

lib_base /home/pkasper/53_CMAQ/lib/x86_64/gcc;

lib_1 ioapi/lib;

lib_2 ioapi/include_files;

lib_4 ioapi/lib;

cpp_flags “”;

f_compiler mpifort;

fstd “-O3 -funroll-loops -finit-character=32 -Wtabs -Wsurprising -march=native -ftree-vectorize -ftree-loop-if-convert -finline-limit=512”;

dbg “-Wall -O0 -g -fcheck=all -ffpe-trap=invalid,zero,overflow -fbacktrace”;

f_flags “-ffixed-form -ffixed-line-length-132 -funroll-loops -finit-character=32”;

f90_flags “-ffree-form -ffree-line-length-none -funroll-loops -finit-character=32”;

link_flags “-fopenmp”;

ioapi “-lioapi”;

netcdf "-lnetcdff -lnetcdf ";

netcdff “-lnetcdff”;

// project repository location: /home/pkasper/CMAQ_REPO/PREP/icon/src

// required common
Module common;

// options are m3conc
Module m3conc;

// options are profile
Module profile;

setenv GRID_NAME April_2km_Nest_2
setenv GRIDDESC /home/pkasper/53_CMAQ/data/mcip/April_2km_Nest_2/GRIDDESC
setenv IOAPI_ISPH 20
setenv IOAPI_LOG_WRITE T
setenv IOAPI_OFFSET_64 YES
setenv EXECUTION_ID ICON_v531.exe
setenv ICON_TYPE echo $ICTYPE | tr "[A-Z]" "[a-z]"
tr [A-Z] [a-z]
echo regrid
set OUTDIR = /home/pkasper/53_CMAQ/data/icon
set DATE = 2017-03-10
set YYYYJJJ = date -ud "${DATE}" +%Y%j
date -ud 2017-03-10 +%Y%j
set YYMMDD = date -ud "${DATE}" +%y%m%d
date -ud 2017-03-10 +%y%m%d
set YYYYMMDD = date -ud "${DATE}" +%Y%m%d
date -ud 2017-03-10 +%Y%m%d
if ( regrid == regrid ) then
setenv CTM_CONC_1 /home/pkasper/53_CMAQ/data/CMAQ_Output/output_CCTM_v531_gcc_April_2km_Nest_1/CCTM_CONC_v531_gcc_April_2km_Nest_1_20170310.nc
setenv MET_CRO_3D_CRS /home/pkasper/53_CMAQ/data/mcip/April_2km_Nest_1/METCRO3D_180613.nc
setenv MET_CRO_3D_FIN /home/pkasper/53_CMAQ/data/mcip/April_2km_Nest_2/METCRO3D_180613.nc
setenv INIT_CONC_1 /home/pkasper/53_CMAQ/data/icon/ICON_v531_180613_regrid_20170310 -v
endif
if ( regrid == profile ) then
if ( ! -d /home/pkasper/53_CMAQ/data/icon ) mkdir -p /home/pkasper/53_CMAQ/data/icon
ls -l /home/pkasper/53_CMAQ/PREP/icon/scripts/BLD_ICON_v531_gcc/ICON_v531.exe
-rwxrwxr-x 1 pkasper pkasper 17043712 Mar 12 2020 /home/pkasper/53_CMAQ/PREP/icon/scripts/BLD_ICON_v531_gcc/ICON_v531.exe
size /home/pkasper/53_CMAQ/PREP/icon/scripts/BLD_ICON_v531_gcc/ICON_v531.exe
text data bss dec hex filename
1661373 14907152 19049888 35618413 21f7e6d /home/pkasper/53_CMAQ/PREP/icon/scripts/BLD_ICON_v531_gcc/ICON_v531.exe
unlimit
limit
cputime unlimited
filesize unlimited
datasize unlimited
stacksize unlimited
coredumpsize unlimited
memoryuse unlimited
vmemoryuse unlimited
descriptors 1048576
memorylocked 64 kbytes
maxproc 379890
maxlocks unlimited
maxsignal 379890
maxmessage 819200
maxnice 0
maxrtprio 0
maxrttime unlimited
/home/pkasper/53_CMAQ/PREP/icon/scripts/BLD_ICON_v531_gcc/ICON_v531.exe

 This program uses the EPA-AREAL/MCNC-EnvPgms/BAMS Models-3      
 I/O Applications Programming Interface, [I/O API] which is      
 built on top of the netCDF I/O library (Copyright 1993, 1996    
 University Corporation for Atmospheric Research/Unidata         
 Program) and the PVM parallel-programming library (from         
 Oak Ridge National Laboratory).                                 
 Copyright (C) 1992-2002 MCNC,                                   
 (C) 1992-2018 Carlie J. Coats, Jr.,                             
 (C) 2003-2012 Baron Advanced Meteorological Systems, LLC, and   
 (C) 2014-2019 UNC Institute for the Environment.                
 Released under the GNU LGPL  License, version 2.1.  See URL     
                                                                 
     https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html     
                                                                 
 for conditions of use.                                          
                                                                 
 ioapi-3.2: $Id: init3.F90 136 2019-10-16 13:57:49Z coats $
 Version with PARMS3.EXT/PARAMETER::MXVARS3= 2048
 netCDF version 4.7.0 of Oct 30 2019 12:37:17 $
  
  
 EXECUTION_ID: ICON_v531.exe
 Value for ICON_TYPE:  'regrid'
 Value for ICON_TYPE:  'regrid'
 Value for GRID_NAME:  'April_2km_Nest_2'
 Value for GRID_NAME:  'April_2km_Nest_2'

 File "GRIDDESC" opened for input on unit:  99
 /home/pkasper/53_CMAQ/data/mcip/April_2km_Nest_2/GRIDDESC

 Value for IOAPI_CHECK_HEADERS not defined;returning default:   FALSE
  
 "MET_CRO_3D_FIN" opened as OLD:READ-ONLY   
 File name "/home/pkasper/53_CMAQ/data/mcip/April_2km_Nest_2/METCRO3D_180613.nc"
 File type GRDDED3 
 Execution ID "mcip"
 Grid name "April_2km_Nest_2"
 Dimensions: 188 rows, 108 cols, 32 lays, 18 vbles
 NetCDF ID:     65536  opened as READONLY            
 Starting date and time  2017069:120000 (12:00:00  March 10, 2017)
 Timestep                          010000 (1:00:00 hh:mm:ss)
 Maximum current record number      1459

###############################################################################

Input file section

###############################################################################

 Initial Concentrations from a CMAQ CTM concentration file.
 Value for SDATE not defined; returning default:  0
 Value for STIME not defined; returning default:  0


 Input CMAQ CTM file names: 
 Value for CTM_CONC_1:  '/home/pkasper/53_CMAQ/data/CMAQ_Output/output_CCTM_v531_gcc_April_2km_Nest_1/CCTM_CONC_v531_gcc_April_2km_Nest_1_20170310.nc'
 Value for CTM_CONC_1:  '/home/pkasper/53_CMAQ/data/CMAQ_Output/output_CCTM_v531_gcc_April_2km_Nest_1/CCTM_CONC_v531_gcc_April_2km_Nest_1_20170310.nc'
 Value for CTM_CONC_2 not defined; returning defaultval ':  '                '
 Value for CTM_CONC_2 not defined; returning defaultval ':  '                '
 Value for CTM_CONC_3 not defined; returning defaultval ':  '                '
 Value for CTM_CONC_3 not defined; returning defaultval ':  '                '
  
 "CTM_CONC_1" opened as OLD:READ-ONLY   
 File name "/home/pkasper/53_CMAQ/data/CMAQ_Output/output_CCTM_v531_gcc_April_2km_Nest_1/CCTM_CONC_v531_gcc_April_2km_Nest_1_20170310.nc"
 File type GRDDED3 
 Execution ID "CMAQ_CCTMv531_pkasper_20201215_220558_272598291"
 Grid name "April_2km_Nest_1"
 Dimensions: 103 rows, 103 cols, 1 lays, 13 vbles
 NetCDF ID:    131072  opened as READONLY            
 Starting date and time  2017069:120000 (12:00:00  March 10, 2017)
 Timestep                          010000 (1:00:00 hh:mm:ss)
 Maximum current record number        25
 Input SDATE equals zero; SDATE reset to MET_CRO_3D_FIN file start date:  March 10, 2017
 Input STIME equals zero; STIME reset to MET_CRO_3D_FIN file start time:  12:00:00

 NCOLS_IN:          103
 NROWS_IN:          103
 NLAYS_IN:            1
 NSPCS_IN:           13
 NLAYS:              32

 Value for IOAPI_ISPH:  '20'
 Value for IOAPI_ISPH:  '20'
 INITSPHERES:  input sphere Normal Sphere (MM5 & WRF-ARW) R=6370000

###############################################################################

Output File Section

###############################################################################
Value for EXECUTION_ID: ‘ICON_v531.exe’
Value for EXECUTION_ID: ‘ICON_v531.exe’

 Total number of ouput files created: 1
 Value for INIT_CONC_1:  '/home/pkasper/53_CMAQ/data/icon/ICON_v531_180613_regrid_20170310 -v'
 Value for INIT_CONC_1:  '/home/pkasper/53_CMAQ/data/icon/ICON_v531_180613_regrid_20170310 -v'


 IO/API Header data for IC file 1: INIT_CONC_1     

      EXEC_ID: ICON_v531.exe                                                                   
      FTYPE =    1
      SDATE = 2017069
      STIME =  120000
      TSTEP =   10000
      NCOLS =  108
      NROWS =  188
      NLAYS =   32
      NTHIK =    1
      NVARS =   13
      GDTYP =    2
      P_ALP =    3.00000E+01
      P_BET =    6.00000E+01
      P_GAM =   -9.80000E+01
      XCENT =   -9.80000E+01
      YCENT =    4.10000E+01
      XORIG =   -1.31150E+06
      YORIG =   -1.37000E+05
      XCELL =    2.00000E+03
      YCELL =    2.00000E+03
      VGTYP = ****
      VGTOP =    5.00000E+03
      GDNAM = April_2km_Nest_2

      VGLVS =    1.00000E+00   9.93815E-01   9.85951E-01   9.76014E-01   9.63558E-01
                 9.48093E-01   9.29124E-01   9.06191E-01   8.78942E-01   8.47208E-01
                 8.11078E-01   7.70949E-01   7.27525E-01   6.81755E-01   6.34504E-01
                 5.86030E-01   5.36650E-01   4.86730E-01   4.36688E-01   3.86988E-01
                 3.38128E-01   2.90630E-01   2.45023E-01   2.01822E-01   1.61506E-01
                 1.27054E-01   9.81442E-02   7.38860E-02   5.35306E-02   3.64503E-02
                 2.21179E-02   1.00915E-02   0.00000E+00


      Output Variables:
      Name               Type  Units             Description
      O3                   5   ppmV              Instantaneous Molar Mixing Ratio O3
      NO                   5   ppmV              Instantaneous Molar Mixing Ratio NO
      NO2                  5   ppmV              Instantaneous Molar Mixing Ratio NO2
      FORM                 5   ppmV              Instantaneous Molar Mixing Ratio FORM
      ISOP                 5   ppmV              Instantaneous Molar Mixing Ratio ISOP
      ANO3I                5   ug m-3            Instantaneous Aerosol Concentration of ANO3I
      ANO3J                5   ug m-3            Instantaneous Aerosol Concentration of ANO3J
      ANH4I                5   ug m-3            Instantaneous Aerosol Concentration of ANH4I
      ANH4J                5   ug m-3            Instantaneous Aerosol Concentration of ANH4J
      ASO4I                5   ug m-3            Instantaneous Aerosol Concentration of ASO4I
      ASO4J                5   ug m-3            Instantaneous Aerosol Concentration of ASO4J
      NH3                  5   ppmV              Instantaneous molar mixing ratio of NH3
      W_VEL                5   m s-1             Derived vertical velocity component
 
 Value for IOAPI_CHECK_HEADERS not defined;returning default:   FALSE
  
 "INIT_CONC_1" opened as UNKNOWN(R-W)    
 File name "/home/pkasper/53_CMAQ/data/icon/ICON_v531_180613_regrid_20170310"
 File type GRDDED3 
 Execution ID "ICON_v531.exe"
 Grid name "April_2km_Nest_2"
 Dimensions: 188 rows, 108 cols, 32 lays, 13 vbles
 NetCDF ID:    196608  opened as VOLATILE READWRITE  
 Starting date and time  2017069:120000 (12:00:00  March 10, 2017)
 Timestep                          010000 (1:00:00 hh:mm:ss)
 Maximum current record number         0

###############################################################################

Vertical Interpolation Section

###############################################################################

 The Layer Defn and CTM vertical grid types are the same, but the resolution is different.
 Vertical interpolation using VGLVS (listed below).

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0 0x2B1247FA8E08
#1 0x2B1247FA7F90
#2 0x2B1248A354BF
#3 0x2B1248B4E0C0
#4 0x2B124807A354
#5 0x416733 in m3_vinterp_
#6 0x411B46 in m3_icout_
#7 0x40EF5F in m3_inic_
#8 0x4042EC in MAIN__ at icon.F:?
Segmentation fault (core dumped)
0.284u 0.012s 0:00.41 70.7% 0+0k 0+32io 0pf+0w
exit ( )

I just did the coarse domain using the icon and bcon exe so I am surprised I am getting this problem. The only thing I have changed is the file names for the profile section.

lawless · January 12, 2021, 4:18pm

@lizadams Any ideas of how I could fix this? I am kind of stuck on my research until I am able to get this going.

cjcoats · January 12, 2021, 4:39pm

As a first step, let’s find out exactly where the seg-fault is happening: compile the executable for traceback and checking and re-run to get the file and line number at which the problem is happening; in the Makefile, replace the optimization (e.g. -O3) by the following:

For ifort, -g -traceback -check bounds -check uninit -debug-parameters all
For gfortran: -g -fbacktrace -fbounds-check
For pgf95: -g -traceback -Mbounds

then do make clean; make and re-run.

lawless · January 12, 2021, 6:46pm

Thanks for helping! Here is the output after I changed the makefile. I am running gfortran btw

icon_output.txt (11.6 KB)

Edit: Uploaded wrong output file

lizadams · January 12, 2021, 7:48pm

Hi,
The problem seems to reveal itself in the header data for the INIT_CONC_1 file

/home/pkasper/53_CMAQ/data/icon/ICON_v531_180613_regrid_20170310
Which contains the following header information for VGTYP:
VGTYP = ****

This causes an error at line 259 of file m3_vinterp.F
Fortran runtime error: Index ‘-9999’ of dimension 1 of array ‘vgdesc’ below lower bound of 1
Line 259 of m3_vinterp.F is:
m3_vinterp.F: WRITE( LOGUNIT, 92060 ) VGDESC( VGTYP_GD )

I am not sure why the VGTYP is getting clobbed. Can you check the value of VGTYP in your input CCTM_CONC file for the coarse domain, and also check that you are using the GRIDDESC file for the course domain in the run_icon.csh.

lawless · January 12, 2021, 8:31pm

Doubled checked my GRIDDSEC file and it is now for the coarse domain. I had changed it around to try anything to get ICON to work haha. New output from ICON still gives the error

At line 259 of file m3_vinterp.F
Fortran runtime error: Index ‘-9999’ of dimension 1 of array ‘vgdesc’ below lower bound of 1
0.940u 0.016s 0:00.95 100.0% 0+0k 0+32io 0pf+0w
exit ( )

The header for the CCTM_CONC file gives VGTYP as = -9999. I am using the first day of the simulated run, is that a problem or do I need to start it on the second day?

CCTM_CONC_Header.txt (8.8 KB)

lizadams · January 12, 2021, 8:33pm

Hi,
Are you removing the previously generated
ICON file before rerunning?

lawless · January 12, 2021, 8:35pm

Yes, just ran it again after deleting the file to make sure and it was the same result

cjcoats · January 12, 2021, 8:38pm

I am wondering about this CCTM_CONC file: looking at the header, I see

dimensions:
	TSTEP = UNLIMITED ; // (25 currently)
	DATE-TIME = 2 ;
	LAY = 1 ;
	VAR = 13 ;
	ROW = 103 ;
	COL = 103 ;

but a CCTM_CONC file should not be a single-layer file. “Missing/not-applicable” is in fact a proper vertical coordinate type for a file that does not have any vertical layering.

Perhaps you’ve got an ACONC file?

lawless · January 12, 2021, 8:40pm

Definitely makes sense but I just double checked and it was a CCTM_CONC file.

lawless · January 12, 2021, 8:45pm

Just double checked my runscript and noticed something that might be the problem. I had based it off the benchmark script “run_cctm_Bench_2016_12SE1.csh” that has

setenv CONC_BLEV_ELEV " 1 1" #> CONC file layer range; comment to write all layers to CONC

Could this be the reason it only has one layer?

lizadams · January 12, 2021, 8:56pm

The header for the CCTM_CONC file gives VGTYP as = -9999.

This is likely the problem, but I am not sure what step in the course domain run would cause that value for VGTYP.

The following information is available in the VGRD_DEFN.F file:

C The definitions and declarations in this F90 Module follow those
C given in the I/O-API include file FDESC3C.EXT and replace similar
C definitions. This F90 Module is compatible with FDESC3C.EXT.

C VGTYP_GD:
C The vertical grid type:
C 1: VGSGPH3 : hydrostatic sigma-P
C 2: VGSGPN3 : non-h sigma-P
C 3: VGSIGZ3 : sigma-Z
C 4: VGPRES3 : pressure (mb)
C 5: VGZVAL3 : Z (m) (above sea lvl)
C 6: VGHVAL3 : H (m) (above ground)
C 7: VGWRFEM : WRF mass-core sigma
C 8: VGWRFNM : WRF NMM
C 9: IMISS3 for vertical coordinates not stored in VGLVSD
C (e.g., temporally or spatially changing vertical coordinates)

  INTEGER, SAVE :: VGTYP_GD

This may indicate that there is a problem in the MCIP run in identifying the VGRTYP from WRF.

setgriddefs.f90 in MCIP contains the following:
IF ( met_model == 2 ) THEN ! WRF-ARW
IF ( met_hybrid <= 0 ) THEN ! sigma-type vertical coordinate
vgtyp_gd = vgwrfem ! terrain-following dry hydrostatic pressure
ELSE ! hybrid sigma-pressure vertical coordinate
vgtyp_gd = imiss3
ENDIF
ENDIF

Can you check the header information in your MCIP output data that you are using.
For the course domain:
ncdump -h /home/pkasper/53_CMAQ/data/mcip/April_2km_Nest_1/METCRO3D_180613.nc
For the finer domain:
ncdump -h /home/pkasper/53_CMAQ/data/mcip/April_2km_Nest_2/METCRO3D_180613.nc

Did you generate the MCIP course and MCIP fine domain data from WRF runs for the corresponding course domain and fine domain?

lizadams · January 12, 2021, 8:57pm

Yes, you need to comment out that line.

# setenv CONC_BLEV_ELEV " 1 1"

lawless · January 12, 2021, 9:01pm

Here the header outputs for the fine and coarse domain

METCRO3D_Coarse.txt (10.2 KB) METCRO3D_Fine.txt (10.2 KB)

Yes I did generate the MCIP coarse and MCIP fine domain data from the WRF runs for the corresponding coarse and fine domain.

Should I rerun with CONC_BLEV_ELEV commented out before investigating further?

lizadams · January 12, 2021, 9:15pm

Yes, rerun with CONC_BLEV_ELEV commented out. I think that will solve your problem.

lawless · January 20, 2021, 4:29pm

Hey @lizadams

I reran the outer domain with those lines in the runscript commented out and I was able to create a BCON and ICON for the nested domain however I realized that I only created a BCON for one day of the simulation. I tried combining all the CCTM_CONC files for the coarse domain using the combine tool in PREP but it reduces the CONC files back to one layer. How would I go about combining my CCTM_CONC files to account for all the days in my simulation when running the BCON script?

Thanks,
Zach

Topic		Replies	Views
Error when run MCIP 5.0 MCIP	6	1300	October 27, 2019
Have problem when run MCIP 5.0 with WRF 4.1 output MCIP	13	2240	May 27, 2021
Error running mcip MCIP	3	845	April 17, 2020
ERROR in run mcip, not SAME DOMAIN MCIP	9	2334	March 18, 2021
Error when runnung mcip MCIP	5	941	March 22, 2020

MCIP Nested Domains

Vertical Interpolation Section

Input file section

Output File Section

Vertical Interpolation Section

Related Topics