Failed when running smk_edgar_HEMI108k.csh

SMOKE
1

Hi people,
I am facing the same problem as reported in this topic (Grdmat failed when running smk_edgar_HEMI108k.csh), however I am using the SMOKE v4.6. Smkinven, spcmat and temporal had normal completion, but GRDMAT stops at the same point as reported:
(…)

 File "GRIDDESC" opened for input on unit:  96
 /scratch/ykaore/guildford/summer/aqm/smoke/../mcip_d03/GRIDDESC

 NOTE: Grid settings initialized using Guildford        in
      grid description file.
 NOTE: Output grid "Guildford" set; described as
      No description available
 Value for IOAPI_GRIDNAME_1:  'Guildford'
 Value for IOAPI_GRIDNAME_1:  'Guildford'

 File "AGSUP" opened for output on unit:  96
 /scratch/ykaore/guildford/summer/aqm/smoke/run_guildford/static/agsup.edgar.Guildoford.guildford.txt

 Generating gridding matrix...

 "GRIDMASK" opened as OLD:READ-ONLY
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/ge_dat/EDGAR/GRIDMASK_EDGAR.ncf"
 File type GRDDED3
 Execution ID "????????????????"
 Grid name "LATLON_EDGAR0.1"
 Dimensions: 1800 rows, 3600 cols, 1 lays, 2 vbles
 NetCDF ID:    131072  opened as READONLY
 Time-independent data.

 "GRID_CRO_2D" opened as OLD:READ-ONLY
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/../mcip_d03/GRIDCRO2D_d03.nc"
 File type GRDDED3
 Execution ID "mcip"
 Grid name "Guildford_CROSS"
 Dimensions: 118 rows, 118 cols, 1 lays, 7 vbles
 NetCDF ID:    196608  opened as READONLY
 Time-independent data.
 Value for IOAPI_ISPH:  '12'
 Value for IOAPI_ISPH:  '12'
 INITSPHERES:  input sphere WGS 84
 This file seems to use WMO-style "longitude" that violates ISO Standard 6709
 Number of coeffs     13924

And also, got an error during in smkmerge related to that:
(…)
“ATMP1” opened as OLD:READ-ONLY
File name “/scratch/ykaore/guildford/summer/aqm/smoke/run_guildford/scenario/atmp.edgar.guildford.2015181.1.ncf”
File type GRDDED3
Execution ID “???”
Grid name “”
Dimensions: 45360000 rows, 1 cols, 1 lays, 5 vbles
NetCDF ID: 131072 opened as READONLY
Starting date and time 2015181:000000 (0:00:00 June 30, 2015)
Timestep 010000 (1:00:00 hh:mm:ss)
Maximum current record number 775

 "ATMP2" opened as OLD:READ-ONLY
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/run_guildford/scenario/atmp.edgar.guildford.2015181.2.ncf"
 File type GRDDED3
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 45360000 rows, 1 cols, 1 lays, 5 vbles
 NetCDF ID:    196608  opened as READONLY
 Starting date and time  2015181:000000 (0:00:00   June 30, 2015)
 Timestep                          010000 (1:00:00 hh:mm:ss)
 Maximum current record number       775
 NOTE: Time zone initialized using ATMP             hourly emissions file.

 Value for PROMPTFLAG:  N returning FALSE
 AGMAT           :/scratch/ykaore/guildford/summer/aqm/smoke/run_guildford/static/agmat.edgar.Guildoford.guildford.ncf

 >>--->> WARNING in subroutine OPEN3
 File not available.

 Could not open file "AGMAT".

 *** ERROR ABORT in subroutine OPENMRGIN
 Could not open file "AGMAT".

It was said that the problem was solved adding “unlimit the stacksize in the script smk_edgar_HEMI108k.csh”. However I could not find anything related to neither in the documentation and variables. Someone could help me how can I do that?

2

What does the command

ls -l /scratch/ykaore/guildford/summer/aqm/smoke/run_guildford/static/agmat.edgar.Guildoford.guildford.ncf

tell you?

3

It was not generated.

image
image757×176 6.02 KB

4

Can you share what is the error message from Grdmat log file? That will give me an idea what it went wrong.

5

Hi B.H.,
Thank you for helping me. (Other logs had Normal completion (excluding grdmat and smkmerge))

grdmat.edgar.guildford.Guildoford

 This program uses the EPA-AREAL/MCNC-EnvPgms/BAMS Models-3      
 I/O Applications Programming Interface, [I/O API] which is      
 built on top of the netCDF I/O library (Copyright 1993, 1996    
 University Corporation for Atmospheric Research/Unidata         
 Program) and the PVM parallel-programming library (from         
 Oak Ridge National Laboratory).                                 
 Copyright (C) 1992-2002 MCNC,                                   
 (C) 1992-2013 Carlie J. Coats, Jr.,                             
 (C) 2003-2012 Baron Advanced Meteorological Systems, LLC, and   
 (C) 2014-2016 UNC Institute for the Environment.                
 Released under the GNU LGPL  License, version 2.1.  See URL     
                                                                 
     https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html     
                                                                 
 for conditions of use.                                          
                                                                 
 ioapi-3.2: $Id: init3.F90 98 2018-04-05 14:35:07Z coats $
 Version with PARMS3.EXT/PARAMETER::MXVARS3= 2048
 netCDF version 4.4.1.1 of Nov 18 2019 11:03:18 $
  
  
 Missing environment variable EXECUTION_ID
 SMOKE ---------------
 Copyright (c)2004 Environmental Modeling for Policy Development
 All rights reserved

 Program GRDMAT, Version SMOKEv4.6_Sep2018
 Online documentation 
     http://www.cep.unc.edu/empd/products/smoke                 
  
 Program GRDMAT to take a SMOKE area, mobile, or point source
 inventory file, gridding surrogates,  surrogate cross-
 reference, and an optional link definitions file, and produce
 a SMOKE gridding matrix for a grid defined at run time. For
 mobile sources, an "ungridding" matrix is also created to
 allow the use of gridded temperature data in assigning
 factors to mobile sources.
  
  
 You will need to enter the logical names for the input and
 output files (and to have set them prior to program start,
 using "setenv <logicalname> <pathname>").
  
 You may use END_OF-FILE (control-D) to quit the program
 during logical-name entry. Default responses are given in
 brackets [LIKE THIS] and can be accepted by hitting the
 <RETURN> key.
  
 Value for PROMPTFLAG:  N returning FALSE
 Returning default value TRUE for query:
 "Continue with program?"
 Value for GRDMAT_ADJUST not defined;returning default:   FALSE
 Value for GRDMAT_LINKDEFS not defined;returning default:   FALSE
 Value for USE_VARIABLE_GRID not defined;returning default:   FALSE
 Value for IMPORT_GRDNETCDF_YN:  Y returning TRUE
 Value for SMK_SOURCE:  'A'
 Value for SMK_SOURCE:  'A'
 Value for PROMPTFLAG:  N returning FALSE

 File "AREA" opened for input on unit:  98
 /scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/area.map.edgar.guildford.txt

 Value for AREA:  '/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/area.map.edgar.guildford.txt'
 Value for AREA:  '/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/area.map.edgar.guildford.txt'

 File "ASRC" opened for input on unit:  97
 /scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgarsrc.guildford.txt

 Value for IOAPI_CHECK_HEADERS not defined;returning default:   FALSE
  
 "AREA" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 45360000 rows, 1 cols, 1 lays, 4 vbles
 NetCDF ID:     65536  opened as READONLY            
 Time-independent data.
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/CO.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 1985845 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/NH3.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 3146407 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/NOX.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 1876919 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/PEC.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 3143520 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/PM10.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 2500786 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/PM2_5.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 2943229 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/POC.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 3237604 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/SO2.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 3229403 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/VOC.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 2889774 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/PMC.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 1871596 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/CO.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 1985845 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/NH3.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 3146407 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/NOX.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 1876919 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/PEC.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 3143520 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/PM10.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 2500786 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/PM2_5.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 2943229 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/POC.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 3237604 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/SO2.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 3229403 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/VOC.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 2889774 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
  
 "TMP_POL_FILE" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/inventory/guildford/edgar.guildford_dat/PMC.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name ""
 Dimensions: 1871596 rows, 1 cols, 1 lays, 7 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
 Closing file TMP_POL_FILE
 Reading source data from inventory file...
 Value for GRIDDESC:  '/scratch/ykaore/guildford/summer/aqm/smoke/../mcip_d03/GRIDDESC'
 Value for GRIDDESC:  '/scratch/ykaore/guildford/summer/aqm/smoke/../mcip_d03/GRIDDESC'

 File "GRIDDESC" opened for input on unit:  96
 /scratch/ykaore/guildford/summer/aqm/smoke/../mcip_d03/GRIDDESC

 Value for IOAPI_GRIDNAME_1:  'Guildford'
 Value for IOAPI_GRIDNAME_1:  'Guildford'

 File "GRIDDESC" opened for input on unit:  96
 /scratch/ykaore/guildford/summer/aqm/smoke/../mcip_d03/GRIDDESC

 NOTE: Grid settings initialized using Guildford        in  
      grid description file.
 NOTE: Output grid "Guildford" set; described as 
      No description available
 Value for IOAPI_GRIDNAME_1:  'Guildford'
 Value for IOAPI_GRIDNAME_1:  'Guildford'

 File "AGSUP" opened for output on unit:  96
 /scratch/ykaore/guildford/summer/aqm/smoke/run_guildford/static/agsup.edgar.Guildoford.guildford.txt

 Generating gridding matrix...
  
 "GRIDMASK" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/ge_dat/EDGAR/GRIDMASK_EDGAR.ncf"
 File type GRDDED3 
 Execution ID "????????????????"
 Grid name "LATLON_EDGAR0.1"
 Dimensions: 1800 rows, 3600 cols, 1 lays, 2 vbles
 NetCDF ID:    131072  opened as READONLY            
 Time-independent data.
  
 "GRID_CRO_2D" opened as OLD:READ-ONLY   
 File name "/scratch/ykaore/guildford/summer/aqm/smoke/../mcip_d03/GRIDCRO2D_d03.nc"
 File type GRDDED3 
 Execution ID "mcip"
 Grid name "Guildford_CROSS"
 Dimensions: 118 rows, 118 cols, 1 lays, 7 vbles
 NetCDF ID:    196608  opened as READONLY            
 Time-independent data.
 Value for IOAPI_ISPH:  '12'
 Value for IOAPI_ISPH:  '12'
 INITSPHERES:  input sphere WGS 84
 This file seems to use WMO-style "longitude" that violates ISO Standard 6709
 Number of coeffs     13924
6

Looks like Grdmat failed with segmentation fault without any proper error messag. This may be caused by lack of ram memory. Can you allocate more memory to run Grdmat? Because there are more than 6 million grid cells from these HTAP or EDGAR files, you need to have enough RAM memory to process them.

7

Thanks B.H.
I already solved it by running each sectors separately, and in the end I merged them.