Error when run edgar in smoke

hi,friends
i run the edgar from the testcase.my problems:
1.something defined in the assign file not exist in the folder,like REPCONFIG MET_CRO_2D MET_DOT_3D,do we not need them?
2.in case of lacking memory,i changed:(1)ncols 18 ,nrows 18 in GRIDDESC ,SRGDESC,do i need to change GRID_CRO?(2)leave one file in arinv.edgar.lst,(3)change assign as:
setenv G_STDATE 2005709 # Julian start date
setenv G_STTIME 000000 # start time (HHMMSS)
setenv G_TSTEP 60000 # time step (HHMMSS)
setenv G_RUNLEN 250000 # run length (HHMMSS)
setenv ESDATE 20050709 # Start date of emis time-based files/dirs
setenv MSDATE 20050709 # Start date of met time-based files
setenv NDAYS 1 # Duration in days of each emissions file
setenv MDAYS 1 # Duration in days of met time-based files
setenv YEAR 2005 # Base year for year-specific files
what i did,is anything wrong? is that helpful to reduce the memory need or not? anything else like Temporal,gridmask, speciation,cut inventory half ?
error i got in smkinv:
File “ARINV” opened for input on unit: 97
/usr/local/smoke/data/inventory/nei2005/edgar/arinv.edgar.lst
Value for WKDAY_NORMALIZE: N returning FALSE
NOTE: Setting inventory to use full-week normalizer for weekly profiles
*** ERROR ABORT in subroutine SMKINVEN:CHECKMEM
Failure allocating memory for “CEXTORL”: STATUS= 5014
i searched the forum found its about memory,is it? unlimted didnt work.
Thanks for your time.

Sorry for the delay on our response but can you give me an update whether you still have this issue or not?

Hi.
I have recieved the same error while run EDGAR in smoke.

***ERROR ABORT in subroutine SMKINVEN:CHECKMEM
Failure allocating memory for “CEXTORL”: STATUS= 5014

I have split those EDGAR inventories by sectors. And I added limit stacksize unlimited in my .cshrc .

Here is my log filesmkinven.edgar.BRW.log.txt (5.2 KB)

BTW, can SMOKE use the monthly data provided by EDGAR ? Should I just write the file path in the arinv.edgar.lst, or do I need to change somthing in ASSIGN file as well ?

Thanks a lot.

What compiler and versions? These status-numbers are compiler dependent.

Assuming gfortran, that probably means you are trying to allocate an array too large for your current executable’s memory model – see https://cjcoats.github.io/ioapi/AVAIL.html#medium

It may work to re-compile smkinven with the medium memory model (BIN=Linux2_x86_64gfort_medium, Linux2_x86_64ifort_medium, or Linux2_x86_64pg_medium) and see if that works. And try this on a machine with lots of RAM and memoryuse unlimited (not just stacksize…)

Before recompiling, try to run it with more RAM memory since it requires lots of memory due to its size of files.

Also, there could be an issue with your arinv.edgar.lst file, and that’s close to where the log stopped. If adding memory doesn’t work please provide the file for review.

Thanks for your reply! @cjcoats @bbaek @eyth.alison

Here are the compiler I use and the versions of the softwares:

gnu/4.8.5
mvapich2/2.3
netcdf/4.3.0
hdf5/1.8.20
zlib/1.2.7

I think the HPC I use needs me to write a PBS job script to read large files and run the program.
However, when I reduced the number of input files, a new problem appeared:

Value for PROMPTFLAG:  N returning FALSE
Returning default value TRUE for query:
"Continue with program?"
Value for GRDMAT_ADJUST not defined;returning default:   FALSE
Value for GRDMAT_LINKDEFS not defined;returning default:   FALSE
Value for USE_VARIABLE_GRID not defined;returning default:   FALSE
Value for IMPORT_GRDNETCDF_YN:  Y returning TRUE
Value for SMK_SOURCE:  'A'
Value for SMK_SOURCE:  'A'
Value for PROMPTFLAG:  N returning FALSE

File “AREA” opened for input on unit: 98
/nuist/scratch/pb02005030/lism/v47_smoke/data/inventory/BRW/area.map.edgar.BRW.txt

Value for AREA:  '/nuist/scratch/pb02005030/lism/v47_smoke/data/inventory/BRW/area.map.edgar.BRW.txt'
Value for AREA:  '/nuist/scratch/pb02005030/lism/v47_smoke/data/inventory/BRW/area.map.edgar.BRW.txt'

*** ERROR ABORT in subroutine RDINVMAP
Corrupted map formatted file - missing lines

Thank you for your time !

Hi, regarding the new error:

We are interested in seeing the Smkinven log, the ARINV .lst file used by Smkinven, and the area.map.edgar.BRW.txt file.

This error is from Grdmat, but the AREA file causing the error is created by Smkinven, so it’s possible something went wrong in Smkinven…

Did you also try running Smkinven with the additional memory?

Hi, thank you for your reply!

It seems that I can’t enlarge the RAM, but I tried to recompile it with -mcmodel=medium in the src/Makefile and still got the same error…

I found that after reducing the files, it appeared such words in foreground (shown below). Maybe this is the real reason why I got the error about AREA file.
image

And these are my log files
grdmat.edgar.BRW.Pol_71N_BRW.log.txt (3.4 KB) smkinven.edgar.BRW.log.txt (7.5 KB) and list file arinv.edgar.lst.txt (1.8 KB)

I can’t upload the area.map.edgar.BRW beacause it is empty…

Thanks for your time.

Hi.

I think there was a mistake in my previous arinv.lst. After I corrected it, there were no more error from SMKINVEN and GRDMAT.

But this showed up in smkmerge:

"AGMAT" opened as OLD:READ-ONLY
File name >"/nuist/scratch/pb02005030/lism/v47_smoke/data/run_ODE/static/agmat.edgar.Pol_71N_BRW.ODE.ncf"
File type SMATRX3
Execution ID "????????????????"
Grid name "2019_BRW_CROSS"
Dimensions: 39420 rows, 72405 cols, 1 lays, 1 vbles
NetCDF ID:    196608  opened as READONLY
Time-independent data.
ASMAT           :/nuist/scratch/pb02005030/lism/v47_smoke/data/run_ODE/static/asmat_l.edgar.cmaq_cb05_soa.ODE.ncf

>>--->> WARNING in subroutine OPENSET
File not available.

Could not open file set "ASMAT".

***ERROR ABORT in subroutine OPENMRGIN
Could not open file set "ASMAT".

Here is my new log file. grdmat.edgar.BRW.Pol_71N_BRW.log.txt (13.6 KB) smkinven.edgar.BRW.log.txt (27.9 KB) smkmerge.edgar.BRW.20190328.Pol_71N_BRW.log.txt (10.1 KB)

BTW, there are many warnings in my smkinven.edgar.BRW.log …

This is some good news that you have made progress. We note that the Smkmerge error is because the speciation matrix is missing. Assuming the end goal is speciated, hourly, model-ready emissions, you also need to run Spcmat and Temporal. There is a way to bypass those programs if they don’t want speciation or temporalization, via the MRG_SPCMAT_YN and MRG_TEMPORAL_YN settings: https://www.cmascenter.org/smoke/documentation/4.5/html/ch06s14s03.html

Note that EPA has a package posted for running SMOKE with hemispheric emissions including the EDGAR/HTAP inventories. Some of the scripts provided with this package may be helpful for you: ftp://newftp.epa.gov/Air/emismod/2011/hemispheric/

The 2016 hemispheric platform has a TSD posted, but note the data: https://www.epa.gov/air-emissions-modeling/2016-hemispheric-emissions-modeling-platform

1 Like