Illegal instruction (core dumped) error

Spatial Allocator
1

Hi, I am new to SA tools. After I installed the software following the instruction, I tried to run the test case for TN. When running ‘generate_surrogates.csh’, the program crashed with the following error. I turned on the ‘debug’ option, but didn’t get more useful information. Could you please me to figure out the problem? Thanks!

Hao

Writing surrogates to file /local/data/hhe/Spatial-Allocator/output/srg_M08_NASH.txt
WARNING: Environment variable: MAX_LINE_SEG, not set
Illegal instruction (core dumped)

GENERATING AIRPORT SURROGATE - category = 2
Surrogate Creator Version 4.3, 01/20/2017

EV: OUTPUT_FILE_TYPE=RegularGrid
Setting output grid

EV: OUTPUT_FILE_TYPE=RegularGrid
Reading Regular Grid

EV: OUTPUT_GRID_NAME=M08_NASH
WARNING: Environment variable: MAX_LINE_SEG, not set
MAX_LINE_SEG not set, discretization intervals disabled
griddesc file name = /local/data/hhe/Spatial-Allocator/data/GRIDDESC.txt

Ellipsoid var = OUTPUT_FILE_ELLIPSOID
EV: OUTPUT_FILE_ELLIPSOID=+a=6370000.0,+b=6370000.0
Ellipsoid=+a=6370000.0,+b=6370000.0
EV: OUTPUT_GRID_NAME=M08_NASH
Illegal instruction (core dumped)
0.000u 0.015s 0:00.28 3.5% 0+0k 0+0io 0pf+0w
Error generating surrogate for category 2
ozone-nox{hhe}1094: ./generate_surrogates.csh
Writing surrogates to file /local/data/hhe/Spatial-Allocator/output/srg_M08_NASH.txt
Illegal instruction (core dumped)

GENERATING AIRPORT SURROGATE - category = 2
Surrogate Creator Version 4.3, 01/20/2017

EV: OUTPUT_FILE_TYPE=RegularGrid
Setting output grid

EV: OUTPUT_FILE_TYPE=RegularGrid
Reading Regular Grid

EV: OUTPUT_GRID_NAME=M08_NASH
EV: MAX_LINE_SEG=1000
griddesc file name = /local/data/hhe/Spatial-Allocator/data/GRIDDESC.txt

Ellipsoid var = OUTPUT_FILE_ELLIPSOID
EV: OUTPUT_FILE_ELLIPSOID=+a=6370000.0,+b=6370000.0
Ellipsoid=+a=6370000.0,+b=6370000.0
EV: OUTPUT_GRID_NAME=M08_NASH
Illegal instruction (core dumped)
0.000u 0.013s 0:00.29 3.4% 0+0k 0+0io 0pf+0w
Error generating surrogate for category 2

2

Since you are new at SA tool. I would like for you to use our new SA tool using postgreSQL/postGIS. The older version of SA tool could run into this segmentation fault due to the size of shapefiles as well as the size of memory required. Here is the information of new PG surrogate tool.

3

Thanks for your suggestion, and I will try it. Another question, in the shapefile_catalogy.csv file, I found some data from emiss_shp2003/2010/2014. Where can I download these shapefiles?

Hao

4

@eyth.alison. Can you point @haohe.umcp to the location of latest shapefiles? I sent you the package of latest shapefiles last week.

5

Here are 2011 platform shapefiles:
ftp://newftp.epa.gov/air/emismod/2011/v3platform/spatial_surrogates/shapefiles/

Here are 2016 platform shapefiles (similar to 2014):
ftp://newftp.epa.gov/air/emismod/2016/alpha/spatial_surrogates/shapefiles/

I don’t think we have the older 2003 version posted anywhere now…

6

Additional shapefiles are here https://drive.google.com/drive/u/0/folders/1idGoi6I3GvKFCcf87O_8zMirM7Gtd0_E

image001.jpg

7

I have set up the postgreSQL/postGIS on our server. I followed all the steps without any error. But when running the ‘run_pg_srgcreate.csh’, the similar error showed again. What I should try next?

Run Date: Thu Apr 02 11:32:19 EDT 2020

            Main Control CSV File

GENERATION CONTROL FILE ./surrogate_generation_pg.csv
SURROGATE SPECIFICATION FILE ./surrogate_specification_pg.csv
SHAPEFILE CATALOG ./shapefile_catalog_pg.csv
SURROGATE CODE FILE ./surrogate_codes.csv
SHAPEFILE DIRECTORY …/data/emiss_shp2010
SRGCREATE EXECUTABLE …/bin/64bits/srgcreate.exe
DEBUG_OUTPUT Y
OUTPUT_FORMAT SMOKE
OUTPUT_FILE_TYPE RegularGrid
OUTPUT_GRID_NAME us12k_444x336
GRIDDESC ./GRIDDESC.txt
OUTPUT_FILE_ELLIPSOID +a=6370000.0,+b=6370000.0
OUTPUT DIRECTORY ./outputs/us12k_444x336
OUTPUT SURROGATE FILE ./outputs/agpro_us12k.txt
OUTPUT SRGDESC FILE ./outputs/srgdesc_us12k.txt
PGSCRIPT DIRECTORY ./pgscripts
OVERWRITE OUTPUT FILES YES
LOG FILE NAME ./LOGS/srg_conus12k.log
PG_SERVER localhost
PG_USER pgsurg
PGBIN /usr/bin
SRID_FINAL 900921
DBNAME surrogates
DENOMINATOR_THRESHOLD 0.0005
COMPUTE SURROGATES FROM SHAPEFILES NO
COMPUTE SURROGATES YES
MERGE SURROGATES NO
GAPFILL SURROGATES NO

Warning: PGSCRIPT DIRECTORY is not needed in /local/data/hhe/Spatial-Allocator/pg_srgtools/control_variables_pg.csv.
Warning: PG_SERVER is not needed in /local/data/hhe/Spatial-Allocator/pg_srgtools/control_variables_pg.csv.
Warning: PG_USER is not needed in /local/data/hhe/Spatial-Allocator/pg_srgtools/control_variables_pg.csv.
Warning: PGBIN is not needed in /local/data/hhe/Spatial-Allocator/pg_srgtools/control_variables_pg.csv.
Warning: SRID_FINAL is not needed in /local/data/hhe/Spatial-Allocator/pg_srgtools/control_variables_pg.csv.
Warning: DBNAME is not needed in /local/data/hhe/Spatial-Allocator/pg_srgtools/control_variables_pg.csv.
Warning: COMPUTE SURROGATES is not needed in /local/data/hhe/Spatial-Allocator/pg_srgtools/control_variables_pg.csv.
Get Grid Header For Surrogate Files
SRGCREATE_ERROR>WARNING: Environment variable: MAX_LINE_SEG, not set
SRGCREATE_ERROR>Illegal instruction (core dumped)
ERROR IN RUNNING THE EXECUTABLE: SRGCREATE

Error: Getting Grid Header Failed, check grid description file.

End Date: Thu Apr 02 11:32:19 EDT 2020
Elapsed time in minutes: 0.00585

ERROR – The Program Run Stopped

8

Make sure you use gov.epa.surrogate.ppg.Main package, not gov.epa.surrogate.SurrogateTool. gov.epa.surrogate.SurrogateTool package is used for gap filling and old surrogate tools.

9

I commented out the second line:

#!/bin/csh -f

source pg_setup.csh
java -classpath ./SurrogateTools-2.1.jar gov.epa.surrogate.ppg.Main control_variables_pg.csv

#merge and gapfilling
#java -classpath ./SurrogateTools-2.1.jar gov.epa.surrogate.SurrogateTool control_variables_pg.csv

But still got the error:

Run Date: Fri Apr 03 15:00:06 EDT 2020

            Main Control CSV File

GENERATION CONTROL FILE ./surrogate_generation_pg.csv
SURROGATE SPECIFICATION FILE ./surrogate_specification_pg.csv
SHAPEFILE CATALOG ./shapefile_catalog_pg.csv
SURROGATE CODE FILE ./surrogate_codes.csv
SHAPEFILE DIRECTORY …/data/emiss_shp2010
SRGCREATE EXECUTABLE …/bin/64bits/srgcreate.exe
SRGMERGE EXECUTABLE Java
DEBUG_OUTPUT Y
OUTPUT_FORMAT SMOKE
OUTPUT_FILE_TYPE RegularGrid
OUTPUT_GRID_NAME us12k_444x336
GRIDDESC ./GRIDDESC.txt
OUTPUT_FILE_ELLIPSOID +a=6370000.0,+b=6370000.0
OUTPUT DIRECTORY ./outputs/us12k_444x336
OUTPUT SURROGATE FILE ./outputs/agpro_us12k.txt
OUTPUT SRGDESC FILE ./outputs/srgdesc_us12k.txt
PGSCRIPT DIRECTORY ./pgscripts
OVERWRITE OUTPUT FILES YES
LOG FILE NAME ./LOGS/srg_conus12k.log
PG_SERVER localhost
PG_USER pgsurg
PGBIN /usr/bin
SRID_FINAL 900921
DBNAME surrogates
DENOMINATOR_THRESHOLD 0.0005
COMPUTE SURROGATES FROM SHAPEFILES NO
COMPUTE SURROGATES YES
MERGE SURROGATES NO
GAPFILL SURROGATES NO

            Get Grid Header For Surrogate Files

SRGCREATE_ERROR>WARNING: Environment variable: MAX_LINE_SEG, not set
SRGCREATE_ERROR>Illegal instruction (core dumped)
ERROR IN RUNNING THE EXECUTABLE: SRGCREATE

Error: Getting Grid Header Failed, check grid description file.

End Date: Fri Apr 03 15:00:07 EDT 2020
Elapsed time in minutes: 0.010733333333333333

ERROR – The Program Run Stopped

10

I found the link for emiss_shp2003:

https://drive.google.com/uc?id=0B4Gx-y00i4D0MGoyNHRKTFNMNHc&export=download

in the page (https://www.cmascenter.org/download/software/spatial_allocator/spatial_allocator_3-6.cfm?DB=TRUE)

But, pophu2k_tnnc.shp was not contained in the link. The pophu2k_tnnc is necessary for running srgtools.

11

I was out of town for a few days, so it took me a while to respond; sorry.

Recail that there are several different generations of Intel processors, each with its own instruction set: they keep adding new arithmetic instructions that are potentially faster, but are not backwards compatible (i.e., don’t run on previous-generation processors). For example, the AVX2 vector instructions introduced with Haswell-generation processors are potentially twice as fast as the previous-generation’s AVX vector instructions for the right kind of computations, which are again potentially twice as fast as the two-generations-before-that SSE4.2 instructions.

This is what happens when you try to run a program that uses later-model instructions on an earlier-model processor. What generation of processor are you using (cat /proc/cpuinfo will tell you that, in rather more detail than you probably want…), and what processor-generation was the program-executable compiled for? (how to determine this varies from compiler to compiler…)

12

Hi, I found a lot of the shapefiles listed in the US_SpatialSurrogate_Workbook_v093013 in https://gaftp.epa.gov/Air/emismod/2011/v3platform/spatial_surrogates/shapefiles/ and in [Google Drive - Virus scan warning ]. However, I am missing the following shapefiles.
state_pophu02
drycleaner_surrogate
drycleaner_surrogate_1_29
drycleaner_surrogate_30_56
exits
gas_station_1_29
gas_station_30_56
gas_station_surrogate
mjrrds
Ports_081412
transterm
us_lw2k
us_oilgas.

Where can I find these files and is there one place where I can download all the shapefiles listed in the surrogate_Specification script for either 2011/2014/2016 years? Any lead is appreciated.

Thanks,
Sham

13

We have located some of these files and posted them here:

https://gaftp.epa.gov/Air/emismod/2011/v3platform/spatial_surrogates/shapefiles/drycleaner_shapefiles_12oct2012.zip

https://gaftp.epa.gov/Air/emismod/2011/v3platform/spatial_surrogates/shapefiles/gas_station_shapefiles_12oct2012.zip

https://gaftp.epa.gov/Air/emismod/2011/v3platform/spatial_surrogates/shapefiles/surfzone_poly_st.zip

https://gaftp.epa.gov/Air/emismod/2011/v3platform/spatial_surrogates/shapefiles/us_l2wk_shapefile_16apr2009.zip

https://gaftp.epa.gov/air/emismod/2011/v3platform/spatial_surrogates/shapefiles/US_oilgas_Ports032310_Shapefiles.zip

https://gaftp.epa.gov/Air/emismod/2011/v3platform/spatial_surrogates/shapefiles/Ports_081412_shapefile_01oct2012.zip

We don’t have the state_pophu02 (this is probably data circa 2002), exits, mjrrds, or transterm Shapefiles available.

You could substitute newer surrogates for this where needed.

1 Like