logfile.txt (9.3 KB)
attachment is my logfile
pls help me deal with the problem
Run Date: Fri Apr 19 18:36:06 MDT 2019
Main Control CSV File
GENERATION CONTROL FILE ./surrogate_generation_grid_2010.csv
SURROGATE SPECIFICATION FILE ./surrogate_specification_2010.csv
SHAPEFILE CATALOG ./shapefile_catalog_2010.csv
SHAPEFILE DIRECTORY …/data/shapefiles_2010/emiss_shp2010
SURROGATE CODE FILE ./surrogate_codes_2010.csv
SRGCREATE EXECUTABLE …/bin/32bits/srgcreate.exe
DEBUG_OUTPUT Y
OUTPUT_FORMAT SMOKE
OUTPUT_FILE_TYPE RegularGrid
OUTPUT_GRID_NAME US27_180x132
GRIDDESC ./GRIDDESC.txt
OUTPUT_FILE_ELLIPSOID +a=6371200.0,+b=6371200.0
OUTPUT DIRECTORY …/output/US27_180x132
OUTPUT SURROGATE FILE …/output/US27_180x132/srg_total.txt
OUTPUT SRGDESC FILE …/output/US27_180x132/SRGDESC.txt
OVERWRITE OUTPUT FILES YES
LOG FILE NAME srgRun_US27_180x132_2010.log
DENOMINATOR_THRESHOLD 0.0005
COMPUTE SURROGATES FROM SHAPEFILES YES
MERGE SURROGATES YES
GAPFILL SURROGATES YES
Get Grid Header For Surrogate Files
SRGCREATE_ERROR>WARNING: Environment variable: MAX_LINE_SEG, not set
SRGCREATE_OUTPUT>Surrogate Creator Version 4.3, 01/20/2017
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>EV: OUTPUT_FILE_TYPE=RegularGrid
SRGCREATE_OUTPUT>Setting output grid
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>EV: OUTPUT_FILE_TYPE=RegularGrid
SRGCREATE_OUTPUT>Reading Regular Grid
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>EV: OUTPUT_GRID_NAME=US27_180x132
SRGCREATE_OUTPUT>MAX_LINE_SEG not set, discretization intervals disabled
SRGCREATE_OUTPUT>griddesc file name = ./GRIDDESC.txt
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>Ellipsoid var = OUTPUT_FILE_ELLIPSOID
SRGCREATE_OUTPUT>EV: OUTPUT_FILE_ELLIPSOID=+a=6371200.0,+b=6371200.0
SRGCREATE_OUTPUT>Ellipsoid=+a=6371200.0,+b=6371200.0
SRGCREATE_OUTPUT>EV: OUTPUT_GRID_NAME=US27_180x132
SRGCREATE_OUTPUT>Not using BB optimization
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>EV: OUTPUT_FILE_TYPE=RegularGrid
SRGCREATE_OUTPUT>#GRID US27_180x132 -2430377.300000 -1782275.300000 27000.000000 27000.000000 180 132 1 LAMBERT meters 30.000000 47.000000 -97.000000 -97.000000 40.000000
SUCCESS IN RUNNING THE EXECUTABLE: SRGCREATE
Run srgcreate.exe to generate surrogate ratios for USA_100: Population
SRGCREATE_ERROR>WARNING: Environment variable: MAX_LINE_SEG, not set
SRGCREATE_ERROR>WARNING: Environment variable: MAX_LINE_SEG, not set
SRGCREATE_ERROR>WARNING: Environment variable: MAX_LINE_SEG, not set
SRGCREATE_OUTPUT>Surrogate Creator Version 4.3, 01/20/2017
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>EV: OUTPUT_FILE_TYPE=RegularGrid
SRGCREATE_OUTPUT>Setting output grid
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>EV: OUTPUT_FILE_TYPE=RegularGrid
SRGCREATE_OUTPUT>Reading Regular Grid
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>EV: OUTPUT_GRID_NAME=US27_180x132
SRGCREATE_OUTPUT>MAX_LINE_SEG not set, discretization intervals disabled
SRGCREATE_OUTPUT>griddesc file name = ./GRIDDESC.txt
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>Ellipsoid var = OUTPUT_FILE_ELLIPSOID
SRGCREATE_OUTPUT>EV: OUTPUT_FILE_ELLIPSOID=+a=6371200.0,+b=6371200.0
SRGCREATE_OUTPUT>Ellipsoid=+a=6371200.0,+b=6371200.0
SRGCREATE_OUTPUT>EV: OUTPUT_GRID_NAME=US27_180x132
SRGCREATE_OUTPUT>Not using BB optimization
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>EV: OUTPUT_FILE_NAME=…/output/US27_180x132/grid_USA_100
SRGCREATE_OUTPUT>EV: WEIGHT_ATTR_LIST=POP2010
SRGCREATE_OUTPUT>EV: FILTER_FILE=NONE
SRGCREATE_OUTPUT>Filter File= NONE
SRGCREATE_OUTPUT>Reading data polygons
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>EV: DATA_FILE_ELLIPSOID=+a=6370000.0,+b=6370000.0
SRGCREATE_OUTPUT>Ellipsoid=+a=6370000.0,+b=6370000.0
SRGCREATE_OUTPUT>EV: DATA_FILE_MAP_PRJN=+proj=lcc,+lat_1=33,+lat_2=45,+lat_0=40,+lon_0=-97
SRGCREATE_OUTPUT>EV: DATA_FILE_NAME_TYPE=ShapeFile
SRGCREATE_OUTPUT>EV: DATA_FILE_NAME=…/data/shapefiles_2010/emiss_shp2010/county_pophu2010
SRGCREATE_OUTPUT>MAX_LINE_SEG not set, discretization intervals disabled
SRGCREATE_OUTPUT>max_line_seg=
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>Reading Shapefile …/data/shapefiles_2010/emiss_shp2010/county_pophu2010
SRGCREATE_OUTPUT>Shapefile Type: 5 Polygon # of Shapes: 3143
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>Input projection:
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>args[1]=+a=6370000.0
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>args[2]=+b=6370000.0
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>param 0 = +proj=lcc,
SRGCREATE_OUTPUT>param 3 = +lat_1=33,
SRGCREATE_OUTPUT>param 4 = +lat_2=45,
SRGCREATE_OUTPUT>param 5 = +lat_0=40,
SRGCREATE_OUTPUT>param 6 = +lon_0=-97,
SRGCREATE_OUTPUT>PROJ args=+proj=lcc+a=6370000.0+b=6370000.0+lat_1=33+lat_2=45+lat_0=40+lon_0=-97
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>Output projection:
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>args[1]=+a=6371200.0
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>args[2]=+b=6371200.0
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>setting Lambert Conic Conformal – 2SP parameters
SRGCREATE_OUTPUT>PROJ args=+proj=lcc+a=6371200.0+b=6371200.0+lat_1=30.000000+lat_2=47.000000+lon_0=-97.000000+lat_0=40.000000+units=m
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>Total area = 7.9703e+12
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>Skipped 98 polygons and 69486 vertices
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>File bounding box:
SRGCREATE_OUTPUT>xmin = -6992237.655 xmax = 2321867.522 ymin = -1604329.909 ymax = 4569533.113
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>EV: DATA_ID_ATTR=FIPSSTCO
SRGCREATE_OUTPUT>EV: DATA_ID_ATTR=FIPSSTCO
SRGCREATE_OUTPUT>Not using function for weights
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>Using attribute FIPSSTCO
SRGCREATE_OUTPUT>PolyMShapeInOne.c – put multiple shapes with the same ID into one PolyObject…
SRGCREATE_OUTPUT>Number of shape Objects = 3143
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>Base data poly attribute type is STRING
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>New limiting bounding box:
SRGCREATE_OUTPUT>xmin = -2268808.353 xmax = 2321867.522 ymin = -1604329.909 ymax = 1271849.652
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>Existing limiting bounding box:
SRGCREATE_OUTPUT>xmin = -2268808.353 xmax = 2321867.522 ymin = -1604329.909 ymax = 1271849.652
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>Reading weight points/lines/polygons
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>EV: WEIGHT_FILE_ELLIPSOID=+a=6370000.0,+b=6370000.0
SRGCREATE_OUTPUT>Ellipsoid=+a=6370000.0,+b=6370000.0
SRGCREATE_OUTPUT>EV: WEIGHT_FILE_MAP_PRJN=+proj=lcc,+lat_1=33,+lat_2=45,+lat_0=40,+lon_0=-97
SRGCREATE_OUTPUT>EV: WEIGHT_FILE_TYPE=ShapeFile
SRGCREATE_OUTPUT>EV: WEIGHT_FILE_NAME=…/data/shapefiles_2010/emiss_shp2010/pophu_bg2010
SRGCREATE_OUTPUT>MAX_LINE_SEG not set, discretization intervals disabled
SRGCREATE_OUTPUT>max_line_seg=
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>Reading Shapefile …/data/shapefiles_2010/emiss_shp2010/pophu_bg2010
SRGCREATE_OUTPUT>Shapefile Type: 5 Polygon # of Shapes: 217740
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>Input projection:
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>args[1]=+a=6370000.0
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>args[2]=+b=6370000.0
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>param 0 = +proj=lcc,
SRGCREATE_OUTPUT>param 3 = +lat_1=33,
SRGCREATE_OUTPUT>param 4 = +lat_2=45,
SRGCREATE_OUTPUT>param 5 = +lat_0=40,
SRGCREATE_OUTPUT>param 6 = +lon_0=-97,
SRGCREATE_OUTPUT>PROJ args=+proj=lcc+a=6370000.0+b=6370000.0+lat_1=33+lat_2=45+lat_0=40+lon_0=-97
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>Output projection:
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>args[1]=+a=6371200.0
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>args[2]=+b=6371200.0
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>setting Lambert Conic Conformal – 2SP parameters
SRGCREATE_OUTPUT>PROJ args=+proj=lcc+a=6371200.0+b=6371200.0+lat_1=30.000000+lat_2=47.000000+lon_0=-97.000000+lat_0=40.000000+units=m
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>Total area = 1.0121e+13
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>Skipped 0 polygons and 0 vertices
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>File bounding box:
SRGCREATE_OUTPUT>xmin = -6992237.655 xmax = 2321867.522 ymin = -1604329.909 ymax = 4569533.113
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>EV: WEIGHT_ATTR_LIST=POP2010
SRGCREATE_OUTPUT>EV: WEIGHT_ATTR_LIST=POP2010
SRGCREATE_OUTPUT>EV: SURROGATE_ID=100
SRGCREATE_OUTPUT>Not using function for weights
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>Using attribute POP2010
SRGCREATE_OUTPUT>Intersecting weight objects with data polygons
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>xmin = -6992237.655 xmax = 2321867.522 ymin = -1604329.909 ymax = 4569533.113
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>xmin = -6976865.747 xmax = 2321867.522 ymin = -1604329.909 ymax = 4543538.713
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>numpoly1=217740, numpoly2 = 3143
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>xmin = -6992237.655 xmax = 2321867.522 ymin = -1604329.909 ymax = 4569533.113
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>Intersecting weight-data objects with grid polygons
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>xmin = -2268808.353 xmax = 2321867.522 ymin = -1604329.909 ymax = 1271849.652
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>xmin = -2430377.300 xmax = 2429622.700 ymin = -1782275.300 ymax = 1781724.700
SRGCREATE_OUTPUT>
SRGCREATE_OUTPUT>numpoly1=276507, numpoly2 = 23760
SRGCREATE_OUTPUT>
SRGCREATE_ERROR>gpc malloc failure: edge table creation
SRGCREATE_OUTPUT>xmin = -2268808.353 xmax = 2321867.522 ymin = -1604329.909 ymax = 1271849.652
SRGCREATE_OUTPUT>
SUCCESS IN RUNNING THE EXECUTABLE: SRGCREATE
Error: File - …/output/US27_180x132/USA_100_NOFILL.txt - Does Not Exist
Surrogate Generation Summary
USA_100 Population SRGCREATE Success
End Date: Fri Apr 19 19:14:52 MDT 2019
Elapsed time in minutes: 38.76546666666667
ERROR – The Program Run Stopped
Looks like you are generating, merging and gap filing at the same time. Based on your log file, it did generate the surrogate successfully but then got an error message because it tried to gap fill before generating the surrogates. I think you should run these three steps one at a time by setting them YES/NO. Make And then merge and/or gapfill based on your needs.
Thank you for quick response.
yesterday I just tried to change the grid name into M08_NASH (default), Then it works.
Is possible my grid setting in GRIDDESC wrong?
Blockquote
! coords --line: name; type, P-alpha, P-beta, P-gamma, xcent, ycent
‘NA_LAT_LONG’
1, 0, 0, 0, 0, 0
‘LAM_40N_100W’
2, 30, 60, -100, -100, 40
‘LAM_40N_97W’
2, 33, 45, -97, -97, 40
‘LAM_40N_90W’
2, 30, 60, -90, -90, 40
‘OMERC_’
3, -100, 20, 10, 127000, 52000
‘STERE_40_100’
4, 40, -100, 0, 0, 0
‘UTM_ZONE10’
5, 10, 0, 0, 0, 0
‘STERE_POLAR_N’
6, 80, -100, 1, 0, 0
‘EMERC_40_100’
7, 40, 0, -100, 0, 0
‘TMERC_40_100’
8, 40, -100, 1, 0, 0
‘AEA_40_100’
9, 30, 60, -100, -100, 40
‘LAEA_40_100’
10, 40, -100, 0, 0, 0
‘SINU_100’
11, -100, 0, 0, 0, 0
‘LAM_40N97W’
2, 30, 47, -97, -97, 40
’ ’ ! end coords. grids: name; xorig,yorig,xcell,ycell,ncols,nrows,nthik
‘US27_180x132’
‘LAM_40N97W’, -2.4303773E+06, -1.7822753E+06, 2.7000000E+04, 2.7000000E+04, 180, 132, 1
‘US27_181x133’
‘LAM_40N97W’, -2.4438773E+06, -1.7957753E+06, 2.7000000E+04, 2.7000000E+04, 181, 133, 1
‘US27_181x132’
‘LAM_40N97W’, -2.4438773E+06, -1.7822753E+06, 2.7000000E+04, 2.7000000E+04, 181, 132, 1
‘US27_180x133’
‘LAM_40N97W’, -2.4303773E+06, -1.7957753E+06, 2.7000000E+04, 2.7000000E+04, 180, 133, 1
‘M08_NASH’
‘LAM_40N_100W’, 1.000E+06, -5.360E+05, 8000, 8000, 46, 42, 1
‘M16_NASH’
‘LAM_40N_100W’, 1.000E+06, -5.360E+05, 16000, 16000, 23, 21, 1
‘US36’
‘LAM_40N_100W’, -2196.D3, -1692.D3, 36.D3, 36.D3, 132, 90, 1
‘LATLONG_GRID’
‘LATLONG’, -97, 23, 0.25, 0.25, 100, 80, 1
‘EGRID_359X573’
‘NA_LAT_LONG’, -129.9572906, 13.22372627, 0.1, 0.1, 359, 573, 1
‘US36’
‘LAM_40N_100W’, -2196.D3, -1692.D3, 36.D3, 36.D3, 132, 90, 1
‘US36KM_148X112’
‘LAM_40N_97W’, -2736000, -2088000, 36000, 36000, 148, 112, 1
’ ’
what’s meaning of this error?
This means you don’t have enough memory allocated to your process to run the program. First, see what happens if you do limit stacksize unlimited; limit memoryuse unlimited before you run the program – see https://www.cmascenter.org/ioapi/documentation/all_versions/html/AVAIL.html#medium
If that doesn’t work, you may either need to compile the surrogate tool with -mcmodel=medium -shared-intel or you may need a larger machine or you may have a badly specified problem (grid size isn’t what you really think it is…)
I have added “limit stacksize unlimited” before run the program.
the error of “SRGCREATE_ERROR>gpc malloc failure: edge table creation” still exists.
Besides, what’s meaning of your comment?
I have added “limit stacksize unlimited” before run the program.
the error of “SRGCREATE_ERROR>gpc malloc failure: edge table creation” still exists.
Besides, what’s meaning of your comment?
cjcoats:
you may have a badly specified problem (grid size isn’t what you really think it is…)
Perhaps you have a scripting error?
Hi,
I have added the limit stacksize unlimited before my running.
then I have tried three tests:
- my own small scale grid: 27 km * 27 km with 80, 32 grids, it worked successfully.
- my own large scale grid: 27 km * 27 km with 180, 132 grids, it failed.
- default grid (US36KM_148X112) : 36 * 36 km with 148, 112 grids, it failed.
- default grid (M08_NASH), 8 * 8 km with 46, 42 grids, it successed.
Should I compile with mcmodel=medium??
Besides,
how to recompile the srgtools separately? nor to compile the whole spatial-allocator?
Is it possible to specify a bigger memory for you failed runs on your machine? On UNC cluster machine, we use something like this:
#SBATCH -p general
#SBATCH -N 1
#SBATCH -n 1
#SBATCH --mem=60g
#SBATCH -t 8-00:00:00
sounds good. I will try it. Thanks for your comments.