Question about using spatial surrogates

I am using scripts from and try to run USA_* files for my own grids. I got problems in using the script for test run in step 4 “Load the shapefiles into tables in the database”. The error is: /Spatial-Allocator//src/libs/gdal-2.0.2/local/bin/ogr2ogr: error while loading shared libraries: cannot open shared object file: No such file or directory.
Is there anyone know how to solve this issue?

I guess this error is attributed to the exe of ogr2ogr which is pre-compiled. In general, you may need to install all libs and re-compile the spatial allocator for use (at least for my case it works after re-compile).

Thanks for your response, Ryan! I changed all libraries to my own and got another error:ERROR: could not form projection (LWPROJ) from ‘srid=4269’ to ‘srid=900921’
Have you ever got this kind of error? Do you know how to deal with it?

What platform are you wanting to recreate the surrogates for and for what region / modeling domain?

900921 is the reference for the Lambert Conformal Conic projection commonly used for SMOKE and CMAQ modeling. Try running step (3) in the README on the pg_srgtools github to add this srid to the database:

I am using surrogate tool on the linux system. I am runing for a projection other than SMOKE pre-existed USA_* files so I might need to run this to generate the input for SMOKE.

Thanks for your suggestion. I have run the step 3 and it shows “INSERT 0 1” on my screen, once I run step 4, the error became:
proj_create: cannot build geodeticCRS 4269: SQLite error on SELECT name, ellipsoid_auth_name, ellipsoid_code, prime_meridian_auth_name, prime_meridian_code, area_of_use_auth_name, area_of_use_code, publication_date, deprecated FROM geodetic_datum WHERE auth_name = ? AND code = ?: no such column: publication_date

Do you know how to solve this issue?

I have a quick question. If I want to generate USA_FILL* files for my own grid using spatial surrogates script, what should I change in the script?
I list the file that should changed here to make sure I don’t miss anything:

  1. proj,lat_1,lat_2,lat_0,lon_0 and others in create_900921.sql
  2. xorig, yorig, xcellsize, ycellsize, cols and rows in “”

I have my own GRIDDESC file from MCIP run and I am trying to match those USA_FILL_* file with the same grid setting in GRIDDESC so that I can use them in SMOKE simulation

I have not seen that issue but it looks like a library mismatch. Can you please share which versions of libproj and GDAL you are using?

Thanks for your response. I reinstalled all software and libraries and re-ran all the steps, now the test run is done successfully, thanks for your help. I am curious how should I use the same script to generate the USA_* files for my own grids? I have grids that has different projection than the default inputs so I have to re-generate them for further running SMOKE.

It would be helpful to know which EPA platform you are trying to reproduce surrogates for – 2016, 2017, some other?

Also are you using the original Spatial Allocator or the SurrogateToolsDB?

FYI, there are examples for the 2017 platform on the Surrogate Tool github:

“run_srgtool_fill.csh” and “surrogate_specification_pg.2017.csv” are the most relevant to gapfilling.

Thanks for your response, Alison! I am generating the files for 2017 platform, I have successfully generate the USA_* files with matching header with my own GRIDDESC and I will test if it works on SMOKE. Thanks again for your help.

Thanks for response. I am using 2017 platform. But I notice that the compressed input shape file “PG_SurrogateTool_scripts_files.2017v1.31Jul2020.tar.gz” not includes all needed input when I trying to use load_shapefile_reproject_multi.2017.csh to load all data to generate all USA_* files. Do you you know where can I get completed dataset?

Which files appear to be missing from the zip file? I believe some were added as separate zipped files to the same GoogleDrive area as the large file.

Thanks for your response Alison. I am trying to generate all needed files for SMOKE simulation and once I
“runload_shapefile_reproject_multi.2017.csh” there are a lot files seems missing. I got some example error message here:
Unable to open datasource `/spatial_allocator/SurrogateToolsDB-1.3/data/emiss_shp2017/US/tl2019Counties_LCC_WRF.shp’ with the following drivers.

Unable to open datasource `/spatial_allocator/SurrogateToolsDB-1.3/data/emiss_shp2017/US/acs2016_5yr_bg.shp’ with the following drivers.

Unable to open datasource `/spatial_allocator/SurrogateToolsDB-1.3/data/emiss_shp2017/NCES/public_schools_2018_2019.shp’ with the following drivers.

Do you know where I can download the full inputs for generate all USA_* files for SMOKE?

From the Google Drive location linked in the section “Download additional shapefiles”, click on the folder named “NEI 2017/2016v2 Platforms”. You’ll see archives for two of the missing shapefiles:

For the acs2016_5yr_bg shapefile, there was an error in the utility script load_shapefile_reproject_multi.2017.csh where the input directory didn’t get reset properly. I’ve updated the GitHub repository with the fixed version:

(post deleted by author)

Thanks for your responses. There are some other errors that show up when running the updated load_shapefile_reproject_multi.2017.csh.
When loading fema_bsf_2002bnd, the error is:
ERROR 1: PROJ: proj_as_proj_string: Unsupported conversion method: Lambert_Conformal_Conic
ERROR: canceling autovacuum task
CONTEXT: while scanning block 13324 of relation “public.fema_bsf_2002bnd”
automatic vacuum of table “surrogates.public.fema_bsf_2002bnd”
NOTICE: index “fema_bsf_2002bnd_geom_900921_idx” does not exist, skipping
ERROR: ST_Transform: Input geometry has unknown (0) SRID
STATEMENT: UPDATE public.fema_bsf_2002bnd SET geom_900921 = ST_Multi(ST_Transform(wkb_geometry, 900921));
ERROR: ST_Transform: Input geometry has unknown (0) SRID

Do you know how to deal with it?


I have run into this specific problem when loading the FEMA shapefile with certain versions of the GDAL library. An edit to the projection file for the shapefile seems to take care of it. I’ve documented the specific change in this GitHub issue:

(post deleted by author)