Question about using spatial surrogates

Spatial Allocator
29

Can you tell if there is any data loaded in the table that is being vacuumed?

32

And also, I noticed the spatial surrogate for Canada and Mexico is not included in the script. But I need them to run those sectors such as onroad_mex. Do you know how could I generate the needed CA_XXX files for these simulations?

33

And also, I noticed the spatial surrogate for Canada and Mexico is not included in the script. But I need them to run those sectors such as onroad_mex. Do you know how could I generate the needed CA_XXX files for these simulations?

34

I found some input from NEI 2017/2016v2 Platforms - Google Drive.
I am wondering if this link contains all the needed input for running surrogates for Canada and Mexico.
Also, may I know where I can find the scripts for running surrogates for Canada and Mexico?
Many thanks for your time and help!

Best,
K

35

Yes, that folder includes the Canada2015 Shapefiles in CANADA2015_shapefiles_for_2016_platform and the Mexico shapefiles in Mexico_shapefiles.tar.gz

It also includes Mexico_surrogate_specifications_2016platform.xlsx and surrogate_tool_inputs_for_canada_2016v1_19aug2021.zip

That show how the surrogates are made.

We have some updates for Canada surrogates in the 2020 platform that we are working on posting soon.

1 Like
37

Thanks for your reply, Alison. I am wondering if I want to use the surrogate tool to generate the CA_xxx files. Does it work if I modify the load_shapefile_reproject_xxx file and load the shape file and the table name listed in the surrogate_specification_2015_canada_12oct2018_v0.csv and run the same steps as in GitHub - CEMPD/SurrogateToolsDB? Looking forward to your help. Thanks!

Best,
K

38

I noticed that I am not able to change all the details in the file load.shapefile.csh for Canada and Mexico. Do you know if there is any other way I can use the other dataset or scripts for previous years to generate the CA_XXX file for running other projection in SMOKE?

39

You will need to update the load_shapefile.csh to add the additional weighting tables for Canadian shapefiles that have more than one attribute used for generating surrogates. For each of the attributes you will need to add an attribute density column to the database table defined as the attribute column divided by the polygon area calculated in your target projection.
When only one attribute is used from the weighting shapefile the existing load_shapefile.csh script can be used unmodified.

Here is an example for the 2018 Canada population data table (da_popdwell_100m_nolakes_1nov17) where 8 attribute densities were calculated in load_shapefile.csh:

if ( $table == “da_popdwell_100m_nolakes_1nov17” ) then
echo “pop”
$PGBIN/psql -h $server -U $user -q $dbname << END1
ALTER TABLE $schema.$table
add column area_900921 double precision,
add column pop_dens double precision,
add column urdwell_dens double precision,
add column uadwell_dens double precision,
add column radwell_dens double precision,
add column cap_uapop_dens double precision,
add column cap_rapop_dens double precision,
add column cap_urdwel_dens double precision,
add column capurpop_dens double precision;
update $schema.$table
set area_900921=ST_Area(geom_900921);
update $schema.$table set
pop_dens=pop / area_900921,
urdwell_dens=urdwell / area_900921,
uadwell_dens=uadwell / area_900921,
radwell_dens=radwell / area_900921,
cap_uapop_dens=cap_uapop / area_900921,
cap_rapop_dens=cap_rapop / area_900921,
cap_urdwel_dens=cap_urdwel / area_900921,
capurpop_dens=capurpop / area_900921;
END1
endif

41

Thanks for your information, James! That is really helpful! Do you happen to have scripts for generating all Canada and Mexico surrogate files for the NEI platform? I am running emissions using the 2018 platform and my projection is a bit different than the preset from the EPA domain. I really need these surrogate files to run the sectors such as onroad_can, onroad_mex and othafdust. I don’t know how to modify all the tables for Canada and Mexico. I have no idea what attribute densities I should add to the script for other data tables.

Best,
K

42

Unfortunately I do not have readily available scripts for generating the Canada and Mexico surrogate files for the 2018 NEI platform using the surrogate DB tool. To find the attributes that need densities added to the load_shapefile.csh script you should look in the surrogate specification table/CSV. This table lists all surrogates, the weight shapefiles used for the surrogate, and the associated weight attributes.

45

Thanks for your response, James! Or do you have any ready script for the elder platform such as the 2016 platform? or any model ready emission files for the othafdust and onroad_can/onroad_mex?

46

So did you look at the data on the Google Drive and those were not sufficient?

48

Thanks for your suggestion, Alison. May I know where can I find the surrogate script for Canada and Mexico for 2016? or any model-ready SMOKE outputs for 2018?

49

Sorry – we don’t have the scripts set up for the PostgreSQL version of SurrogateToolsDB – we ran them with the regular Spatial Allocator.

You can use the regular Spatial Allocator tools and run them with the input files on the Google Drive NEI 2017/2016v2 Platforms - Google Drive.

Mexico_surrogate_specifications_2016platform.xlsx and surrogate_tool_inputs_for_canada_2016v1_19aug2021.zip

51

Thanks for your reply, Alison. I am curious if there is any 2018 SMOKE output similar to the 2017platform (premerged - Google Drive) that I would like to check some of the emissions.

52

And also, do you happen to have a regular spatial allocation script for Canada and Mexico that I can try to modify them in my case?

53

I already pointed you to the Surrogate Tool input files on the Google Drive site. Those are the input files to the older Surrogate Tool which is part of this package:

https://www.cmascenter.org/sa-tools/ Please review this and the provided Canada and Mexico input files earlier in the thread.

Also, there is a version of 2018 posted on the CMAS Dataverse: https://dataverse.unc.edu/dataset.xhtml?persistentId=doi:10.15139/S3/LDTWKH

These data may be based on slightly different inventories than the OAQPS 2018 case, although some will be similar.

55

Thanks for your help. Can I directly use SA tools to convert the provided CA_xxx files from the 2018 platform to my own projection (within the domain of 12US1) or should I use the generate_surrogate_xx.csh file to generate all surrogate ID files for Canada and Mexico?

56

Use the SurrogateTools.jar with the provided input files but updated to use your grid:

java -classpath SurrogateTools.jar gov.epa.surrogate.SurrogateTool control_variables_grid.csv

See https://github.com/CMASCenter/Spatial-Allocator/blob/master/docs/User_Manual/SA_ch05_surrogate.md#running

57

Thanks for your help. It works now!