I’m using the EPA’s 2016 alpha platform to process emissions. My simulated time period is over the summer of 2018. I’d like to know how to download and format the 2018 CEM data from the EPA Air Markets as inputs instead. It seems I’ll need the FF10 egucems* files, a “cemsum” file, and HOUR_UNIT files for each month I want to simulate. What is the recommended approach?
1 Like
Hi, the beta platform is available now and has more updated data for many sectors.
The inputs for that platform and documentation are here:
http://views.cira.colostate.edu/wiki/wiki/10197
The 2018 CEMS data are here:
https://ampd.epa.gov/ampd/
Go to the Prepackaged Data tab and go to the bottom to find the data in SMOKE format.
Keep in mind that the annual inventory is 2016 and there could be retirements and other changes in the operation of the sources between 2016 and 2018.
I am trying to use the 2016 v2 platform ptegu scripts to process CEM emissions from 2013. I downloaded the 2013 files from the link you provided on this thread. I ran cemscan with this script:
run_cemscan_2013.csh (525 Bytes)
When I run the ptegu daily script, I get this error:
SCRIPT ERROR: Some file patterns not matched for style Hourly
According to MULTIFILE_NAMEBREAK, the file prefixes are:
/uufs/chpc.utah.edu/common/home/daq-group2/SMOKE4_7/2017by_13winter/inputs/cem/HOUR_UNIT
Check file names and rerun
ERROR: Could not run combine_data.csh for hourly list file in Annual_area
I suspect the files I downloaded form AMPD are not quite in the right format for use with this script.
I see in the 2016v2 platform documentation that “we’ve split
the data into months and days as needed for our scripts, and run the data
through a “CEM correct” program.” I found this cem correct program in the 2016 v1 folder:
https://gaftp.epa.gov/Air/emismod/2016/v1/2016emissions/cemcorrect_2.1_Nov12.zip
But I’m unsure how to use the “cem correct” program. Is there guidance for this program?
You can try changing the ptegu daily run script to increase the number of the NAMEBREAK_HOURLY variable by 1.
If that doesn’t work, please send your list of hourly CEMS files, i.e.:
/uufs/chpc.utah.edu/common/home/daq-group2/SMOKE4_7/2017by_13winter/inputs/cem/HOUR_UNIT*
I changed the NAMBREAK_HOURLY variable from 4 to 5 and am still getting the same error as before. Here is my file list that I input to cemscan:
filelist2013.txt (1.2 KB)
Did you download the files from the SMOKE-ready files area? There are a couple areas of the CAMD site…
Here are the first few lines of my january HOUR_UNIT file:
3,"1","130101",0,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",1,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",2,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",3,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",4,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",5,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",6,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",7,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",8,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",9,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",10,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",11,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",12,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",13,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",14,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",15,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",16,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",17,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",18,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",19,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",20,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",21,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",22,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","130101",23,-9,-9,-9,0,-9,-9,-9,,,,,-9
Here is a screenshot of where I downloaded the files:
Do they match the format of the files for the 2016/2017 platforms?
It looks like it - but maybe I am missing something! Here are the first few lines from the january HOUR_UNIT file from the 2017 platform:
3,"1","170101",0,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",1,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",2,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",3,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",4,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",5,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",6,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",7,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",8,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",9,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",10,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",11,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",12,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",13,-9,-9,-9,0,-9,-9,-9,,,,,-9
3,"1","170101",14,-9,-9,-9,0,-9,-9,-9,,,,,-9
I notice that the 2017 platform files have 2 HOUR_UNIT files for each month:
while the 2013 files I downloaded only have 1 per month:
Is this where the cem correct script might come into play?
So there is an additional step involved in using the files. The SMOKE scripts expect to find two files per month, one with the last day of the month and one with all of the other days of the month, plus an additional file for December 31 of the previous year. A sample script to create these files from the full-month CEMs is pasted below. This script (after modifying for 2013) should be run in the directory with the HOUR_UNIT_2013_* files. The outputs will go in a subdirectory called SMOKE/, and the SMOKE run scripts should then point to the SMOKE/ files.
A sample script is provided for 2018 as an example. Save this text to an executable .csh file
#!/bin/csh -f
grep 180131 HOUR_UNIT_2018_01.txt > SMOKE/HOUR_UNIT_2018_01_31jan_2018.txt
sed ‘/180131/d’ HOUR_UNIT_2018_01.txt > SMOKE/HOUR_UNIT_2018_01_1-30jan_2018.txt
grep 180228 HOUR_UNIT_2018_02.txt > SMOKE/HOUR_UNIT_2018_02_28feb_2018.txt
sed ‘/180228/d’ HOUR_UNIT_2018_02.txt > SMOKE/HOUR_UNIT_2018_02_1-27feb_2018.txt
grep 180331 HOUR_UNIT_2018_03.txt > SMOKE/HOUR_UNIT_2018_03_31mar_2018.txt
sed ‘/180331/d’ HOUR_UNIT_2018_03.txt > SMOKE/HOUR_UNIT_2018_03_1-30mar_2018.txt
grep 180430 HOUR_UNIT_2018_04.txt > SMOKE/HOUR_UNIT_2018_04_30apr_2018.txt
sed ‘/180430/d’ HOUR_UNIT_2018_04.txt > SMOKE/HOUR_UNIT_2018_04_1-29apr_2018.txt
grep 180531 HOUR_UNIT_2018_05.txt > SMOKE/HOUR_UNIT_2018_05_31may_2018.txt
sed ‘/180531/d’ HOUR_UNIT_2018_05.txt > SMOKE/HOUR_UNIT_2018_05_1-30may_2018.txt
grep 180630 HOUR_UNIT_2018_06.txt > SMOKE/HOUR_UNIT_2018_06_30jun_2018.txt
sed ‘/180630/d’ HOUR_UNIT_2018_06.txt > SMOKE/HOUR_UNIT_2018_06_1-29jun_2018.txt
grep 180731 HOUR_UNIT_2018_07.txt > SMOKE/HOUR_UNIT_2018_07_31jul_2018.txt
sed ‘/180731/d’ HOUR_UNIT_2018_07.txt > SMOKE/HOUR_UNIT_2018_07_1-30jul_2018.txt
grep 180831 HOUR_UNIT_2018_08.txt > SMOKE/HOUR_UNIT_2018_08_31aug_2018.txt
sed ‘/180831/d’ HOUR_UNIT_2018_08.txt > SMOKE/HOUR_UNIT_2018_08_1-30aug_2018.txt
grep 180930 HOUR_UNIT_2018_09.txt > SMOKE/HOUR_UNIT_2018_09_30sep_2018.txt
sed ‘/180930/d’ HOUR_UNIT_2018_09.txt > SMOKE/HOUR_UNIT_2018_09_1-29sep_2018.txt
grep 181031 HOUR_UNIT_2018_10.txt > SMOKE/HOUR_UNIT_2018_10_31oct_2018.txt
sed ‘/181031/d’ HOUR_UNIT_2018_10.txt > SMOKE/HOUR_UNIT_2018_10_1-30oct_2018.txt
grep 181130 HOUR_UNIT_2018_11.txt > SMOKE/HOUR_UNIT_2018_11_30nov_2018.txt
sed ‘/181130/d’ HOUR_UNIT_2018_11.txt > SMOKE/HOUR_UNIT_2018_11_1-29nov_2018.txt
grep 181231 HOUR_UNIT_2018_12.txt > SMOKE/HOUR_UNIT_2018_12_31dec_2018.txt
sed ‘/181231/d’ HOUR_UNIT_2018_12.txt > SMOKE/HOUR_UNIT_2018_12_1-30dec_2018.txt
sed ‘s/181231/171231/g’ SMOKE/HOUR_UNIT_2018_12_31dec_2018.txt > SMOKE/HOUR_UNIT_2017_12_31dec_2018.txt
Here is a list of the resulting files that come out for 2018:
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2017_12_31dec_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_01_1-30jan_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_01_31jan_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_02_1-27feb_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_02_28feb_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_03_1-30mar_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_03_31mar_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_04_1-29apr_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_04_30apr_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_05_1-30may_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_05_31may_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_06_1-29jun_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_06_30jun_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_07_1-30jul_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_07_31jul_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_08_1-30aug_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_08_31aug_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_09_1-29sep_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_09_30sep_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_10_1-30oct_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_10_31oct_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_11_1-29nov_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_11_30nov_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_12_1-30dec_2018.txt
/work/EMIS/em_v8.1/2018platform/work/cems/corrected/SMOKE/HOUR_UNIT_2018_12_31dec_2018.txt
2 Likes
Yes I just sent another post on this topic