ERROR: detected in movesmrg
ERROR: Running smk_run for part 4 for 20160801 testing SRGPRO_PATH set by SRGPRO input: /scratch/general/lustre/copycky/models/NEI/nei_2016_v1//ge_dat/gridding/surrogates/CONUS12_2014_30apr2019/
The movesmrg log file is also attached. It seems like the program stoped without any errors. I use met4moves from same platform to generate the MOVES files for this run. Could you please help me the see if there is anything wrong for my run? movesmrg_RPP_onroad_aug_2016fh_16j_20160801_US_12km_cmaq_cb6.log.txt (21.2 KB)
I do not see any particular error message in your log file either. This means the program probably ended with segmentation fault. @copycky28 Were you able to process RPD, RPV and RPH sectors successfully?
Yes, all other sectors were successfully done. It seems like this program ended without any error but I guess could be due to the warning message exceeding the MAXWARN in the script. I tried to modify the MAXWARN to a higher value but still same issue. It is possible due to the memory of my node or any other setting instead of the program itself?
I do not think it is related to no of warning messages but possibly memory issue or a bug since there was no specific error message. But if you were able to process RPD sector successfully, your server has enough memory since RPD sector requires the most memory to process. It means it may not be related to the memory but something else.
Can you share the terminal screen output when the run fails? If this is a segmentation fault, there should be some information for me to understand the issue better.
Also, do you mind sharing whether you made any changes to the original NEI modeling platform package from EPA or not? Like updated the inventories, change the domain, and so on?
Based on the info at the top of the log, it seems that you are using their own SMOKE compilation rather than the pre-compiled executables. If that is correct, you could give precompiled executables a try – sometimes that fixes issues like these.
Thanks for your suggestion. I attached the standard log file to this reply. It shows errors in MOVESMRG and the MOVESMRG log file is what I upload previously. I have no idea why the program ends without error. So I am wondering the possible reason. Thanks again for your time. RPP.o.txt (68.8 KB)
Are you running on AWS VM’s? How much memory are you using?
If this is not a memory issue (i.e. provisioning differences on your VM), I suggest you check out the latest SMOKE from github. We have recently fixed some issues with movesmrg crashing.
We typically run onroad sectors using one core with a lot of memory. I’m not exactly sure what the minimum is for a CONUS domain but I’m sure 4GB is not enough based on my tests.
Thanks for your suggestion. Now I can run the program without unknown interruption. But I got another error from MOVESMRG:
Processing MOVES lookup tables for reference county 000000006059 of fuel month: 7
Reading emission factors file rateperprofile_smoke_aq_cb6_saprc_1Aug2019_2016v1platform-2016-20190718_6059_07.csv
WARNING: Emission factors file does not contain requested model species ACET
WARNING: Emission factors file does not contain requested model species ALD2
WARNING: Emission factors file does not contain requested model species ALDX
WARNING: Emission factors file does not contain requested model species CH4
WARNING: Emission factors file does not contain requested model species ETH
WARNING: Emission factors file does not contain requested model species ETHA
WARNING: Emission factors file does not contain requested model species ETHY
WARNING: Emission factors file does not contain requested model species FORM
WARNING: Emission factors file does not contain requested model species KET
WARNING: Emission factors file does not contain requested model species MEOH
WARNING: Emission factors file does not contain requested model species NAPH
WARNING: Emission factors file does not contain requested model species TERP
WARNING: Emission factors file does not contain requested model species UNK
*** ERROR ABORT in subroutine MOVESMRG
Could not find minimum or maximum temperatures for county 000000006075 and episode month 8
Date and time 0:00:00 Aug. 25, 2016 (2016238:000000)
I tried recompiling SMOKE and checked that model works well for all other sectors except RPP. The program still ends without any errors once I use the V1 platform.
We thought there might be a message similar to the one he pasted into one of his replies:
*** ERROR ABORT in subroutine MOVESMRG
Could not find minimum or maximum temperatures for county 000000006075 and episode month 8
Date and time 0:00:00 Aug. 25, 2016 (2016238:000000)
Since the error is for county 06075, that must have occurred with the onroad_ca_adj sector, not the onroad sector. The log you just now provided is onroad, and that still has the original error in which Movesmrg just stops without giving an error message. So there might be one thing going on for onroad and another thing going on with onroad_ca_adj.
Consider the following:
Do new, fresh runs for onroad RPP and onroad_ca_adj RPP using our executables from the EPA FTP site, not your compiled ones
Let it loop through all days, whether they complete or not
Review the logs from onroad/RPP/logs and oncrad_ca_adj/RPP/logs
Also, what is the grid you are running on?
Review the METMOVES file (SMOKE_DAILY_2016fc_v2_16j_US_12km_2016184-2016243.k1.ncf) to see if the size is correct compared to the other days or if there are any other anomalies.
If nothing becomes apparent following this, we may ask you to send some additional files.
Thanks for your suggestion. I run onroad and onroad_ca_adj sector separately and both have same issue (stop without error message). I do use pre-compiled executable from EPA FTP and my own compiled excutables to test but same outcome.
I run this test with my own domain setting which is not included in pre-generated files so I run met4moves for my own domain. Is there anything else I need to generated for my own domain for RPP?