I’m using the most recent version of AMET. I downloaded only AQS data for April, 2017 and used the combine and sitecmp tools in CMAQ to create a merged csv file for AMET use. I’m running AMET without the database (with modifications from this thread: How to use AMET without mySQL? - #12 by qluo ) and was able to run the spatial plots, but I am getting an error in the stat plots that is confusing. See the screenshot:
This file definitely exists and is the file that is used in the other Spatial plots I’ve generated. The multi_network_stats_data.csv file is created in the plots output but there are no pngs or pdfs. Below are the run/config files I’ve modified and the log output. Any help is appreciated, and if possible I’d like to avoid using the database.
In addition, AMET takes a long time to run, about 15+ minutes just to generate Spatial plots for O3. Is this normal? I’ll likely be using it a lot more for several different species over several months of data.
Hi Grant,
I checked your log in [Run_Spatial_Plots_All_Batch.Rout.txt.
I found some interesting thing.
I guess you are doing the AMET evaluation for 2006.
#start_date ← “20060101” # Set the start date of the analysis #end_date ← “20061231” # Set the end date of the analysis
If you look at the log you see the observations time period it is looking for
[1] " and s.stat_id=d.stat_id and d.ob_dates >= 20170401 and d.ob_datee <= 20170430 and ( d.ob_dates between 20170401 and 20170430 and d.ob_datee between 20170401 and 20170430 )" is from 1st April to 30th April 2017. Can you please check and confirm your settings.
I think those lines are commented out and shouldn’t affect things. I am running over the month of April and was able to get results from other Spatial plots like difference. I mostly want to be able to generate spatial plots of mean bias.
I’ve confirmed I can reproduce your error. So, I’ll see if I can diagnose what’s going on. I don’t often use AMET without the database, so it might take me a little bit longer to figure out. But, hopefully, it’s something fairly simple. I’ll let you know when I have a solution.
I believe I’ve tracked down the source of the error. The fix is addition of the definition of “stat_id_noPOC” in the AQ_Misc_Functions.R script.
In your AMET installation directory, in the R_analysis_code directory, you’ll need to edit the AQ_Misc_Functions.R script to add the missing line. In the AQ_Misc_Functions.R script, in the definition for the “read_sitex” function you’ll find the following line toward the end of the function (aorund line # 1381 in the AMETv1.5 code):
Save the file. That’s all you’ll need to do in the AQ_Misc_Functions.R file.
Once you make that change you should be able to successfully run the AQ_Stats_Plots.R script. The change to the AQ_Misc_Functions.R script may also fix other errors you may encounter with other scripts.
Let me know if you the following change fixes the error you are having. Thanks.
Regarding the use of the database, it’s up to you and is probably somewhat determined by what types of analysis you ultimately want to do. There’s obviously some overhead to getting the MySQL database installed and setup for AMET on your system, but it does provide a great deal of power to tailor analyses. The database allows you to more easily subset your data (e.g., by date, region, obs/model values) and do comparisons between multiple simulations and, in my opinion, makes it much easier to organize and access a large number of simulations.
So, it really comes down to what you want to do. If you envision using AMET for numerous model simulations going forward, it may be worth it to you. If you’re just trying to get basic statistics and plots for a couple of simulations, than working directly from the csv files may be sufficient.
I’m happy to discuss further with you about if you want. We can take that conversation offline. Just let me know.
I’d throw in the question of how much control do you have over your system as in root access?
If you do, setting up a mysql database on most top linux distros is quite simple. I’ve done it multiple times recently on a home linux network for my personal business and on AWS for AMET. Literally a few steps.
