Format the output data of moves for smoke (moves2smkEF.pl)

Hi.
I ran moves 2014b (on windows) and used SMOKE_MOVES to prepare formatted data for smoke, but while running moves2smkEF.pl, I got the below error:

> C:\Users\KPS\Desktop\SMOKE-MOVES-master>perl scripts/moves2smkEF.pl -u moves -p moves -r RPD --formulas scripts/pollutant_formulas_AQ.txt --proc_agg process_aggregation.csv scripts/config.txt scripts/pollutant_mapping_AQ_CB6.csv

`

**

**> Unknown pollutant “CO_INV” in formula “CO=0.0357*CO_INV”**

**

`

config.txt (87 Bytes)
pollutant_formulas_AQ.txt (1.3 KB)
pollutant_mapping_AQ_CB6.csv (2.8 KB)
process_aggregation.csv (1.3 KB)

Could anyone help me?
Thanks.

no one ??? @cseppan

Nothing’s jumping out at me in terms of the input files you’re using. CO_INV is defined in the pollutant mapping file, so it should be fine in the pollutant formulas file.

I haven’t run the moves2smkEF script on Windows, so maybe something isn’t working as compared to Linux. As a test, try removing the formulas option from the command altogether ("–formulas scripts/pollutant_formulas_AQ.txt") and see what happens.

Also, what’s the version listed at the top of the moves2smkEF.pl script? Like this:
“# Version : 1.10”

One more follow-up, what version of Perl are you using?

perl --version

This is perl 5, version 32, subversion 1 (v5.32.1) built for x86_64-linux-thread-multi

Thank for response.
# Version : 1.10

This is perl 5, version 14, subversion 2 (v5.14.2) built for MSWin32-x86-multi-thread
Copyright 1987-2011, Larry Wall
Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Hi again.
I updated my Perl to the 5.32.0 version.
But it did not work and I got the same error about the formula.

What happens when you run the script without the formulas option?

@cseppan
On perl 5.32.0

perl moves2smkEF.pl -u root -p moves -r RPD --proc_agg process_aggregation.csv pollutant_mapping_AQ_CB6.csv config.txt
install_driver(mysql) failed: Can't locate DBD/mysql.pm in @INC (you may need to install the DBD::mysql module) (@INC contains: C:/Users/KPS/AppData/Local/activestate/ed12a3ae/site/lib C:/Users/KPS/AppData/Local/activestate/ed12a3ae/lib) at (eval 6) line 3.
Perhaps the DBD::mysql perl module hasn't been fully installed,
or perhaps the capitalisation of 'mysql' isn't right.
Available drivers: DBM, ExampleP, File, Gofer, Mem, Proxy, Sponge.
 at moves2smkEF.pl line 344.

On perl 5.14.2

DBI connect('2,Carbon Monoxide (CO),CO_INV,mass,;MOVES2014 Pollutant ID,MOVES2014 Pollutant Name,SMOKE Pollutant/Model Species Name,mole/mass,MOVES2014 Process IDs (optional)','root',...) failed: Unknown MySQL server host 'MOVES2014 Pollutant ID,MOVES2014 Pollutant Name,SMOKE Pollutant/Model Species Name,mole/mass,MOVES20' (11001) at moves2smkEF.pl line 344
Could not connect to database: 2,Carbon Monoxide (CO),CO_INV,mass,

Have you installed DBD::mysql so that Perl can connect to the MySQL database?

It looks like your Perl 5.14.2 installation is able to access the DBD::mysql driver, but Perl 5.32.0 isn’t.

Regarding the Perl 5.14.2 error, the various input text files don’t seem to be getting read correctly. From the files you shared earlier, config.txt has Windows line endings (CRLF) while the other three files have UNIX line endings (newlines). It looks like Perl adjusts its file reading based on platform, so I’d suggest converting all the files to have Windows line endings.

Thanks for your response.@cseppan
I fixed my Perl and solved the connection error but the recent error unknown co_inv there were. also, I ran it without the --formula. although it worked and I saw the completed processing message, the Mrclist file (output) was empty and RPD_smoke had not been produced.
so I decided to run it on Linux.it worked but in the processing step, my laptop froze.
I wanna know this work uses RAM or CPU?

The bulk of the work in the moves2smkEF script is queries in the MySQL database. I would guess that memory is more of a limiting factor than CPU. MySQL has various settings that control how much of the available memory it can use.

The rate-per-distance (RPD) processing is the most intensive due to the number of records that need to get processed. You could try one of the other modes, like RPP or RPH, which should complete much faster, just to make sure everything is working correctly.

1 Like

I am really thankful to you. :pray:
all problem solved. then I ran it again and I saw the complete message and …
but there was not any file produced. just the mrclist file was produced in the true path but it was empty.
I do not have an idea. :neutral_face:

Sat Jul 3 10:04:36 2021 Processing database isfahan2017_out…`

  • Completed drop tables at Sat Jul 3 10:04:36 2021

Starting rate per vehicle processing…

  • Completed ratepervehicle_smoke at Sat Jul 3 10:04:36 2021
  • Completed formula processing at Sat Jul 3 10:04:37 2021
  • Completed ratepervehicle_smoke export at Sat Jul 3 10:04:37 2021

@cseppan. Thank you a lot.
I founded out why the file was produced (but empty) and rateperdistance_smoke.csv was not produced.

In the Perl script in line382 was

MOVESScenarioID LIKE ‘rd_%’

But my MovesscenariID did not begin with rd.
Just this.

The moves2smkEF script is specifically designed to process MOVES results from the run specification files created by the runspec_generator script. runspec_generator uses a standard naming scheme for its MOVES scenario IDs, where the rate-per-distance names start with ‘rd_’.

There may not be anyone in town this week to dig into your question further.

Can you remind us – are you running MOVES and moves2smkEF on Windows?

How did you set up your MOVES runspecs? Did you use the script provided with the smoke-moves package?

Or did you set it up some other way? Moves2smk expects you to use the provided scripts.

Ok. I did not know about it.
Thanks.

Thank you for your response.
I used Moves2014b(rate mode) on windows so I need to change the format of them for use in SMOKE(lookup table). Then I used moves2smkEF.pl to do it but there were some problems with the format of files we used(windows could not read some input files), so I decided to run moves2smkEF.pl on Linux.
therefore, we ran MOVES21014b on windows and moves2smkEF.pl on Linux.

The runspec generator will take information about the meteorology and will prepare a set of runspec files for MOVES to be run an appropriate number of times to generate the range of emission factors that, once formatted, are used by SMOKE-MOVES based on the hourly gridded meteorology when it runs. The SMOKE-MOVES interface is here:

https://github.com/CEMPD/SMOKE-MOVES

The process is described further in the SMOKE Users’s guide which is linked from the above Github readme.