M3diff tool help

This has been a long thread. Do you now have a file for 5/1 that is a similar to the other dates or is this still a problem?

If you are still having problems, it would be helpful to attach your smkinven log.

The warning message which is tripping up the log analyzer is:

0,False,1,‘WARNING: Duplicate source in inventory. Will store only one value for’,/home/Michael/Documents/SMOKE/2016setupEPA/2028fh1_16j/intermed/ptegu_20160501/logs/smkinven_ptegu_20160501_may_daily_2028fh1_16j.log

All other messages in the log analyzer, including the “DEFAULT_CONV_FAC_ERROR” and “MAXERROR” messages, are okay. The first column in that level 1 file indicates a message priority. Anything with a 2 or 3 is a known common warning listed in our “known messages” file. Anything with a 0 or 1 is generally worth looking into. 0 usually means the warning is not known, and that is the case with this “duplicate source in inventory” warning.

We aren’t familiar with this error, so that is why your log file might be helpful.

Hello @eyth.alison

Yes, I have a file 5/1 which size are same as other dates and EPA.

Please see the smkinven log files smkinven_ptegu_20160501_2028fh1_16j.log.txt (35.6 KB) smkinven_ptegu_20160501_may_daily_2028fh1_16j.log.txt (480.0 KB)

Thank you

Your file is probably fine if it’s size is fine. We checked our logs and found that we saw a similar warning in our processing for the 2028fh1 run. The source in question doesn’t actually have any duplicate records in the inventory, so we think this may be a superfluous warning message. This could be added to the known_messages list so it doesn’t trip up the log analyzer going forward, but for your processing, you should be OK if the file size looks fine.

1 Like

Hello @cjcoats @bbaek @eyth.alison

Looks like I am able to run it. Here is what I did:

  1. I used inlineto2d tool to modify both (EPA & My run) emission files for 20160501

  2. Then I ran m3diff infileA infileB DEFAULT, (provides A-B)
    which gives me m3diff20160501de.txt (586.5 KB) file without an error. I checked MAX A-B value , most of the are zero , but some values are not. Such as top three values are:
    3.34E-06
    9.54E-07
    9.54E-07

Is this an issue?

  1. Then I tried to run m3diff for ‘non-default’ option (A-B)/RMS , and I get the error " A:B nowhere defined" m3diff20160501mo.txt (9.5 KB) the part of the terminal output is below:

    Value for IOAPI_CMAQMETA not defined; returning defaultval ': ‘NONE’
    Value for IOAPI_CMAQMETA not defined; returning defaultval ': ‘NONE’
    Value for IOAPI_SMOKEMETA not defined; returning defaultval ': ‘NONE’
    Value for IOAPI_SMOKEMETA not defined; returning defaultval ': ‘NONE’
    Value for IOAPI_TEXTMETA not defined; returning defaultval ': ‘NONE’
    Value for IOAPI_TEXTMETA not defined; returning defaultval ': ‘NONE’

    “OUTFILE” opened as UNKNOWN(R-W)
    File name “OUTFILE”
    File type GRDDED3
    Execution ID “???”
    Grid name “12US2”
    Dimensions: 246 rows, 396 cols, 1 lays, 1 vbles
    NetCDF ID: 196608 opened as READWRITE
    Starting date and time 2016122:000000 (0:00:00 May 1, 2016)
    Timestep 010000 (1:00:00 hh:mm:ss)
    Maximum current record number 0

    Processing . . .

    Value for IOAPI_LOG_WRITE not defined;returning default: TRUE

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 0:00:00 May 1, 2016 (2016122:000000)
    Could not write “PSO4” to “OUTFILE” for 0:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 1:00:00 May 1, 2016 (2016122:010000)
    Could not write “PSO4” to “OUTFILE” for 1:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 2:00:00 May 1, 2016 (2016122:020000)
    Could not write “PSO4” to “OUTFILE” for 2:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 3:00:00 May 1, 2016 (2016122:030000)
    Could not write “PSO4” to “OUTFILE” for 3:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 4:00:00 May 1, 2016 (2016122:040000)
    Could not write “PSO4” to “OUTFILE” for 4:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 5:00:00 May 1, 2016 (2016122:050000)
    Could not write “PSO4” to “OUTFILE” for 5:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 6:00:00 May 1, 2016 (2016122:060000)
    Could not write “PSO4” to “OUTFILE” for 6:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 7:00:00 May 1, 2016 (2016122:070000)
    Could not write “PSO4” to “OUTFILE” for 7:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 8:00:00 May 1, 2016 (2016122:080000)
    Could not write “PSO4” to “OUTFILE” for 8:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 9:00:00 May 1, 2016 (2016122:090000)
    Could not write “PSO4” to “OUTFILE” for 9:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 10:00:00 May 1, 2016 (2016122:100000)
    Could not write “PSO4” to “OUTFILE” for 10:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 11:00:00 May 1, 2016 (2016122:110000)
    Could not write “PSO4” to “OUTFILE” for 11:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 12:00:00 May 1, 2016 (2016122:120000)
    Could not write “PSO4” to “OUTFILE” for 12:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 13:00:00 May 1, 2016 (2016122:130000)
    Could not write “PSO4” to “OUTFILE” for 13:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 14:00:00 May 1, 2016 (2016122:140000)
    Could not write “PSO4” to “OUTFILE” for 14:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 15:00:00 May 1, 2016 (2016122:150000)
    Could not write “PSO4” to “OUTFILE” for 15:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 16:00:00 May 1, 2016 (2016122:160000)
    Could not write “PSO4” to “OUTFILE” for 16:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 17:00:00 May 1, 2016 (2016122:170000)
    Could not write “PSO4” to “OUTFILE” for 17:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 18:00:00 May 1, 2016 (2016122:180000)
    Could not write “PSO4” to “OUTFILE” for 18:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 19:00:00 May 1, 2016 (2016122:190000)
    Could not write “PSO4” to “OUTFILE” for 19:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 20:00:00 May 1, 2016 (2016122:200000)
    Could not write “PSO4” to “OUTFILE” for 20:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 21:00:00 May 1, 2016 (2016122:210000)
    Could not write “PSO4” to “OUTFILE” for 21:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 22:00:00 May 1, 2016 (2016122:220000)
    Could not write “PSO4” to “OUTFILE” for 22:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 23:00:00 May 1, 2016 (2016122:230000)
    Could not write “PSO4” to “OUTFILE” for 23:00:00 May 1, 2016

    —>> WARNING in subroutine WRITE3
    Variable PSO4 not in file: OUTFILE
    M3WARN: DTBUF 0:00:00 May 2, 2016 (2016123:000000)
    Could not write “PSO4” to “OUTFILE” for 0:00:00 May 2, 2016
    netCDF error number -40

    —>> WARNING in subroutine RDTFLAG
    Error reading netCDF time step flag for infileB
    M3WARN: DTBUF 1:00:00 May 2, 2016 (2016123:010000)
    netCDF error number -40

    —>> WARNING in subroutine RDTFLAG
    Error reading netCDF time step flag for infileA
    M3WARN: DTBUF 1:00:00 May 2, 2016 (2016123:010000)

    —>> WARNING in subroutine M3DIFF:DIFFSTEP
    Read failure: file infileB variable PSO4
    M3WARN: DTBUF 1:00:00 May 2, 2016 (2016123:010000)

    —>> WARNING in subroutine M3DIFF:DIFFSTEP
    Read failure: file infileA variable PSO4
    M3WARN: DTBUF 1:00:00 May 2, 2016 (2016123:010000)

    —>> Normal Completion of program M3DIFF
    M3DIFF completed successfully

I would think that differences that small would be OK.

@eyth.alison thank you so much for the response.

Would you please suggest me why ‘non-default’ option (A-B)/RMS is not working? .

As I said earlier, you don’t want an output file (which you have not set up properly anyway, which is the cause of your errors); what you want is the grid-statistics report to STDOUT, using difference normalized by pointwise mean (menu item 11), not difference normalized by joint root mean square (menu item 13).

@cjcoats Thank you so much. I got it. I just set output file ‘N’ and it gives me output in terminal for option 11.

Hello @eyth.alison
m3diff tool gives difference and top three are 3.34E-06, 9.54E-07, 9.54E-07. You mentioned this tiny differences are okay ( because of different machines?) and I understand that. But I have a question:

What is the limit / threshold of differences to say ‘difference is not significant’ or ’ difference is significant’?

This gives the relative difference: 3.34E-06 says that the numbers agree to about one part in
1/3.34E-06 = 299401, i.e., the agree to about 5.5 significant decimal digits.

Do a round-off error analysis of the calculations involved,.

The inputs are IEEE REAL*4, accurate to the binary equivalent of almost 7 significant decimal digits. Mathematics (numerical analysis) problem: how large do you expect the round-off error to be? You certainly cannot count on the 7th significant digit in the results; how much round-off error you can expect depends on the nature of the calculation… agreeing to 5 decimal digits is reasonably to be expected.

1 Like

Thank you for the explanation @cjcoats . I greatly appreciate you.

Thank you for the suggestions. I greatly appreciate that. .

1 Like