# MPI\_ABORT was invoked on rank 3

**URL:** <https://forum.cmascenter.org/t/mpi-abort-was-invoked-on-rank-3/1635>\
**Category:** CMAQ\
**Created:** [July 11, 2020, 7:31am UTC](https://forum.cmascenter.org/t/mpi-abort-was-invoked-on-rank-3/1635 "2020-07-11T07:31:02Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Duyen](https://avatars.discourse-cdn.com/v4/letter/d/e274bd/32.png) [@Duyen](https://forum.cmascenter.org/u/Duyen)\
**Post date:** [July 11, 2020, 7:31am UTC](https://forum.cmascenter.org/t/mpi-abort-was-invoked-on-rank-3/1635/1 "2020-07-11T07:31:02Z")

</div>

I run CMAQ 5.2.1 when March 16 2017 at 01:00:00 to March 31 2017 at 18:00:00. But it success until March 16 2017 at 17:00:00.  
It have maseage that:

> > —\>\> WARNING in subroutine Read BCs on PE 000  
> > Applying fix to aerosol Boundary Conditions for aerosol modes. Turn on verbose\_rdbcon preprocessor flag to learnmore.  
> > M3WARN: DTBUF 20:52:30 March 16, 2017(2017075:205230)  
> > after HADV G 8.9076178E+02 A 7.0179551E+12 N 3.1367821E-01

* * *

MPI\_ABORT was invoked on rank 3 in communicator MPI\_COMM\_WORLD  
with errorcode 538976288.

NOTE: invoking MPI\_ABORT causes Open MPI to kill all MPI processes.  
You may or may not see output from other processes, depending on  
exactly when Open MPI kills them.

Can i fix it? Please tell me what happen.  
Thanks

---

<div class="post-metadata">

**Author:** ![cgnolte](https://yyz1.discourse-cdn.com/flex027/user_avatar/forum.cmascenter.org/cgnolte/32/53_2.png) [@cgnolte](https://forum.cmascenter.org/u/cgnolte)\
**Post date:** [August 11, 2020, 5:25pm UTC](https://forum.cmascenter.org/t/mpi-abort-was-invoked-on-rank-3/1635/2 "2020-08-11T17:25:48Z")

</div>

```
MPI_ABORT was invoked on rank 3

```

Go look at the log file for processor #3, it should have the name beginning CTM\_LOG\_003. Near the end it should have an ERROR ABORT message indicating what went wrong.