I think you should both stay and get over it.
We've had spats before, generally when someone is not well and it sounds like to me anyway that neither of you are feeling very well.
If you want out, then that's that. I'd rather you didn't, but don't start a fight over who is going and who is staying.
There is no bad guy here, I am not blaming anyone for an outburst of any description. But it is up you (either or both) you are free agents