I'm not trying to be sarcastic here at all so please don't take it this way:
Have you ever thought that it's in your head? Body Image Distortion?
Also, as far as sizes go -- we all know how misleading those can be. Also, I have found that people that weigh more than me and are shorter than me wear a smaller size. How that works I'm not really sure. May just be how I carry my weight or I'm guessing is a height issue.
The important thing here is how you feel. Do you feel better than you did? If so... go with it. Plus, I'm pretty sure most guys aren't gonna complain about your booty or your boobs.