I had that same thing happen to me multiple times when I started hitting the gym and came to realize...SO WHAT? These people, intentionally or unintentionally were making me self conscious...when it should be the other way around.
Most of them have never been overweight and out of shape and have no idea how hard you are working to get fit. I know - I was one of them at one time. It would have been a LOT easier maintaining fitness all these years than it is for me now trying to retake my body back.
So be proud of yourself! Assume they aren't doing it to annoy you, move on/ignore them, and keep moving! Pretty soon you'll be in better shape than them.