I can definitely relate to your post. When you lose an obviously large amount of weight, people notice and talk and it makes you feel a little like you're always on display. There is a girl at my work place who has made comments to other people that she's sure I was on drugs because "there's no way you can lose that much weight just by eating less and exercising"
Now mind you, to me she's never been anything other than complimentary. Oy. It can make you a little paranoid.
It seems to get better over time. I don't really find myself worrying about it too much any more. In fact with the high turnover at my workplace lately I'm surrounded by people who have never known me any other way. That's kind of weird in an entirely different way.