You're at a great weight for your height. I've heard the same comment and I weigh 175 lbs. which I tell them and they say, "oh but your tall." All I say is, look at a bmi calculator, I'm def going to be on the higher end of the scale, but I can lose a few more.
People just need to shut it.

For 5'3" BMI of 20 would be 113 pounds

Really hard to guess, you look great though.