We’re all composed a little differently. I’m about 165 5’7-8 ish and am in a size 8 to 10. If you look small for your weight, you probably have a lower body fat composition in comparison to what this person is thinking. I’d take what others say worth a grain of salt, unless they are your doctor or a qualified nutritionist whose advice you are seeking.
Echoing Jules above. Your BMI appears to be a touch overweight (by like a point) but it's not a perfect system and doesn't take into account things like a larger chest. The more important thing is are you happy? Is your doctor happy? If yes, then who cares what anyone else thinks? And hopefully you won't have any more people telling you their opinions about you. >_<