No, some men love the athletic look. Mine does.
However, Kassie, if I recall - you are very young and you are not in very serious or long-term relationships, right?? It is hard for everyone to not 'take to heart' what our significant other thinks but there are times where it deserves more emphasis than others. If you are dealing with young, casual relationships (as opposed to say a marriage with children), there's really no compromises to be made or much view points to be considered. You do what you like and that's that.
Visible abs are very difficult for women to obtain, I had them at 5'5 and 118lbs. It's not a topic to even discuss with him at this point

And it shows his ignorance of female body fat % figures anyways. Please don't guide your journey according to the wishes of some young guy, this is about you.