My vote is, "what someone else thinks of me is none of my business." It's so empowering not to give a f.
Good, bad, it would just be speculation. That speculation would reveal ourselves, but not necessarily that guy. I find that our interpretations of other people are often like Rorschachs to see our own inner state. We're more prone to interpreting judgment when we're feeling insecure, flirtation when we're feeling feisty, etc.
If he didn't harass you or ask you out, he's meaningless.
Save your attention for the guys who give you something good to notice.