I had a similar experience a few weeks back. I told someone I know that I was going on a diet and he turned to me and said "Yeah, you need it." The thing was, he was joking but it didn't matter to me because I was very hurt by it. Joke or not, it would always effect someone who is self conscious, like I am. To me, often a lot of people's jokes are what or how the feel about someone. I will admit that when I got home, I couldn't stop thinking about the comment and I did get very upset. The morning after I told myself that while I can change my weight, he can never change the way he feels about people or the comments he makes to people. And he can certainly NEVER take it back.
It's hard to get over hurtful comments, you can do it though.
Try to think like I can. You can change your appearance but he can never change his ugly personality. From your stats, you are doing brilliantly already and should be very proud of yourself.