I used to be the 'smaller' friend... my heavier friends would compare themselves to me and say we are the same size. I think it's because I'm a pear and have huge thighs that make me look chunky :/ But I've used those instances as motivation to lose weight...
So I guess that's the only thing to do, OP! Push yourself more and be as small as your friend
Now I look lighter than I weigh and get comments like "you look normal now", "stop losing weight", "don't diet anymore"... even if I'm still overweight! I hate those more!!! >.<