Honestly I had (have) the same issue, but when you think about it logically it does not matter if you're the one struggling to keep up, or the sweatiest, or the reddest, or whether you feel like a big blog of jelly compared to the lithe flexible people next to you (my own fears here

) because you are there to improve your fitness, you are there to change your health. It's not about keeping up but maintaining a pace that is conductive to improving your fitness, it should not be a competition (unless that really drives you). I think the gym is the only place I don't suck it in as i'm not there to look good, but to push myself a little more than I do whilst running. Having said that I haven't been since I came to live with the parents over summer, waiting to get back to uni and really get into it.
I do know some people who have made negative comments about the larger people at the gym (forgetting that I am one of those larger people ...) but to be honest their opinion means diddlysquat. I'm not sure what they'd rather people do, the gym is there so out of shape individuals can become a little less squidgy
Gabe really said everything I wanted to say in a far more eloquent manner.