This is such an interesting topic and I totally have those feelings, even though I'm still in the 220s. I'll see myself in the mirror at the gym and think, "Where am I? I'm wasting away!" Hardly!
Wendyg, I have totally been the same way, gaining 100 pounds with EACH of my serious relationships and then dropping the weight pretty easily after the break up. I was a size 10 when I met my husband. In January this year, I was a size 22/24 and felt the same way that you described (I'm finally back in a 16, though!). It's so hard to carry that guilt.