I have similar issues, but more with my friends =/ My mother went through gastric bypass 10 years ago, so she understands the struggles and is supportive, and tries to make healthier choices for dinner when I come over.
My friends, however, are completely the opposite. There is a set of twins, both are 6'4, and one is probably closing in on 450 lbs, while the other is probably around 375 lbs. I worry about them because they are so heavy, and they are only about 34 years old. What kills me is they come over to my house with a bucket of chicken from KFC, or Burgers from Five Guys (one of my weaknesses), and sometimes they goad me with it.
I don't think they realize how hard it is for me to say "no". And I don't think they realize how often they really do it. Sometimes I think them offering is just because they want to make sure I'm included, but more often than not, I take it as them taunting me with things I can't have.
Like the others have said, learning to say "no" is the only way to really deal with it, and sadly it's one of the hardest hurdles to get over. And if you do binge, don't let it get you down! Wake up the next day and keep going with your diet! Everyone stumbles now and then, we just need to make sure we are picking ourselves back up again =) You can do this!!!!