Everything in moderation, including moderation. Some days no matter how hard you try and what you want the diet just will not work and dies a horrible screaming death, riding you to the ground with guilt. Don't let it. Yes everyone falls off the wagon sometimes, yesterday walking through the grocery store I smelled pie, and I would have gladly ripped off someones arm and beaten them to death with it to have that pie. I did not break, though there was much drooling and quite a bit of profanity...and I think my resentment of lettuce has gotten a little worse. But the important thing is I survived, and although i may dream of pie for a couple of days, I stuck with it...ok, so I snagged a couple of hershies kisses when I got home, but it was not a faceplant in a pie, so I can live with that.
As for your husband...well, anyone who will blame you for not being able to keep his own pants up is not worth your time, and sure as **** not worth your effort or stress. I know it is hard now, and it will be hard, someone you have spent that much time with is not easy to get over. Been there, done that, got the t-shirt. And as much as it hurts, it was a dick move of him to use your weight to justify him being a big slut. If you want to make changes in your life they should be for you, so you will feel better, and so later you can flaunt it and say "yeah you gave this up, feel the burn, ouch!" but that is just the spiteful side of me showing. I have always been too vindictive for my own good.
Best of luck with things and drop me a line...or check out my blog I have on here, it might be something for you to look into doing as well.