I can remember very clearly when I felt just like you do except it wasn't just with my husband, it was with everyone in some way. If my mother was eating bread I thought she was trying to sabotage my low fat diet. If my friends were eating dessert I thought they were being insensitive to me. If my husband wanted chips I thought he was being a selfish jerk. I could go on. I felt this way constantly no matter what diet I was on. In truth I knew that I was the common problem because normal people don't get angry about food or defensive about diets, normal people don't seeth in anger because the pizza smells too good while eating chicken.
So while I can sympathize with you I also don't think your boyfriend has done anything wrong. You can't control what others do and you can't even expect them to be sensitive to this issue because this is really an issue you need to work through. I see things completely differently now since I've been doing IE for over a year and I can easily tell you that my husband can sit next to me and eat a bowl of chips without me having any kind of emotional response to it. I don't feel angry, sabotaged, scared of my cravings, or nervous around that food anymore. Potato chips are literally just an inanimate object when just a year ago they were my weakness, my nemesis, my comfort, the source of my self loathing, the cause of my fatness and my trigger.
I never would have believed it is I didn't experience it for myself but yes moderation is possible and it does lead to weightloss and more importantly peace of mind which is the main issue here.