At least if you hurled the cake out the window, nobody would have to worry about who is going to eat it. :p

If other people are irritated or upset about you not eating as much food as they want you to, that is their problem. If they can't understand that you need to stick to this diet, again, that is their problem. Just keep refusing that cake, or leave the room, and as long as you're happy with the food choices you've made, that's all that matters.