Honestly, I'm calorie counting so I don't have to give up anything. As long as I account for it I can have it. But if I was on a restriction diet then I would never give up sugar in my coffee. I can't stand artificial sweeteners in coffee. I like them in tea, but don't put nothing but sugar in my coffee