Who said you can't have them? I would venture to say that none of us got overweight by having sweets or not exercising regularly. No, we got overweight by having too many sweets and not exercising at all. Work on portion control and moderation rather than in cutting them out completely. Perhaps set aside just one portion as a treat for reaching certain goals you set.