I have definitely been there. Of course for me it wasn't just the occasional goodie that set off my guilt. It was everything. Every time. Whether good for me or bad, whether in moderation, or not, I felt guilty. That was back when was at my heaviest (220) and I knew nothing about eating well... Actually I knew nothing about health in general, and I thought that in order to lose weight I needed to, well, not eat. So of course, when I did eat, I felt guilty.
Thankfully those days are behind me now. I know what and how to eat, and everything I put in my mouth no longer triggers those emotions. I enjoy food now, and the guilt free feelings!
But, after saying all that, I have come to realize that I am not at the point where I can let myself have that occasional goodie. Sweets are my weakness, and right now, it is all or nothing.

I am hoping someday in the future that will change for me as well.
Congrats on the huge step you have made!!