I gotta say, I would give you just about the exact opposite advice as Palestrina, but that is probably because there is a such a wide-variability in how different things work for different people. It isn't that I think her advice is wrong at all, it is just wrong for me.
I no longer truly believe in "everything in moderation" as a concept -- mostly because I believe our food environment is so crazily-over-engineered. If the foods we craved were unmodified and mostly in a whole natural state, I believe I might be able to buy into "everything in moderation".
But many of the foods I crave are like the ones you describe LovelyLeah -- for me it is Swedish Fish -- which are made of something like straight sugar, corn syrup, upside down sugar, more sugar, something really artificial, and more corn syrup.
Emotionally eating a food like that becomes very difficult for me to control because my body just doesn't react well to that combination of stuff (apparently), and it will put me on the road to having to fight off the urge to binge.
I'm not saying I don't EVER eat them, or my other triggery-type-foods, because I do so loooove them, but it always comes at a price that isn't usually worth paying in retrospect. It is playing with fire -- so I tread lightly.
I have found that
for myself, in my experiment of one, trigger-food-avoidance (if you want to call that 'restriction', I'm fine with that) doesn't lead to binging. Eating trigger foods does result in either having to fight the binge urges (which is exhausting in every way) or actual binging.
It has been better for me to find other ways to emotionally soothe myself -- because diving into a bag of Swedish Fish just wasn't my best option overall. Not saying that it is easy to develop a different emotion-coping arsenal, but it is doable and perhaps worth the effort over time.
Your mileage may vary.