Hiya, i think most people can relate to what your going though. I definitely can. I've been dieting on and off for the past 7 years just the way you described it and it didn't help that i worked in sainsbury's for a year with a stick thin coworker that ate huge amounts of food and tried to copy her, that was a major fail. Then fell pregnant in 2012 and used that as an excuse to eat any and everything that crossed my path, surprising during the moments like my pregnancy when i didn't stress about food i lost the most amounts of weight.
I definitely don't have a solution for you since i'm currently doing the trial and error thing and gradually still putting on more weight. I'm trying out the 5:2 diet at the minute and trying to increase my water intake (it's really bad at the min) and forcing myself to regularly exercise. Thats about all i'm doing.
I feel like if we take little steps at a time, like change one thing at a time rather than trying to do a drastic change it'll be more beneficial in the long run. Maybe write down everything you want to change e.g. cut out/down sugar, work out, drink more water or eat more vegetables then pick one and do it consistently for a week or month then it starts to become a habit so you pick another to do consistently then before you know it you've picked up all these healthy habits that have become second nature to you.
Hope i've been at least a little bit helpful