Well blueeyes, I just had a piece of leftover birthday cake and some pizza last night. Yet I'm still holding my loss. Because? I don't do it ALL the time, and I exercise regularly.
I'd say that YES your fear is true that if you DIET (I think that's such a nasty word), you may gain it all back. I did, many times. Instead, don't diet, but change your DAILY diet. Don't have pizza every week, but once a month, get the chips out of the house, cut back the burgers and go with grilled chicken, cut out white pasta and bread and replace them with whole grains. Just learn a new way of eating daily, then have the pizza, the cake, the chips ocassionally.
You should also learn to incorporate SOME exercise into your daily routine. You can lose without exercising, but you can lose BETTER with it. IOW, you can tone up better, look better and feel better. And, it gives you a little more wiggle room to indulge ocassionally than if you just dieted alone.
Start with a few daily changes at a time. Like, increase your water intake and cut out sodas if you drink them and maybe switch to whole grains. Once you've done those for a couple of weeks and think you can sustain them, add others, like increasing your intake of fresh fruits and veggies, or adding a 30 minute walk just 3 days a week. And keep building onto that.
Then once you HAVE lost the weight, there's no diet to quit. You've made a lifestyle change that will stick with you. And there's nothing then to be afraid of.