Yeah, I really do catch myself thinking like that sometimes... it can be so hard not to! I just turned 23, and have been losing and gaining the same amount of weight over and over again for four years now. The ONLY way I can stay focused is to tell myself that this is the LAST time I'm losing it: the weight is never coming back, and I'm going to continue gaining lean muscle mass and increasing my fitness level, indefinitely, for the rest of my lifetime!
It might be a great idea to come up with some sort of positive message that you can repeat to yourself every time you find yourself worrying about the future. You might also want to try visualizing your future self the way you'd love to be, rather than the way you fear you may be. I know that helps me when I do it!
Best of luck with everything. You're certainly not alone in those worries, just try not to dwell on them