I never really worried about the age thing in general, but since hitting 50 this year, I'm a bit more conscious of it. I recall my mother losing a bunch of weight when she was in her 60s and she really did look old. But, she was a heavy smoker and was very sedentary, so I've just hoped that those factors were responsible for her appearance and put it out of my mind. As others have said, I'd rather feel good and be healthy than worry about my appearance. (Which, clearly, I haven't been too concerned with lately, as I let myself gain all this weight!)
If I wasn't convinced before, I am now. Just this afternoon, I ran into an old neighbor. Her son and mine became best buddies when they were 4 years old (they're now 21), so we still see each other maybe once a year. Today, she kept interrupting our conversation to say, "I really can't get over how good you look. You haven't aged a day since you lived next door!" Of course, that's far from true, but made me feel good none the less. The only response that I could come up with was, "I'm really happy these days." And I am.