I've also been reluctant to pick a goal weight. At the same time, the idea of set points really seems true for me. For much of my 20's, I weighed 150, give or take a few pounds. I thought I wanted to lose weight in a vague sort of way, but I was fit, and it was never that much of a priority. 150 suited me. But I went abroad for a long stretch in my late 20's and lost a bunch of weight--not on purpose, more circumstantial. And being in the 130s made me realize that was not where my body wanted to be, even if it's a very healthy weight for my height.
Sometimes I wonder what would have happened if I hadn't lost that weight. (And really, I wasn't trying. I was living in India, it was really hot for a lot of the time, I got sick, I fell in love--you know, the stuff that makes you lose weight sometimes.) Because when I got back, eventually, over 5 or 6 years, I gained it back and then some, all the way up to 180. (And this was after being steady at 150-ish for years.)
In my whole adult life, I've never maintained at a weight less than 150, or, really, more than 180. I went back down from 180 to 150 once, about 7 years ago, gained it back, discovered I had thyroid issues, stayed set at 180 for 3 years or so, and only this spring, when I realized I was going to be closer to 190 than 180 if I didn't watch out, did I decide I had to do something.
At this point, there's no way I'd set a goal lower than 150. That seems a recipe for failure. I'd be happy with another 10 pounds off. (And I feel and look good now, at 170, though I don't really want to stop here.) Because I've always exercised, I've never had any weight-related health issues, and my numbers have all been fine, even at 180.
I think the lesson for me is that I probably can't, realistically, get any lower than 150, but I also know that my body is comfortable at 180, even if I'm not. So the challenge is keeping myself in the lower end of the zone rather than the upper end. But I do have to say that knowing what my realistic lower end weight is helps in terms of thinking about reasonable weight loss now.