I was very reluctant two years ago, when I first got serious about all of this (not that I haven't been serious in the meantime, but hey, you don't gain 9 lbs back without slacking off a little, so maybe I was semi-serious only?). Right now, I've come to terms with the fact that this was a hindrance as well. I find myself annoyed at not having small enough clothes now, so I assume that the contrary would be annoying as well, and since I have no money to buy plenty of fat clothes, then that would leave me with my back against the wall. Fine with me if this is what it takes to NOT regain the weight back!
On the other hand, my fat clothes are almost all worn threadbare, so it's not such a biggie any more to throw them away...