I generally responded with something like "Quite a bit!" and then turned the subject to something else.
My problem with it, aside from the general rudeness of asking somebody a personal question, is that once you've lost a great deal of weight and you tell somebody the number, you can see the look on their face as they do the math and try to figure out what you weighed to start. You don't have to answer the question many times to figure out that the slightly vacant look on their faces means their internal conversation is "Well, she's lost X, and she's still kind of chunky, so she must have weighed at least X."
My answer is different if the question comes from somebody clearing looking for help with their own weight problem, and it becomes very easy to tell if it's this or simply curiosity.

