It also depends on where the weight is coming off. For example, if you lost mostly on your upper body, you might go down a shirt size but stay the same in pants. Or the other way around. If a lot came off around your waist but your hips/butt and your bust were the same size, you might not go down sizes at all (but I'd bet you'd look better in your current size!).
Congrats on the 20lb loss, that's awesome!