If you are that tall and wearing a size 4, do you really want to get too much smaller? If you drop to a 2 and then a 0, what happens next? Do you disappear?
I would say forget about the number on the scale - if you are happy with yourself, then that's all that matters.