I agree with you on tossing the scales.
I have been exercising for about 3 years, lots of riding my bike and walking and I have gone down a full size in most of my clothes and even less depending on the size.
I can really notice a big difference, but the scale has not gone down at all in about 3 years.
So I am like you I go by my clothes and I keep exercising and I feel very good when I do a lot of exercising.
Suzanne