So how much do you weigh? I don't see that anywhere here. If you way 120 lbs and have lost 1 lb in 3 weeks, I'd say that is normal. If you weigh 300, then I'd say there is something wrong.
I found that by changing my macronutrients around, my weight started to fall off, eating the same amount of calories. I eat a high amount of carbs but I was eating low fat, I started adding more avocado and nuts into my diet and I started to lose weight. I didn't change around the other things too much. I still eat bread, grains, fruit as well as legumes as my main source of protein. It was just ensuring that I added fat that made the change for me.
I would say you need to play around and figure out what works for you.
You can't out-exercise poor eating habits.