I hear ya. I've lost 2 lbs. in 4 weeks. Of course, I don't always make the best food choices, and because my fitness level was practically non-existant I've had to start slowly with exercise and add to it gradually. It's very frustating at times but I'm trying hard to stay positive and accept the fact that this is going to be a long uphill road for me! Some days the only thing that I can be happy about is the fact that at least I've stopped gaining, which for a while there I was doing at a pretty rapid pace!
Aphil is right; 1/2 to 2 pounds a week is a healthy rate. I read somewhere that the body can only burn a maximum of 1 to 2 pounds of fat a week, and if you lose more weight than that you're losing water or muscle tissue or both, which in the long run is not going to help you at all.
Hang in there - you're doing fine and you can do this! Good luck.