My first week calorie counting I lost 7 lbs it was glorious..LOL now I average between 1.5 lbs to- 3 lbs! on rare occasions I don't loose at all... or just a few ounces..
I would only up my calories if the 4 pounds a week is consistent for a month or so.