I think it does come down to calories for weight loss... HOWEVER,
you can't do that forever. Those hashbrowns will NOT fill you up like an bowl of high fiber cereal and a piece of fruit with equal calories.
Just chalk it up to experience and lesson learned.
This is not to say that you can never have hashbrowns. Maybe next time make it just one hashbrown with a white egg omlet. Or maybe some homemade hash browns from left over potatoes?
I found the first month was difficult because I was experimenting a lot as to what "calorie worthy"
I think at least once a day I have something that is not calorie worthy but is still yummy and is in MODERATION. That is what keeps me on plan
Good Luck... and this was an excellent question!!