You need to eat more frequently to keep your metabolism up. Most sources I've seen/heard/read say ideally you should not be going more than 2-3 hours without eating, and you should split your calories up over 5-6 small meals to maximize your metabolism and stabilize your blood sugar. Plus, you don't have to deal with hunger, which is a bonus. Depending on how much granola you eat and whether you eat it with milk, you *may* be within an acceptable range of calories, but probably not. And you're definitely not getting sufficient levels of the macronutrients you need if you're only eating those few foods in such small amounts.
And granola is usually high in sugar and not too high in fiber - not as high as a whole grain high fiber cereal anyway. I eat shredded wheat - high fiber and 0 sugar. By eliminating the high sugar granola and switching to a lower sugar higher fiber cereal you'll not only feel fuller longer, but you'll also have more calories to budget throughout the day for more meals.
But yes going 8 hours with no food is bad for your metabolism, bad for blood sugar, and I'd imagine it's uncomfortable to be so hungry!
Think nutrition/heath first, weight loss second - the one follows the other for long term success.