Definitely don't go hungry! You never have to do that on this plan, which is why I am still on it. Remember the 4.5 cups of veggies/day is a MINIMUM amount! Have as much as you want. And have some protein to go along - some cheese, yogurt, milk, some beans with cheese and salsa...
Life's a journey, not a destination.
It's easier to stay on plan than to get back on plan.