If I waited 3 whole hours after a meal before exercising I'd be eating my next meal
or get weak for my workout... I eat before going to the gym, unless I am working out at 5am then it's on an empty stomach.. I eat, but never a lot, a portion of carbs (half a cup of oatmeal) a portion of protein (4 egg whites)... when I come back I eat within 45-60 mins, again a portion of protein and carbs...
Don't let those time limits limit your workouts, what's most important is that you workout when you can...