I don't eat meat any more but throughout my weight loss (non-vegetarian then vegetarian), I'd still go to Red Robin once in a while, I'd just fit the calories in. Even now, I may or may not go to Red Robin (they have a veggie burger and a whole wheat bun). I'd just change what went on my burger and figure it into my calorie counts. If you are a calorie counter, I'd suggest dropping the mayo and the cheese and filling up on other condiments. I'd even put avocado on (good sub for both cheese and mayo). And even though the fries are 'limitless', I never got refills on the fries or I'd get the side salad instead.
You can't out-exercise poor eating habits.