I lost about 10 pounds (all I needed to lose) about 10 years go with only exercise (weight training, not too much cardio), or at least, minimal attention to diet. It doesn't seem like it would have worked this time, but then again, this time I had much more to lose, had carried the weight a lot longer, was older, and not working out as intensely.
In my mind, the benefit of working out is twofold. First, I am building lean muscle, which will burn more calories for me throughout the day. Second, when I am working out regularly I am much more conscious about what I put in my body. I am better able to focus on choosing foods that nourish, rather than just what I am craving at the moment.
YMMV is a netspeak shorthand to say, "This is my experience/opinion, but it may not be yours."
Seriously, when I first starting trying to lose weight, I exercised most days around 300 calories a day (roughly, of course-YMMV . I'm 5'1", so my calorie intake to a pound a week would be around 1600 calories a day (roughly 1800 base and daily activity plus 300 exercise=2100, minus 500 per day deficit for a pound a week weight loss). Imagine if I weren't exercising: I would still need the same calorie deficit (around 500 calories deficit a day) but without exercise this would mean eating only 1300 calories a day. It can be done--people on 3FC can do that, and do it daily, and I've done it before, on days when I wasn't particularly hungry. But would I rather exercise and eat 1600 cals a day? Or would I rather not exercise and eat 1300 cals a day? 300 cals makes, for me, a big difference in enjoyment. And really, exercising has made me much more toned than I was before--I tried the 30-day Shred video after a couple months of doing other exercise instead and realized I had to step it up to level 2. Woot! but that wouldn't have happened without exercise. So in my experience, yes, exercise definitely makes a difference, and in my opinion it makes everyone healthier. But YMMV.
In my experience, my diet is far more important than exercise when it comes to the number on the scale. I lose weight when I restrict my calories, whether I exercise or not.
BUT, exercise has still been critical to my weight loss:
1. There's that whole issue with losing fat vs. losing muscle. I don't want to lose muscle. By strength training, I actually increase my muscle mass. Even when you are just sitting on the sofa like a slug, muscle burns more calories than fat. A LOT more calories. So maintaining or increasing my muscle mass sets me up for maintaining my weight loss. Not all exercise is equal in terms of this benefit--you have to be strength training, you won't get this benefit from cardio.
2. Then there's the metabolism issue. Eat less and your metabolism slows. Exercise helps by increasing your metabolism. This is also critical for maintenance. I know so many people that lost weight only to gain most, all, or more of it back. I am not going to be one of them and that means exercising to keep my metabolism up.
3. Exercise is key in terms of controlling my diet. I seriously do not know how people diet without exercise. When my exercise is off, I have terrible cravings and it so much harder to stay on plan. I've had days where my eating was spinning out of control, I went into the gym to work out and left with my eating completely back on track. I don't know why it helps, but it does. I've read that exercise is an appetite suppressant.
I HAVE to exercise for weight loss/maintenance. Partly it's because of the extra calorie burn, muscle maintenance, etc. But mostly it's for the psychological benefit that exercise brings. It makes me feel good about myself, energized, and somehow more in control. When I exercise regularly, my diet mostly falls in line too. When I fall off the exercise wagon, then I am much more inclined to eat badly too. Maybe that's not true for everyone, but it certainly is true for me.
This is just my take on things...
I think you need to exercise to make your body capable of loosing weight. Working out builds muscle...you need muscle to burn fat. Not to mention all the energy you gain by working out. And counting calories isn't what strengthens your heart That's the difference between loosing weight and being healthy.
. I'm 5'1", so my calorie intake to a pound a week would be around 1600 calories a day (roughly 1800 base and daily activity plus 300 exercise=2100, minus 500 per day deficit for a pound a week weight loss). Imagine if I weren't exercising: I would still need the same calorie deficit (around 500 calories deficit a day) but without exercise this would mean eating only 1300 calories a day. 
It can be done--people on 3FC can do that, and do it daily, and I've done it before, on days when I wasn't particularly hungry. But would I rather exercise and eat 1600 cals a day? Or would I rather not exercise and eat 1300 cals a day? 300 cals makes, for me, a big difference in enjoyment. And really, exercising has made me much more toned than I was before--I tried the 30-day Shred video after a couple months of doing other exercise instead and realized I had to step it up to level 2. Woot! but that wouldn't have happened without exercise. So in my experience, yes, exercise definitely makes a difference, and in my opinion it makes everyone healthier. But YMMV. 
