I'm just adding layers of information/confusion so feel free to skip it if you want.
Another thing to keep in mind is that if you're exercising, your body gets more efficient over time... both in that particular exercise (ex. running, biking) and in general getting-blood-and-oxygen-to-the-right-places type stuff. Therefore, as you get fitter, you'll burn LESS calories doing the same exercise at the same speed/resistance/etc. Most people will handle this without consciously thinking about it, by bumping up the mph on the treadmill or whatnot as they FEEL able to do more, so that's not really a problem.
One caveat is that building muscle means you'll burn slightly MORE calories -- muscle burns more calories than an equal amount of fat just during day-to-day activities (it takes more resources to maintain). It may not be a noticeable difference, but it's just another reason that strength training is important!!