I by no means give up my coffee...BUT, I don't overindulge and pump myself up with caffeine all day either. I have my coffee, but I have two cups a day. I am a bit of a coffee snob, so I buy whole bean and grind it myself...and if I am in a hurry I use my Senseo.
I usually take it black with a packet of Splenda or other artificial sweetener. This keeps the calories down to about 5 per cup. I drink plenty of water and herbal tea through the day, and if I do have a diet Coke I limit that to once per day-or all together a total of 3 caffeine drinks per day max.
I don;t advise drinking a pot of coffee every day...but I don't think you should have to cut it out completely. Caffeine I think is fine in MODERATION.
I do think you have to watch out what you put into it though. If you like the flavored creamers (either liquid or powder) then measure them out if you use them, and switch to sugar free variety to lessen the calories. If it is 30 calories a teaspoon for instance, and you don't measure and actually use a TABLESPOON, then you are consuming 90 calories of creamer a cup, not 30-so measuring is important.
If you go to a coffee house and get a latte, get a SMALL and request a skim milk latte. If you want to get the calories for that, you can go to the Starbucks website-they have a calorie calculator where you can add the calories of your latte-and change the variables, such as what size, and whether you are using skim, regular, or soymilk, etc. Since a latte is mainly milk with a shot of coffee...buying a large size is actually doubling the calories since you are doubling the milk. However, I think the healthiest coffee drink at a coffee house is a small skim milk latte, sweetened with artificial sweetener if you need it. You are actually getting a serving of skim dairy with it...but it is a full serving, so you NEED to be getting the calories for this and record them.
I don't really recommend all of the foo-foo coffee drinks loaded with syrups, whipped creams, and all that stuff. If it looks like a milkshake or smells like a dessert-it most likely has the calories of one.