I count everything I eat, except for things like spices, chewing gum, and zero-calorie tea. Good, bad, ugly, I have to count it.
Personally, I also try to keep one eye on my nutrient balance and make sure I'm not consuming too many carbs (I figure if I'm getting 1/3 each of protein, fat, and carbs, I'm probably doing all right - others use different ratios, like for Atkins/low-carb or low-fat diets), and also my sodium intake. This is where your "bad" foods will really show up.
Part of the point of counting your not-so-good foods is that they are often high in calories but low in nutritional and fullness value, so when you only have so many calories per day, you learn to allocate them to things that will get you more bang for your buck, so to speak. 250 calories of Dove bar would probably keep you full for an hour, whereas 250 calories made up of a nice blend of protein and good fats (e.g. grilled chicken breast, avocado, nuts, whole grains…) could keep you full for much longer.