Any time you remove something from a food, you add something else to make up the difference. A lot of time when fat is removed from food, sugar, gelatin, or other ingredients are added to either maintain the flavor or the texture.
It's up to you what substitutes you're willing to accept and what flavor/textural changes you're willing to live with.
Like Jay, I won't eat low fat cheese. I love cheese of all types and I'll gladly allocate the calories for them. Also like Jay, I tend to go by taste and content rather than by an absolute rule.
I won't go with "light" olive oil - EVOO is a healthy fat and it's counter productive (IMO) to work around that.
I personally can't stand the texture of low fat cottage cheese, so I eat the full fat variety.
I would far rather have 1t of real butter than any amount of margarine or butter substitute.
OTOH, I eat 2% yogurt and love it. And when I make yogurt at home, I use 2% milk and low-fat milk solids.
I use light mayo (not fat free) for mixing into salads because when it's mixed in, you can't really taste the difference.
I sometimes will use a light salad dressing (my favorite is Kraft Asian Sesame), but most of the time I use red wine vinegar and olive oil.
This is one of those areas where there simply isn't a hard and fast rule. It's all about the decisions you make for yourself about what you'll give up and what you won't, and what you like and what you don't.
.