In my opinion, none of the fats is likely to send you over the edge in terms of cravings. Higher fat will mean higher calories, so if you consume a lot of higher fat choices, it will reduce the amount of weight loss you have, but the main reason Dr. A asks us to limit our dairy products to low fat or fat free is to reduce the amount of bad saturated fat we take in. Remember, Dr. A is, first and foremost, a cardiologist, and so his plan was created to not only reduce our weight, but to make our blood chemistry healthier.
That said, some things are just not very good fat free, while others are not much different. I never eat fat free cheese, as I find it very unpalatable, while reduced fat cheese is just fine. On the other hand, I find fat free cottage cheese and yogurt to taste just the same as the reduced fat stuff, so I buy FF of those. I also love fat free milk, and I buy organic, which often seems creamier than the regular stuff, but I still get some 1% milk for coffee.
Hope that helps! Good for you on being conscientious!