Cheese is not counted as dairy but as meat in this plan. Per the Revised Phase I List in the FAQ
(2 cups allowed daily, including yogurt)
Low-fat milk (fat-free and 1%
Fat-free 1/2 & 1/2 (less than 2 tablespoons)
Low-fat plain, vanilla, or sucralose-containing soy milk (4 grams of fat or less per 8 ounce serving). Be sure the product does not contain high fructose corn syrup
1% or fat-free buttermilk
Fat-free plain yogurt
so you are fine. It's a matter of quantity, not number of servings.