Not everyone loses weight on a whole food diet. For several years (during my refusal to diet years) I maintained my way-too high weight on a primarily whole food diet (I was eating white rice and a few other non-whole foods). I ate healthfully, but I still ate too much.
I'm also insulin-resistant and quite carb-sensitive. I cannot eat fruit or grains in unlimited quantities, and I can't eat them alone. If I eat high-carb (even good-carb) foods like fruit or grains without fat and protein, I'm hungry again in an hour or less (so I eat more, and lose less or not at all).
I need a portion-control element, so I use an exchange plan for balance and to control calories, and I use paleo and whole food principles to guide my choices.
You may be able to tweak your plan without having to resort to calorie, exchange, or point counting. Some people limit grains and/or fruits or even fats to a specific number of servings per day (and then don't count anything else).
That may or may not work for you. Personally, I can overeat anything (except maybe leafy grean veggies) to the point my weight loss can stall.