Neither of them are allowed, unfortunately (neither frozen yogurt nor ice cream). To be honest, most of the SF/FF ice creams are frankenfood...processed food with tons of weird, unpronounceable ingredients. You really want to stay away from them. There are those that make the argument that the popsicles are the same way, and that there is less fat and such in the ice cream than the pudding, which we are allowed, but the truth is, you have to do what works for you.
Stay away from ingredients that cause indigestion, like sugar alchohols, if you can, don't get swayed by "low carb" ice creams...they are no better than the rest, and just plan stay away from ice cream, if you can. It's really the best thing. And this is coming from a woman who ate several bowls worth of full fat ice cream every night for two years. That's partly how I got to where I am.