You did well at the buffet, don't feel bad about yourself
I know I advocate for everything in moderation, say that I go out to eat, etc. but buffets are one place that I avoid. I will not go. If my friends suggest it I will outright refuse to go hang out with them if they truly want to go there.
I haven't been to a buffet in over a year and I'm looking to keep it that way. Honestly it's more of a cost thing than anything else!! I get full on so little food that it's pretty much a waste of money for me to go. In a regular restaurant you can at least take the leftovers with you—you certainly can't do that at a buffet!
If you feel like buffets are too much temptation, then refuse to go. It's hard to initially learn to say no when people are persistent, but it gets easier over time. I'm a pain in the a** sometimes when it comes to places to eat, but I'm not about to just be nice and blow all of my hard work.