Yes it is totally normal to have anxiety when you start doing IE. It can feel chaotic to listen to your body instead of rigidly controlling your food intake. Hunger does become easier to recognize but that takes time. Listening to your body is like listening to an orchestra. If you aren't used to listening intently to music it can be intimidating at first. It all sounds like one big giant loud sound. Once you begin truly listening though you begin to pick up the subtle nuances. Instead of "sounds" you begin to understand that you're hearing a flute or an oboe, that the tempo rises and falls, sounds get louder and softer. This is a clumsy metaphor but it's extremely hard to put into words what it "feels" like to be hungry.
“Do not judge me by my successes, judge me by how many times I fell down and got back up again.” - Nelson Mandela