Yes it is totally normal to have anxiety when you start doing IE. It can feel chaotic to listen to your body instead of rigidly controlling your food intake. Hunger does become easier to recognize but that takes time. Listening to your body is like listening to an orchestra. If you aren't used to listening intently to music it can be intimidating at first. It all sounds like one big giant loud sound. Once you begin truly listening though you begin to pick up the subtle nuances. Instead of "sounds" you begin to understand that you're hearing a flute or an oboe, that the tempo rises and falls, sounds get louder and softer. This is a clumsy metaphor but it's extremely hard to put into words what it "feels" like to be hungry.
"Things are as they are. Looking out into it the universe at night, we make no comparisons between right and wrong stars, nor between well and badly arranged constellations." - Alan Watts