Here's a big misconception - you don't have to shop at places like Whole Foods to be healthy. You can keep going to your regular grocery store.
Don't get me wrong, I LOVE Whole Foods and I live w/in walking distance of one so the temptation to go there all the time is huge. But honestly, there are times you can get food just as good and just as healthy for you at a regular grocery store for 1/2 the price.
Ok, another thing is don't feel like you have to go all-or-nothing. I didn't clean out the pantry and ditch all the food when I started eating more healthily. I just found that those things that we no longer ate started getting pushed to the back. AFTER we'd already not eaten them for a couple of months, then I started getting rid of stuff - giving a bag to the food bank here and there. It's ok to take baby steps.
Speaking of babies! It's easy to get caught up in the whole "I'm going to puree baby food from organic veggies and nothing else will ever touch my baby's lips" mindset.
This goes along with the all-or-nothing thing above - it's ok to buy jars of baby food if you want. You can buy healthy organic jarred food now as well .. so be sure and cut yourself some slack there. Too much change can be overwhelming!
So some tips for starting out slowly:
Start buying fresh veggies and eating seasonal veggies instead of canned. Frozen is the next best thing.
Buy whole grain pasta/bread instead of regular.
Buy brown rice instead of white
If you normally buy canned goods, buy the no-salt versions instead and salt things at home.
Substitute olive oil for regular oil when you cook.
Substitute olive oil and a bit of flavored vinegar for a salad dressing.
One night a week, try to have a meatless meal (this will also help your budget).
Try to add veggies to every meal. If you're going to make smoothies throw in some frozen spinach, for example (I know it sounds yucky, but if you puree it, you really can't taste it - you just have to learn to ignore the faint green tinge!
Sometimes just a handful of little things add up to big changes.