I don't know where you are (it's warm here in TX, but is getting colder in other places), but walking with a pedometer outside is an easy, gentle way to lose weight. We're supposed to walk 10,000 steps a day, but I'm having trouble getting to 8,000 right now!
Wal-Mart has lots of inexpensive equipment (yoga mats, exercise balls, etc.) and some of it comes with a workout DVD included.
Or Half-Price Books might have some workout videos.
Also, if you're in a larger town, check around at ALL the gyms. Mine is a smaller women's gym and it's only $17/month. I know other gyms in the area that run $30-150 a month, which is definitely not in my budget. Perhaps there is a gym you haven't found yet that is much cheaper?
Way to go with starting to eat healthier!!