I have always been active but one thing I will tell you is that your needs will change. Sometimes, like moving to a smaller house, it will be external reasons. Sometimes, you will get bored or your body will, and you will want to change. So, when you are shopping and you aren't certain about what you want, maybe a gym membership would be cheaper in the long run until you decide what you would like to "invest" in.
The cheapest is walking or running but do invest in the best pair of shoes you can and get them fitted by an "expert".
I think, gym memberships like to a specific health club or even the Y are the second cheapest because they are the ones who put all that money into the equipment. Also, they have staff who can help you get started doing the exercises the "correct" way.
Finally, at home workouts and equipment. Dvds are the cheapest. For variety, they can't be beat. And you can always pause them if you have to pee---unlike when you are a gym. "Excuse me madam, will all of you stop what you are doing while I visit the bathroom?" Hey, it's all that water we're drinking.
If you aren't sure you want to get free weights try heavy cans you can easily grasp your fingers and hands around. Remember with these you will outgrow the weights so before you buy a lot of lighter weights consider using canned goods initially.
At home equipment is the most convenient but again space is the biggest issue. I bought a $900 treadmill that is a monster to move but the walking part folds and locks up so actually it doesn't take much more space width wise than a chest of drawers. I bought it through HSN on installment payments. My DH and I love using it. It is especially great for rainy or cold days.
So, when you are shopping, think about how your needs are going to change (and they will) and spend your money accordingly. Then, the only burn you will feel will be in your muscles and not in your wallet!