We have a well, and our water tastes way, way better than most tap water, or bottled water for that matter. I have Nalgene bottles in a variety of sizes, and we always carry a couple with us in the car. When we go camping, we carry 5 gallon jugs of our own water with us. So I seldom buy bottled water, and when I do I try recycle the bottles responsibly. I've also heard that you shouldn't reuse them more than once or twice.
Greens - there are places that will test your water for you if you're worried about the bateria levels. When my grandparents lived in Florida their water had that sulfury smell/tase. They found that by letting it sit most of that evaporated and it was drinkable. But as Diane (OH2007) says, there are filters out there that will help.