My college biology professor explained it this - the body works to keep the salinity of the blood constant (that is the salt content - interestingly about the same as sea water). If you intake a lot of sodium, your body will pull water from what you eat and drink until the salinity is balanced again. Once that happens the body can get rid of excess fluids. At one time, I could explain this at the cellular level, but I don't think that's what you're looking for. Another analogy the professor gave was "flushing a toilet" you add water to get rid of water and wastes (though that doesn't make as much sense biologically, because the science behind a flushing toilet and the body flushing water are not the same).
You can drink too much water, and especially if done quickly or over a long period, it can actually dilute the blood to dangerous levels. If you're on any prescription medications, ask your doctor how much water is too much. For my mother and I it is 3 quarts - which is only 12 glasses, much more and we would be at risk for water intoxication because of the blood pressure medications we are on (relatively low doses).
Yes, drinking a lot of water can affect the scale temporarily, but if you're retaining water, the scale is already "lying" to you. As to how long it can take to get rid of the water, this depends on so many factors that it's impossible to say from person to person, or even incident to incident. Many biochemical processes affect water retention. Besides sodium and other electrolytes, hormone levels affect water retention (which is why some women retain a lot, and some a little during TOM).
Don't know if this helps any, or only adds more confusion.