If you didn't say the part about downloading music, and you were just going to put your own music (from CDs) and some podcasts on it, I'd say get a small, lightweight, inexpensive MP3 player.
I have the Sansa Clip and it's nice for running because it clips on like the iPod Shuffle (and weighs practically nothing!),but it has a million more features and is cheaper (got my 2GB for $45 on sale). It's also insanely easy to put your music files on it... literally ALL you do is drag it from your computer into a folder, and in seconds it's done.
However, I know it can be tricky getting some music downloading services to work with some devices, so if you're doing a lot of that, an iPod is probably your best bet! They make arm bands for all of them, so it's easy enough to use them for running (unless the arm bands drive you nuts
like me).