Get a sport bra that breaths/wicks away sweat better and try to keep it dry until the redness and pain clears up. Best to let it heal before doing heavy sweating workouts.
It sounds like the heat rash I got while watching over a group of girls scouts in a camp with no a/c down here in florida, hot and sweaty with no relief in sight, and oh god did it hurt!!!
Here's a helpful article:
http://www.buzzle.com/articles/heat-rash-treatment.html
Oh, and an antifungal cream might work on it if it is something like a yeast skin infection like Haley says. But it doesn't sound like it's that if it goes away on it's own within a couple weeks.
Hope this helped.