I looked into doing that myself a while back. There are some concerns that just the ball doesn't provide enough back support to sit on all the time. I also was concerned about appearances at the office I was working at.
A good and more discrete alternative seemed to be what is called "balance discs" or "active sitting discs".
Same concept but in a smaller package, plus you have the chair for back support. They also make balance ball chairs.
I can only speak from research and not from experience. I'd be interested too in hearing from those who have tried it out themselves.