I'm with Casey on the first post, you have no obligation to call and it might even lead him on if you do
Not calling someone the day after a first date with them isn't rude...unless you said that you WILL CALL TOMORROW. And even then, it's not that big of a deal after one date.
It would definitely be more awkward if you ran into him in the future after a string of not-so-good dates that you went on because he got the wrong idea.
Just my opinion, though