I can only recommend to do them! I have a tendency to "forget" working my abds too, because it's exercises that I find boring, so I try to correct this behavior and do some at least every 2-3 days in order to keep them in shape. What's sure is that they *are* hard for everyone who isn't trained. No matter the weight or the commitment to exercising, even a thin person will have a hard time pushing out a good series out of her abds if she's never worked on them (well, this is almost true, I guess - some weight-lifting exercises partially rely on your abds, too, thus doing them will strengthen them a little, even if not as much as "full" exercises).
The only thing is to make sure that you're doing them well. I've found that the traditional "sit-ups" aren't worth much, and tend to hurt the lower back more than anything else (especially if this area is already a sore point). Good abds exercises will make you feel like you're working them, and not hurt the back in any case. I found a nice exercise on Krista's site about weight-lifting for women, and I think I still have a link somewhere to an "abds FAQ" in English; I'll add it as soon as I can find it back
PS - Think of it this way, too: better start working them slowly and gradually, rather than wait "until I've dropped lots of weight"... because, once the tummy fat goes away, it's much more pleasant to see already-developed muscles under, rather than a bunch of flabby flesh