I buy from Atlantic Spice Company, on the Outer Cape, whenever I'm in Cape Cod during the summer.
They also do mail order but I love going in there & tend to buy in person rather than online.
http://www.atlanticspice.com/
Other sources are Penzey's in Grand Central Terminal Market and Kalustyan's, which isn't far from the E. 28th Street stop on the No. 6. Both have an online presence.
I'll link to the lesser-known. Kalustyan's is here:
http://www.kalustyans.com/default.asp
I've also bought spices on eBay.
Another interesting source has been the tiny gourmet food sections in TJ Maxx, Marshall's and Home Goods.