One old trick is to freeze liquids in an ice cube tray. Then you can keep the cubes in a ziplock bag in the freezer and only use them one or two at a time.
You could also look in the juice boxes/ children's drinks aisle of the supermarket for the small boxes of milk packaged for use in lunch boxes. You might have to look in a couple of groceries before you find a non-chocolate kind, but they do make them in skim, 2% and full. It's real milk, but UHT or ultra-pastuerized (like the boxed soy milk) so it's shelf-stable at room temperatures, and the boxes are about 8oz. My grandma in Germany, who didn't drink much milk, always kept it around for the kids, and I have some in the closet for emergencies. It tastes a little "cooked", like evaporated canned milk, but you might not notice in tea. In the SuperWalmart I have found the UHT milk on one end of the baking ingredients aisle, next to the powdered milk.
I also use powdered milk (skim) in baking, and a few times when I have run out of fresh milk I have used it in my coffee, and it tastes fine, just as if I had put fresh milk in. I don't know why non-dairy creamer was even invented, unless for lactose-intolerant people, powdered milk tastes better and it is all real milk. Some brands dissolve better than others, so don't be discouraged if you try it and it doesn't work the first time. You can always give it away to a baker or cook if you can't find a way to use it.