First: this isn't the 90's anymore. And thank goodness.
That fat in almond milk is probably healthy fat from the nuts, so there's nothing wrong with it. Fat keeps your skin/hair/nails healthy, keeps you feeling full longer, and is absolutely necessary for your body!
If you had two kinds of animal milk, one with more fat but less calories and one with less fat and more calories, my answer would be different, because saturated fat from animal sources is unhealthy and worth minimizing in your diet.
How much protein is in each kind of milk? That might affect your decision as well.
I tend to think that if you have two decent options, you might as well pick whichever one you prefer, instead of spending a lot of time thinking about the little details. So if you like almond milk better, just go for it!