I think it's always worthwhile to test if dairy affects you negatively, in terms of GI function or congestion or anything else. Sometimes you don't realize what it's doing to you until you cut it out and re-introduce later. And if you aren't affected, then you have that knowledge too.
I don't know that switching necessarily can be expected to affect your weight loss.
I'd also just mention that there are a wide variety of non-dairy milks in addition to soy: oat, hemp, almond, rice, etc. They can even be homemade, although I haven't done that yet!
Milk is hard to evaluate. There's lots of conflicting studies and conclusions. I personally don't think it makes sense for human adults to drink cow's milk for its infants, although I haven't given up cheese as yet.
You also have to acknowledge the way dairy cows are treated if you choose to buy that product.
Some feel that whole, raw milk is much more healthy than reduced-fat, pasteurized, homogenized milk. Kinda makes sense to me, but I can't say I've thoroughly researched it, and there are of course certain risks, which many will debate.
There is also the concern of hormones, if you don't buy brands to avoid it. A corresponding concern with soymilk might be to check for organic brands to avoid genetically engineered soybeans.
Confusing, believe me, I know