You might have an issue with whey. Even if you can drink milk just fine, having a higher concentration of that component of the milk might bring up an issue you don't normally have.
I'd say try some soy protein, or vegetable/hemp protein isolate and see if that helps. There are lots of options out there.
