No harm in dressing up a bit your first few days until you get used to the environment. You may want to mention that you felt really comfortable & confident doing "insert whatever that group was doing that you felt good about". I think it looks good to be upfront about where you see yourself fitting in. Shows initiative. (As long as you do it positively and not "well don't put me with so&so cuz they suck")
Don't stress about the pants too much.

I've had the same thing happen before with clothes I was planning on wearing. If you get the chance, buy a new pair that fit you well & flatter you

You'll feel a lot better about looking good in them.
I'm sorry that most of them made you feel unwelcomed. (I had similar feelings about this new department that I'm in.) But people don't like change, and they're not sure what you're like yet. Do your best to ease the situation by trying to make pleasant and make friendly. (This was difficult for me as I'm not a friendly person

) It'll feel awkward for a couple of weeks for sure, but with time you'll find your groove.