I'm going to go out on a limb here because I'm probably not normal.
The last time I went down to what was my lowest weight, my boobs "grew" as in I went from a 38C (170 lbs) to a 34D (135 lbs.). A few years later and about 20 pounds heavier, I went up to a 34DD and it wasn't until my weight got up to what it is now that I went to a 36DD or 36DDD, depending on bra style. My gain this time around was different then before - my butt, hips and thighs are bigger then they were at any other previous weight gain. I can only attribute that to my age and genetics catching up to me - larger bottom halves run in my family.
I've notice mine sag a bit now as compared to then (I was also 20, I'm now 28). I'm prepared for them to appear more droopy so I'm doing everything in my power to build muscle in my upper body so I can hopefully keep it minimal. Only time will tell and since I'm trying to be healthy, I hope there is a better look to me overall. I do think it really does depend on genetics. I personally want mine to go down as finding bras is a massive headache (seriously, how do you find them in your size?!). I like wearing cute underwear and hate that I tend to find my size in "full coverage" or "reduction" bras which feel like I'm strapping them down. I'm a tad scared they will increase or rather have the fat redistribute and I'll be stuck looking for huge cups and a small band size.
I wouldn't say "don't worry about them" but just wait until you start seeing your body change. Surgery is personal and I would absolutely support anyone who does it - a friend of mine did it and it was the best thing she ever did for herself. She does the gym, she has 2 active huskies and she eats well but her body didn't change the way she wanted. Surgery gave her a shape she didn't have but it also gave her a relaxed confidence - she is more comfortable in her own skin.
Just give yourself some time - you are already making a big change.