Anxiety can alter your metabolism, sure. But that's not a reason to stay anxious!
I would definitely suggest you talk to your doctor about your anxiety and how it affects you around people. You may be able to get some help for this--either medication, therapy, or both.
Any good doctor is not going to prescribe something to speed up your metabolism, by the way.
If you have stopped exercising, then you could be losing both muscle and fat. Is there any way you can keep up at least some exercise?