I’m almost the same as you, almost in my 40’s and always regarded as being different from the normal. I have the same fears as you are but I have learned to accept everything. I try my best to do what is right, whether people accept or don’t accept me for that, I really don’t have to care as long as I don’t hurt anybody. Although I can still feel sad, I always think of our Creator that is just a heartbeat away from me. I find solace in nature, that everything and everyone has a purpose. Learn to meditate and pray, it will be help.  You may be solitary but you’re never alone.