I know someone slightly further down the line than you. She has an epithelial tumour which started before she was born. Now, aged 32, she is beginning to develop symptoms (weakness in arm and leg, vision disturbance), which, even with the operation, won't be reversed. But if she doesn't have the tumour removed, it will eventually have potentially fatal effects - inability to swallow, etc.
She has been told that in a similar case, with someone older and with other health problems where they could not operate, the patient was in a wheelchair within two years.
The sooner they operate, before symptoms develop, the better the chances of being able to remove the tumour completely.
She, like you, is terrified. But putting it off really isn't an option. Have you asked for a second opinion? That might reassure you, even if the answer is the same. As others have said, although the idea is scary, recovery time is quicker than other, less scary, ops and the fact that it is benign is a huge blessing.