That's unusual. B-complex helps keep away my mouth ulcers, have heard that's common, it's also helped keep away cold sores, and sorted my menstrual cycle. Some people do get allergies though of course, sounds like you are unlucky with this one. Perhaps try a different make? One from one of the specialist companies? Maybe it is something in the preparation?
I have also read that the B12 usually used in vit tabs is cyanocobalamin and that it is suggested that it would be better to take methylcobalamin. Liquid drops under the tongue much better absorbed than pills.
If you are worried about your B12 - do get a blood test for B12 and folate.