OP - is it all nuts you are allergic to? If not all, try to get a balance of the other nuts you can eat. Pulses like chickpeas, kidney beans and lentils are all good sources of protein.
Do you eat diary and eggs? Both excellent sources of protein. How would you feel about starting with fish? Ideally we should all be eating 2 portions of fish a week, one of which is oily, and this is even more important during pregnancy as the baby needs the oils to lay down brain tissue.
If you are unsure where to start have a look on Hugh furnley-whittingstalls (?sp) fish campaign website, also his 'fish' cookbook is brilliant for giving well considered researched advise on which are huge most sustainable types of fish to eat. There is also an app you can download from the campaign which tells you which fish to eat/not eat at a glance. (Be careful with fish oil supplements during pregnancy as they can contain very high levels of Vit A depending on the type)....
Yes we are all born with our eggs but they take 3 months to mature (same as sperm) so any dietary changes you make now will make a difference to egg quality in 3 months time.
I know it's frustrating to see people drink/smoke/take drugs etc and still get pregnant.....but it's also worth considering that these population groups tend to have statistically higher numbers of problems during pregnancy (low birth weight, prem labour etc) and their babies are more likely to have developmental problems associated with drug/alcohol abuse etc.....plus, these people may just be younger, or have a naturally higher levels of fertility (good hormone balances etc) so try not to let this dishearten you, and try not to compare.
I know it can be overwhelming when you've been veggie for so long....but from my own experience I was a vegan for over 10 years....I started eating diary, eggs and fish when we were TTC and when I got pregnant I started eating meat too. The thought/smell of meat in the supermarket used to make me wretch before I got pregnant, but I actually started to crave it. I started small with organic, free range chicken once a week in a spicy curry for e.g. (To hide it!!) and built it up to try other things (organic beef etc). I was worried it would make me ill too...but it hasn't at all!! Good place to buy ethical meat is waitrose (if you can't find a good local butchers) they sell free range/organic meat.
Other lifestyle changes you can make that will have a positive impact on fertility include regular exercise, reducing/stopping alcohol intake or smoking, trying to de-stress (hard whilst undergoing IVF I'm sure!) and getting enough sleep. Perhaps ask your fertility Drs if they could refer you to the nutritionist/dietitian at the hospital for some advice?????
Good luck xx