If you, yourself, haven't had an anaphylactic reaction to vaccines/medicines then that's fine. Just because there's history in your family - it's worth mentioning to the team - but it's extremely rare.
I do have a friend, abroad, who had an anaphylactic reaction - they have a history, themselves, of anaphylaxis with lots of foods & so on, they have a set of quite serious other health problems too. They're the only person I've ever heard of in friends and family having a reaction. And what happened is they started to react, were immediately given medical help, and then spent several hours (as it was late afternoon, they stayed overnight, in hospital, recovered but just being monitored).
So, I do understand having your vaccine on a day when your husband is about, just to make sure in case. But if you do have a reaction then the team there will immediately deal with it - it's why they make you wait for 15 minutes afterwards, just to check - and you'll be fine. It won't kill you. And if you've never personally had a reaction before (presumably you've had other jabs when you were young?) then you'll almost definitely be fine. I have allergies (which have never affected my breathing) and the doctor giving the vaccine said that was fine (and it was - both times).
It sounds like you're a bit isolated here, and it's worrying that your husband is thinking you might die. It could be that he feels a bit isolated, too, and so is projecting his catastrophising onto you.
Please go and have the jab - tell the team about your worries beforehand and they'll monitor you even more carefully. But it'll be so much safer to be jabbed than to risk getting the virus. (And even then, you're only likely to get sick. Death is extremely, extremely unlikely).
I get anxiety a lot, catastrophise loads, but this from your husband does seem off the scale. All the best!