What other vaccine did your dad have which led him to develop VITT? I wasn't aware it was linked with any other vaccines than the covid AZ and J&J vaccines.
Over the next few months, there will be many more covid cases, and you will be pretty likely to have some contacts, unless you all live a very distanced lifestyle. This contact I wouldn't think twice about, but I'd use this as an opportunity to decide what to do going forwards.
I would have a calm rational think about whether you prefer to seek a vaccine now, or not. Try and get some objective numbers about your risk, and some facts about the vaccines available to you.
Age 40 and healthy, your risks of being hospitalised or dying from covid are low.
If you are going to have a vaccine at some point, it would make more sense to have it now, as cases will soon go up rapidly, so you will get most benefit from it.
The chance of both you and your dh dying from covid vaccination is absolutely miniscule, you are probably at more risk travelling in a car together, if you want to have a vaccination at some point I would not delay for fear you will both die from the vaccine.
Most people with a history of allergies can have Pfizer vaccination, depending on which allergies.
Most people with a history of blood clots can have AZ vaccine.
Some centres are offering moderna vaccine, and maybe others.
If you want to have a vaccination I would check if any vaccines are definitely not ok for you, make sure the relevant info is recorded in your GP records, then book it.
So you could reasonably decide to have a vaccine now, or not have it, but I'd make a calm rational decision either way.
Also as a family you need to decide what you will all reasonably do in terms of risk. Your dh going into a party "to be polite" then coming home and saying he thinks you'll now end up on a ventilator, and you moving into the spare room, is nuts. If he thinks going inside is too risky, he needs to politely decline invitations to go in. If he chooses to go inside, he needs to maintain realism afterwards and not freak out and try and put his anxiety on you.
What will definitely be harmful to your health, your daughter and your marriage, is spending the next few months repeatedly getting horribly anxious and dh repeatedly saying you are going to die. Both covid infection and the vaccine are very unlikely to kill you, and you both really need to have a look at the numbers and make some calm decisions about what to do going forwards.