OP, I've skimmed through so apologies if I missed key bits. I totally understand your need for a break. I think you should have one.
When you use the word 'difficult' I immediately think Autistic/ADHD. Difficult is the word I always used and still do as a parent.
A professional cannot rule this out at all without a proper assessment. Many psychiatrists in CAMHS will tell you your kid is this and that. If they haven't had an assessment which follows ADOS2 process for Autism, they really can't say for certain he isn't Autistic. It should involve more than one professional also as part of the assessment. We had 3!
I read an article this week saying one area in Scotland has cancelled every person on a waiting list for an autism assessment. No follow up, nothing. They lie, they're incompetent, I would not rule this out. The right diagnosis and even appropriate meds might make everyone's life a bit less shite.
Personality disorders may come along with all this but being Autistic and living in this world can really increase risk of a personality disorder. Maybe it isn't even a personality disorder at all. Women in particular are often told by some random psych they have borderline personality disorder when they're actually Autistic. No one ever assessed them.
What screams out here anyway as not ok is the part where you bring the other adult child with you for company.
Is there any other adult? A friend or anyone with a kid who could tag along?
You and the 3 youngest, yes, that is easily explained and justified. Older son will have to accept that.
My mum would do things like arrange things, leave me out and lie. I always knew it was bullshit. I'm not difficult either. I have a very dysfunctional family! It's just not a good feeling, it's hard to justify and he knows he's difficult already. Not your fault. But, bringing along the other adult child is not going to be something you will be able to lie about and have him buy it imo.