OP, I've read your comments but not all the others.
I think your reasons are completely valid. I've been teaching for 30+ years in secondary schools. Before I had my children, I did a couple of residentials because I was "expected to". They were really hard work, but also fun. However, they were just before the summer holiday so I had time to recover. After my DDs came along, I didn't do any. I had nobody to look after them. DH then got a job where he had to spend extended periods out of the country so could not be relied on and MiL was too frail to look after them. Hence, I never went.
I went on my first residential trip with my current school in 2012. It was a music tour to a different part of the UK. I left DH and my father in charge. The DDs were then 13 and 11 so able to cope. The following year DD1 came along and the tour was to a European capital city. 2 years later both came with us and DH had a great time being home alone.
We're talking about another tour, but my needs have changed. The DDs are grown up, but now my DF needs care. I have a couple of options but there is absolutely no expectation that I will go. I refused to go on an adventure residential this summer, citing being a carer and that was accepted and no pressure was put on me.
I think if you are upfront about your family's needs, it would be unfair and unreasonable to ask you to go. Whilst it wouldn't earn you money, you could offer to plan & deliver in-school activities for those not going, which might accrue brownie points.