Of course you're not being unreasonable. He's been extremely selfish and arrogant IMO - and that does not mean that I disapprove of him trying to re-establish contact with his older children, rather that he has a responsibility towards ALL his children and shouldn't therefore be making any sort of unilateral decision where only some of his children get to enjoy activities while his youngest is effectively sacrificed to enable the older ones to carry on with what they enjoy.
He is supposed to be in a partnership. Any decision he therefore makes which will affect his wife should be discussed with her up front. How can anyone feel doing anything other is acceptable in an equal relationship .... no matter what the decision relates to ?
Those respondents who mention "compromise" have, I think, forgotten what compromise means. It does NOT mean rolling over and letting someone walk all over you (and your child). I've always thought that a compromise is when two or more parties who have different opinions, needs or wants, all accept that they cannot have exactly what they want but instead negotiate and discuss the situation so that as far as possible each party ends up with some of what they want - even if it's not their ideal. It's about give and take .... you "give away" a little of what you'd ideally like to have to minimise negative effects on someone else - be they hurt, inconvenience, disappointment, expense or whatever. No-one's left 100% happy - but that's far far better than one party being 100% unhappy.
And yes, while it's important that the 6 year old has a relationship with her siblings and yes, taking things to the extreme, that is arguably more important than activities - the proposed arrangements are absolutely unfair. If the sibling relationship is so important why is the father only concerned that his older children's routine must remain unaltered ? Surely, with that argument, they too would apparently appreciate the importance of a sibling relationship over and above losing one of their activities (OP states they have activities all nights except Tues) whereas the youngest seems to have just one activity which she will now be forced to miss.
As others have said, this man is letting his child pay the price for what HE wants, and hasn't made any attempt himself at any sort of compromise - and obviously hasn't had the basic courtesy to discuss with his wife pretty major stuff which will have a significant impact on her and their youngest, thus not even giving her the opportunity to discuss and maybe suggest alternative ways round of facilitating contact. I very much doubt that the 6 year old will happily give up something she enjoys and actually understand that it's apparently all worth it so she can have a relationship with her older sibs. I'm sure she won't understand at all and if her dad comes at it from that angle will in fact almost certainly resent the older kids which will be a great start won't it for a fledgling relationship ..... and then of course, as she gets older and starts to understand more, will wonder why the heck the older kids do multiple activities every week yet she was told she could no longer do one.
I also agree that if he's so arrogant now and seemingly so scared to "upset" the ex, that this doesn't bode well for the future. Logistically it's always going to be tricky to blend the needs of two different households where children flit between them both, even when all parties are mature, fair and communicative. If, however, you have a situation where one (or more) parties is unreasonable, stubborn and/or likes causing maximum disruption just for the sake of it then that issue is going to be ten times worse. Very few women would be content with a situation where the needs/wants of one household and/or one set of children are always put above the other - it would drive you mad and cause huge resentment.
I think you need to have a very serious talk with him - what he's "agreed" to with a solicitor isn't binding until it's rubber stamped by a judge. This thing about him not being able to alter this in case the ex withdraws contact again doesn't make sense. You haven't explained the full background to this ..... has he actually applied to court for a contact order, or has what's happened so far been as a result of mediation, or just correspondence with a solicitor ? Whatever the ex wants can't just be imposed on him no matter what .... if this had gone to court and the ex said he's not seeing them at all because he won't pick them up on a Tues (even though he could see them any other day) I think she'd get short shrift, because it's all about true compromise. Most judges would probably suggest that the midweek contact is ditched in favour of the weekends instead if Tuesday is such a sticking point.
I do get that he feels guilty - though not sure what the background is and whose fault it's been that he didn't see the kids previously. However, the only fair way to sort this out is for him to set out what he wants, and for you to set out what you want ..... and then you work together to see how all, or at least some of that can be achieved. It shouldn't be all or nothing with him getting everything he wants. And to be honest, as it's him altering the status quo (regardless of whether or not what some people think should have been happening up until now) I'm inclined to feel there's rather more onus on him to present realistic solutions - such as how an extra car might be financed, researching public transport and/or taxis, sharing pick-ups from school with another parent, and addressing the issue of how you'll get to see your family if you literally can't drive that far without his help.
If he absolutely refuses to do any of this then I'm afraid I'd be thinking very hard about the future. He's basically drawing a line between his kids and clearly prioritising some above others. That'd be a dealbreaker for many.