My DS has similar special needs and if there’s one thing I’ve learned since separating from cheating Ex, is that I just do not budge when it comes to DSs wellbeing. Which means being quite different from a ‘normal’ child.
I did start off with EOW and also letting Ex have time with us all as a family and it was a disaster. Ex spent EOW undoing all the careful work I was breaking my back trying to do with DS, mostly by letting his family do half of it, and by not being in touch with how DS was really. Of course it was all my fault, there was no problem etc. except he didn’t see the changes. He would be out with his new gfs or on the phone to them half the time too. Basically DS was either left too long to his own devices (but he was good as gold Ex would say... yeah he was in a corner by himself) or with his family who were overbearing and grabby (and had no idea of how sensitive DS is). Either way he came back in an agitated state.
So I consulted professionals and made DSs time all about what was good for DS and not me, or Ex or anyone. Ex didn’t like it, as I was saying okay his time needs structuring like this, his family had to back off, and I cut down his hours with Ex until DS was more stable. Only then could we negotiate more time with Ex.
I don’t know what is going to work with you in the future. However if it is truly centred on DS NEEDS and not what your Ex WANTS - then things become clearer. My DS is loads happier and unfortunately part of that is because he spends the majority of the time with me. I wish I could have more time off tbh but he needs stability far more than another child who could cope with changing homes so frequently.
In my view, this is something my Ex totally selfishly overlooked when we started a family together. He just didn’t factor in that having a disabled child would mean that separation has a much much greater impact. It’s just not okay to pretend that you can be as good a father. Some kids need the intact family unit so much more than others, and our DSs are one of those.
So unfortunately OP I think it shows your Ex in a more disgraceful light, he’s not blind, it’s his responsibility to have done the very best for his special needs son. He should not have abandoned him and he will not be able to make up for that.