Hi @Ammy7889. It's very hard to follow your posts, which may be why you haven't had many replies. I'm not sure I've completely understood you, but I will have a go based on what I think you are saying.
As far as I can work out, you are saying that your ex has been a very inconsistent presence in your children's lives, nd eaxh time he comes back into their lives, he is more interested in re-kindling things with you than in actually being a dad. You are now trying to formalised the contact arrangements, and he is not co-operating. Because of his continual coming and going, and the on-again off-again relationship with you, you feel very controlled.
If I've got that right, the first thing to say is that you are not being controlled. He is responsible for his (frankly, terrible) behaviour. But every time you go along with it, and decide to start dating / sleeping with / whatever with this man, that is a decision that you are making. He isn't making it for you. The way you describe all of this is very passive - as if you just go along with whatever he wants at that moment, and are at his beck and call. The first thing that you need to understand is that this is YOUR decision. You are not being controlled. You are choosing. And you need to own that choice.
The good news is that the minute you do this, you take the power back. Because if you are choosing to go along with it every time he flutters his eyelashes or flexes his muscles at you, then you can also choose not to. You can choose to put boundaries in place. You can choose to refuse to talk about "us", and be clear that there is no "us". There is only your role as shared parents. Bluntly, unless and until you do this, your situation is not going to improve. Doing so is within your control. You just have to decide to emotionally disconnect, to mean it, and to stick with it.
This then becomes a practical question of sorting out access arrangements. I strongly think you should apply for maintenance - both as a point of principle (this man should be paying for his kids), but also because it will help you to make the shift in your own mind between a 'romantic' relationship (not that it's very romantic!) and a co-parenting relationship. And it will signal to him that youre serious about that too.
Then make him a structured offer on access arrangements. Put it in writing. Set out which days and times you are offering access. Be clear in the offer that this is to take place away from your home. Invite him to accept, or to make a counter-offer. This needs to be really clear. You may need help putting it together - consider speaking to a solicitor, to citizens advice, or a charity such as Gingerbread for help. If he engages, then you are into a negotiation to agree the access arrangements - you may need formal mediation to help with this. Once they are agreed, stick to them. It's about boundaries - you don't have any at the moment, so you need to establish them.
If he doesn't engage with your offer, maintain your boundaries anyway. The only thing that you will discuss with him is the practical arrangements for co-parentong the children. Absolutely nothing else! If he wont negotiate, make a court application for a child arrangements order. That will stop him demanding access whenever he feels like it - you will make the children available for access as ordered. If he takes it up, great. If he doesn't, then he has had his opportunity..
Basically, it's all about recognising that you are in control of your own life. Stop being passive, nd take positive steps to resolve this. Establish and maintain boundaries. Before you know it, you'll be in a much better place. Finally, if you'll forgive the cheek, I'd suggest that you may want to consider some sort of counselling / therapy for yourself. You have spent a long time accepting the unacceptable. That suggests low self esteem - you may want to do the work on yourself so that you consistently feel able to set and maintain healthy boundaries, and are able tot recognise that your life is your own to manage. Nobody else can or will do that for you. Good luck.