OP why did you go to see a solicitor? Was it to establish what court may grant should it come to it or was it because your ex is taking you to court over contact?
If the former then don't feel you have to go to court. You and your ex will hopefully be able to come to an agreement without the court being involved. Mediation, if you can both be reasonable, is really the best way forward for you all.
Contact doesn't have to instantly be every other weekend and half the holidays. You can build up to it. But yes you can expect your ex to be given that should it go to court. Bear in mind of you start with 1 day every weekend it may be considered the norm and you could have issues then going to every other weekend. It might be better to consider a weekday as every week and then contact alternative weekends. It doesn't matter so much when they are not at school, but as soon as they are you will find weekends precious too.
What kind of involvement does he currently have? Has he been a hands on dad? Do you have any genuine concerns as to their welfare when with dad (or is it simply that you don't like the idea of it?). You might feel you have done all the hard work up till now, but things have changed now and your roles have changed. It's time to let him do some of that hard work too, and to be more involved with his dc. It is hard I know, but you have to accept that things have changed and so has his role as a parent. Many fathers do decide they want to be more involved after separation/divorce and that traditional roles aren't relevant any more.
What do you think would be reasonable to start off with and to move too? Your ex may be more willing to compromise if he can see that you are looking to increase contact in 6 months then 12 months time. Plus it will (hopefully) be easier for you and the dc to buld up gradually.
So perhaps suggest that he have a midweek now and every other weekend have 1 overnight at the weekend (say saturday overnight). Then make it friday and saturday overnight in 6 months time (your youngest will be 2.5 by then). And then in a further 6 months look to making it friday, sat, sun overnight with your ex dropping the oldest at school on monday morning.
That's assuming your ex even wants that much contact. Some do, some don't.
Your dc are entitled to a relationship with their dad and unfortunately your feelings on it shouldn't interfer with it. The first time they are away will be horrible for you. They will be fine and, in time, so will you. It does get easier in time. Bug hugs in the meantime.