I have been in the same situation (still am) and I had to accept that my DC has as much as I wish otherwise another legitimate parent with all the rights, privileges and problems that brings. Also have several close friends that are in the same situation. I know exactly what you are going through, PM me if you want a more detailed response.
Went through the court system, you are right to be afraid of it, avoid it at ALL costs, it destroys lives. Don't trust lawyers advice.
I have a gifted son, in Y9 now, and for ten years his other parent has never done a single homework with them, read a single book with them, or supported them academically in any way. In fact they have actively discouraged homework and study, putting me down every chance possible.
My advice is to find a solution that works for you all, as PrimeDirective said dont stop the other parent from seeing their child. For example my experience is that, building up to contact, every other weekend plus half the school holidays works well. This gives the child school time stability for you to foster good academic standards but plenty of time with other parent, almost considered as down time to relax.
The scenarios that don't work with parents of different ideals, is spitting up the school week, or spontaneous contact. All sort of problems occur with those.