Are you staying permanently, as suggested by "adoptive"?
Do you work in English, or if you are currently a sahm will you realistically be able to once your kids are older and you want to work?
We watch English TV and I only speak English at home - we used to all only speak English in the house until the eldest was 9 or 10 but that has slipped a bit, the main factor I think being the youngest becoming fluent in the community language allowing the children to start speaking that to one another some of the time, depending what they were doing.
I can't bring myself to speak German to my German husband because his English is native speaker fluent, but I have no problem speaking German to other mums and at work. I do recognise that I am a different person in German though - less awkward and opinionated
but also less confident...
I think you will always feel unnatural unless you are in a situation where you actually need to use the language - as you are using an English speaking school that may never happen. If you do immerse yourself, in a work context or by joining a club where you are the only non local for example, it will gradually become more natural to use it and you will eventually lose your hang ups - but you have to choose to leave your comfort zone to do that.
There are people, both men and women, who live in an English speaking bubble in a non English speaking country for decades - presumably until old age... It must be a bit surreal, and most people would say you have failed to "integrate" which is of course a hot topic atm... but does it bother you? What do you actually want? To stop hating the language or to be reassured it is OK not to use it except when accessing services?