Potted plant problem - soil crawling with tiny flies

[HackneyM]

I have 20 + potted plants in my small apartment, all fairly healthy and happy. However, the soil of the olive is crawling with tiny black flies, it is almost as if I am looking at an ant hill.
My best guess is that it was provoked by over-watering a few weeks ago. I have put it on the hottest window sill since then so it should have dried out by now, but the flies are as numerous as ever.
The odd thing is that the plant itself is absolutely fine, evenly green and new shoots coming out. It's just that the flies seem to have settled in for the long haul. Also, quite oddly, the flies doesn't seem particularly interested in the neighboring plants.

Are there any remedies that could discourage the flies but not harm the plant?

It sounds like fungus gnats. They're really common - we had a lot of them at the start of the year. We ordered some nematodes online - it's just a powder that you add to water and apply to the soil - it's a biological control that gets rid of them and stops them coming back.