The way I see it is that moderate supporters and voters are driven away, the parties become more polarised, more people also decide not to vote, the extreme voters become more valuable, and the parties shift more the extremes to get their support. Eventually, the middle ground becomes so wide that a party can emerge to comfortably occupy it and attract a potential large group of non-voting moderate people.
It's notoriously hard to set up a sustainable centre-ground party though and it only really becomes viable once the unoccupied middle becomes so wide.