Errol, I agree that it would probably never happen! Too much of a rejig to ever really be taken on board, I suppose.
In the system I have in my head, schools are still the venues for quite a few of the activities - they have the gym space, school fields, music/art rooms - but places on these courses would not be limited to those children already at those schools, and could be taken for credit by any child. So, from now (half term) until Christmas, say your enrichment activity day is a Tuesday. Morning session and afternoon session. Your school is offering a percussion class or netball in the morning, gymnastics or life painting in the afternoon. The school shown the road is doing D of E training or photography in the morning, tennis or dance in the afternoon. The local private school is offering rugby in the morning, and keyboard playing in the afternoon. You could also join the swimming group at the local leisure centre for one of these sessions for half a term, or have piano or skiing lessons paid for by your parents or by special grant (with evidence provided for the school by whoever is providing the sessions).
The kid without much money chooses percussion class at her own school in the morning, and a swimming session at the leisure centre in the afternoon. A bus takes children to the leisure centre. She uses her vouchers for those activities. The school and the leisure centre both show evidence she is attending those sessions. Next half-term she chooses rugby at the private school, and life painting in the afternoon.
The kid with rich parents chooses photography in the morning, but for the afternoon her parents can afford for her to have 1-1 private trombone lessons. She uses the voucher and her parents make up the difference.
There would need to be transport to other schools and leisure centres, probably by bus from your "home" school.
It could be expensive for the state, and would also have the problem that kids with better off parents would be able to have them pay fees on top of the basic amount allowed for and paid for by the school, but IMO the benefits would be immense.