That's how it works in our children's school, they have TAs who specialise in art, music, RE or French who go round the classes whilst the teacher does their planning, traning, form filling, one day a week.
It works ok.
If you mean not going to school on a Friday well that would be ok for those who can afford all the activities you mention, can't see that the state would pay for specialists to deliver those.
I help as a parent volunteer in primary and I see how the teacher has a lot of things to do that has nothing to do with teaching the 3Rs, although that is what the school is ultimately judged on in the league tables and its what the general public seem to think its what there taxes are going on. At the same time, my children do various out of school activities and they are always more fun and better done than when the equivalent gets done in school. So how about children go to school for four days a week, get taught academic subjects by their teacher for those four days and she uses the fifth day to do her lesson planning, marking, form filling and personal training? Meantime, the children go to a series of activities on the fifth day where they attend a mixture that works for them of art classes, music classes, outdoor activities (camping, hiking etc), sports etc.
I know this is only dreaming.. but why wouldn't it work?