We went for the local outstanding primary across the road and switched to the prep in Y3 and Y4 (different children, different years). The prep is half an hour drive or an hour on the school bus.
Initially the local primary was great, out of the house at 8:50 and back at 3:40. However, as my children got older, the vast majority of children were being carted all over after school for clubs; swimming, karate, ballet, football, rugby, athletics, French, you name it! So it was impossible/very difficult to organise play dates in the week, plus with more than one child (four for us), days became really long as the list of extracurricular became longer and they all got dragged along with their siblings. Also for working parents it would be after school club until 5 or 6pm and before school club in some cases.
So in the end we moved them. People feel sorry for my children as they comment on "what a long day" they have, but they are back home at 4:30 or 5pm every day and do not have to then go back out for activities, as they do absolutely everything at school, so they can just relax when they come home. Play dates are mainly at weekends or school holidays but they seem happy with that. Also they get tired but not as much as I thought they would do, and they love the school bus as it is one hour to chill out, mess around with their friends. They play card games, watch films, chat about their day and exchange cards, it isn't just sit there and be bored until they get home.
Good luck, we agonised loads about it but it's not too bad. Medium to long term you can always choose to move.