I think it does matter, educationally and in terms of extra-curricular activities. It matters much less than senior. But it still matters to me.
We chose to do both, rather than waiting for senior, even though we live in a small village with an outstanding state primary with small class sizes. And even though the local comprehensive senior school is also outstanding and our babysitter goes there and got 10 A*s at GCSE and says the whole of her set are like her. Even taking that into the mix, I think my DDs do better in the independent sector in this area. There are several option independently, and all of them are good. I chose the one that best suited us as a family.
For me, it's that INDEPENDENCE that's important. I loath the term 'private school'. I'm not paying because it's private (and I'd never use a school that was for profit, only ever a school with a board of governors and run on a not for profit basis). I value that independence away from the state, away from the national curriculum, away from SATS, away from dictates and budgetary decisions that are out of control of the school.
I also think that actually a little bit of it is in the saying you get what you pay for. I think I'm paying double per year than the state would spend on my child if they were educated by them. And as the school we use is not-for-profit, that is actually going on the children and the school. They can pay their teachers more - I am proud of the fact that all the staff are getting a small pay rise this year in contrast to pay freezes in the public sector for example.
Better facilities, the freedom to create their own curriculum, great teachers. Not saying facilities and good teachers that can't happen in a state school, but with half the budget per eight/nine year old, it's not going to be easy. If the government doubled the budgets, it would be an equal choice. Until then, it's not.
For what it's worth, I also love the commute as my child just opens up about all sorts of things on occasion!
And in contrast to the above poster, my DH is a head of an independent senior school, and we've educated independently for junior as his opinion is you can see the difference in an 11 year old. Not that it matters much, but it's still there. But then, it's also flaming obvious if a child at 11 has been so a rubbish independent school. As in the state sector, there are good and bad ones.
I organise activities outside school too. I'd never expect any school to do my job as a parent in also educating my children. It's a shared partnership imo. Both of us and the independent school working towards a shared goal of happy, healthy children who meet their potential in whatever it is they are passionate about.
If we couldn't have afforded both, I'd go for independent senior though above junior.