It depends a lot on your individual child. I moved DS from a school with 20 entry (though c. 30 in each class, through mixing year groups) to one with 60 in each year group, and both he and DD have absolutely thrived in the larger school.
HOWEVER, both my children are 'outliers' in terms of ability. In the smaller school, DS was isolated - he was the only child of his ability, and none of the others were anywhere near him. It was, both socially and educationally, a big issue [in reception, his best friends were the Year 6s who allowed him to play football with them and who protected him from those in his own class who objected to his 'difference']. The school as a whole, however, presented as friendly and all children playing together etc - it was just that there were too few children for there statistically to be others who were near DS. Had he been of nearer average ability, then he might well have had a totally different experience.
By moving him to a bigger school, suddenly DS had a 'near peer group' - simply through statistical likelihood, in a year group of 60, there will be more children at each 'point in the ability scale'. He has thrived socially and educationally, as has DD, who though not so obviously 'different' would still have been an outlier in the smaller school.
It has made a difference e.g. in availability of L6 maths - DS passed it along with 9 others, whereas none took it in his old school as the teaching was already stretched over 2 year groups. In terms of teams - yes DS played in some teams, but he hasn't in others ... but then the presence of children better than him in sports as well as in various other subjects has inspired him to do better.
And in terms of transfer to secondary, DS has taken to it like a duck to water - his secondary is only 3x the size of his primary, and so the transition is much less overwhelming.
Personally, I would never have sent my children to any of the small schools i have taught in, purely due to lack of peer group. But as I say again, that's due to their 'outlier' status (OK, mixed age groups are fine when you are in Yr 4 doing Year 6 maths...but when you are in Year 6, where are your peers to stretch you?) as much as anything else.