I would only recommend a really small school for a child of genuine middle ability.
Differentiating e.g. within a R-Yr2 class [with an intake of 10, there will usually be a R-Y2 class of 30, a Yr 3-4 and a Yr 5-6 but there may be other ways of doing it depending on numbers in each year group] is hard enough for a single teacher to manage (3 year age spread, EYFS + KS1) and plan for.
If you have e.g. a very able child, then that's fine in Reception and year 1, as they simply work with the next year group up in many instances (check - some schools teach in rigid year groups rather than mixed year ability groups or whole class) even in a mixed age class, but in yr 2, there simply isn't the planning and teaching capacity in the day in many cases to make the extra differentiation for an really able child - or conversely a very much less able yr R.
Small school doeas not necessarily mean small classes, also. The largest class I have ever taught - 35 - was in a very small school, but where inflexibility of accommodation / unusual year group sizes made one class very big indeed.
Also, if you are of 'not average' ability, the probability of finding others of like ability in your year group is lower in a small school. In DS's fuist school (intake of 20), he was a complete outlier, working virtually alone. In his next school, 60 intake, there was simply much greater statistical likelihood of him having near-peers.