We have a holiday home on the Northumberland coast which we bought two years ago. At the time we were looking for a 2/3-bed cottage, to use periodically and rent out with an agent until such times as we moved closer and could do it ourselves.
We investigated small cottages, but the figures didn't stack up very well, so in the end we bought a 5-bed barn conversion, did it up to a 5-star standard, furnished it with loads of baby and child-friendly kit and marketed it via a small, local agency. We get lots of small groups, house parties, and two-family/extended family holidaymakers, and are probably full around 75% of the time (more from March-October). I think having a larger property works really well if you are in a good, popular location.
You really need to look at the figures - done out beautifully you can get maybe 4 times more weekly rental for a 5-bedroom house than you would for a 2-bedroom cottage, but the price to buy it would probably not be 4 times more.
As someone said earlier though, it can cost a lot to furnish it, decorate it, and keep up the level of decor and style, although we've been lucky and had no real damage so far. Fingers crossed! One other thought, bigger properties are harder to fill at short notice, though on the plus side they do tend to book up further in advance.
I would also recommend a local specialist agent. We pay ours 25% to manage bookings and the cleaners (who we pay additionally).
A final idea would be to call a small agency you'd like to be associated with, in the area you are keen on, and ask them what size of property they are desperate for (it could that they have loads of 3 bed cottages and are keen to sign up a house-party house, or a 2-bed bijou cottage).
A bit long winded I know, but hope this helps!