yeah - we've been there too.
With 2 kids, 1 year was great, the next not. I think it's the luck of the draw of the parents in the class.
My approach was to just plug away, and when the kids do come over try and give them a good time... eventually the kids will want to play with your child and if they never go there it doesn't really matter.
Some parents do get busy, some are happier with 1 or 2 friends coming over, some only want kids of their (parents) friends to come over - regardless if they get on.
But if your hope is to help your child to be happy, I'd not worry about it too much and play the numbers game and try and get different friends over to play. This helps give your child something to look forward every week and 'keep him going'.
The policy of a range of friends is better IMO, as if a friend is away or 'not being nice' your child can find another friend to play with at school.
Also I'd say that younger kids go through this and it's not just your kid.