I have DS 2 and DD 4 and so far we have gone out for the day each time and invited friends/family to join us but pay for themselves (we provide cake and our kids). Normally Zoo or park or something like. That way if people can't go on the day or weather is bad it's not costing us money.
I've said no parties with lots of friends till they reach 5th birthday. (Well probably about 8-10 friends)