Op, after many years of learning the hard way, I have the key to this issue. Speaking as an expat and an introvert, with a large family, who is inundated with visitors in normal times, the moment to nip this in the bud, or at least impose limits and boundaries is much much earlier on than you would ever imagine it necessary.
You know how it goes op, someone mentions in passing on the phone how much they fancy a trip to see you next summer say (if times were normal) and because we are polite and socialised to please others, we find ourselves saying "that would be lovely" when, with so much on, we don't think it would be lovely at all . You go away and don't think about it much. They go away hatching plans. You then receive a call from them a few months later saying "you know how you said last winter that we could we could visit, , well we happen to be passing nearby in X number of months for so-and-so's wedding, can we stay for the week? And again, we end up saying "yes" because we are "nice" people, we feel slightly pressured, they seem to have fixed commitments blah blah blah. And in a flash, a vague musing has mutated in to a rock solid plan and you feel put upon and out-manoeuvred.
You need to pre-empt that happening way, way in advance, so have your own plans for the year clearly worked out and written in the calendar, early in the year. Your important family commitments, the times when you want to chill by yourselves. Be very aware that people will make plans for you if you don't make them for yourself. Be clear in your own mind about what you want: only close family member visiting, no visits at all, or short visits. Do only what you feel you can cope with. Banish all guilt from your head. Things are different now you have three dc - that's a lot to manage in itself. And have your stock phrases ready and rehearsed ready to use at the "casual conversation" stage. Write them down and leave them by the phone if necessary:
"Yes, it would be lovely to see you but I have a lot of commitments with the children this summer" then don't sucumb to filling the ensuing silence
"Oh what a shame I'm afraid we are not free that week". Silence.
"It would be lovely to see you for two nights but I'm afraid I can't offer longer because we are very busy with the children at the moment". Silence.
Or a flat at "I'm sorry we are not hosting at the moment as family life is too busy" . Silence.
(I should at this point add that I am not a complete cow and I still do have fairly frequent visits from people but I have found it necessary to manage the process. Some guests are great: arrive when planned with wine, independent, don't need a constant tour guide, muck in with meal prep, leave us alone for part of the day, are responsible for their own DC, and then bugger off promptly when they said they would. They are the ones that are always welcome! Suffice to say, other guests are the complete opposite. And it doesn't matter if they are close family or not. Some close family members can be the worst for taking advantage frankly.)
Sorry for rambling but some last tips: make it very, very clear to your dh/dp that he shares equally in the hosting work for all guests whether they are your friends and family or his. Some men have very odd ideas and think if there is another "female" in the house then they can bugger off and play. And if you do decide to have visitors don't be afraid to have them on your terms. Tell them that they will need to look after themselves at certain times. Or don't hesitate to go off yourself and get some space. Hosting is inevitably a lot of work (unless you have a dedicated guest suite and servants) just all the noise and talk can be exhausting, so don't feel obligated to do anything you don't want to do. The routines of visitors and young DC aren't often terribly compatible.
Be strong op, talk to your other half, then be clear and set your own limits and boundaries, and don't feel guilty