You are definitely NBU. Of course it is nice that they want to be involved but especially in the early days you want to have some notice. And as everybody on here seems to say, what happens now will set the pattern for years to come, so you need to get it sorted out now.
Somebody on here once had the good idea when faced with a similar situation of always opening the door with coats in hand - so she could say she was just about to go out if she didn't want the unwanted visitor. But probably not so useful if you are wearing boxers and tshirts at the time {grin].
Can you get your DH to talk to them - or is it when he is not around or does he not see a problem with it?
Can you turn it on the head and go and visit them unanounced?
Do they expect to eat if they turn up at mealtimes? If so, you need to be direct and rather than rustling something up for them or stretching your food, say sorry, that if only they had called that you could have sorted something out but as it is you have nothing (other than a cup of tea and a biscuit if you are feeling kind maybe).
And do you ever tell them that it is inconvenient? Or has your DH encouraged it in any way by saying something along the lines of 'do drop in to see the baby if you're around...' - something that is easy to say when you are all excited about the baby, when you know that you mean to ring up and sort it out first, and for it to be on an occasional basis. And they have interpreted it as an invitation to an open house and are actually the sort of people to use (and use!) that invitation...
You haven't said what time of day it is, but saying you have got plans that you have got to go out, or even just that you are just about to go upstairs to feed the baby/have a rest/anything that excludes them really! just so that you don't get tied into being a polite host and them coming back again and again.
You could also try taking the initiative and ringing them up to invite them at some point in the future, say that you're aware that you have got a really busy couple of weeks coming up 'what with one thing and another' (always best not to be too specific!) and invite them over (or even get them to invite you) - then they should realise that you are busy until then. Let them suggest a couple of dates that you can refuse so they can see how busy you are . And then slowly extend the periods between visits so that it matches what you want them to be.
If they turn up unannounced, you can then express surprise that they have turned up as you know you were seeing them soon and had they forgotten you were busy?
Sorry, lots of questions. And much too long a reply so will stop right now. but good luck with dealing with it!