I agree with this. It is possible you are overthinking this, and to want to shut things down seems overly defensive. Plenty of people are stay at home parents to young children, it is hardly controversial.
If I told someone who loves and cares about me or my family that I was going to stop work, and they said nothing, or something non-committal like 'hope it goes well' that could easily be taken as being uncaring. Saying 'Oh how long do you think you will take off?' is perfectly normal, and shows they care about you.
What do you want them to say? And most importantly, what impression do you want to give?
Because if you are perfectly happy with your decision, you shouldn't read anything into their (first) responses other than a normal level of care.
Just say 'Well, luckily we have planned out the finances for a while, so I can take a bit of time to think about stuff'. Or 'We have agreed I will concentrate on he family until x is y age'.
And so what if they do say 'don't take too long off or you might find it hard to get back'. Either they say this from a place of knowledge, in which case you can say 'yes, good point, I have taken that into consideration but my priorities are x and y for the time being.' Or they don't know anything about your particular industry in which case you can say 'actually, 3-5 year career breaks are quite common, my boss has just returned from one' etc etc.
Anyone sensible will pick up on whether you are open to further questions. Coming across too defensive too soon will make them think there is something more to this, which is presumably the opposite of the effect you want it to have?
Of course, if they repeat the same question over and over, that gets annoying but then you just repeat the same answer. Or get a bit blunter.
But do not wind yourself up wondering about the motivations of everything they might or might not say, and get too defensive. I know how easy it is to do this but in the end you may put people's backs up unnecessarily.