I think your reasoning why this is wrong is a bit off, in that it can't be about pleasing the other children, but I also wonder if your radar that this is too much is actually accurate in other ways.
Absolutely no-one, just no-one tells me when they can see my child, Even the person my children are closest to outside our nuclear family, my mum, asks to see the children. If they were busy, or I decided they weren't around except once a month, she may be upset, but she would not be able to say no.
I actually don't get this dynamic- they say we want to see her every three weeks, you say 'no', I think every couple of months is fine- then you say, well that's what's on offer take it or leave it. You don't negotiate with non-parents about who your child sees or when.
The power dynamic is all wrong here and it is that which rightly worries you. Your husband is basically missing in action in the parenting stakes, and your word about what is acceptable for your children is being over-ridden by outside people, who whilst lovely and nice, may have their own agenda or just may make you feel uncomfortable for a reason you can't quite put your finger on.