Please take the warnings seriously. Our son is well attached now (2 years in) and older when he came (3) but still very emotional now and not nearly as 'easy' as we thought he would be in the first few weeks!
If you do take her, please take lots of things to help her feel at home (her bedding, bowls, toys etc). And please do not visit wider family (a reason not to go as it is hard to explain to people why this is not a good idea).
If you DH and DS go can you Skype daily so DS can tell you what happens and Dd can hear their voices - not sure what others think of this idea? Not sure how much very young children can understand pictures on a screen as real people but you telling her dad and brother will be back in 4 days, 3 days etc might make some sense but only once they actually arrive back. She won't be able to make much sense of it but I do feel when we tell kids good stuff, and then it happens as we say it will, it all helps to build a bond.
Ds met my sister at our home (she is my closest relative) and this was about 6 weeks in. Sis was very good and didn't crowd ds. He seemed to take this well.
We visited sis's family almost three months in for 2 nights. We had shown photos and videos of who was who.
Everyone was very clued up not to expect hugs and kisses and to give ds space. For us it worked ok. Ds was happy.
We took his own special ready bed, plates and cups etc. It was summer and sis loves by the sea. It worked ok but looking back it does seem early.
Knowing what I know now I don't know if it was the right decision.
Anyway. Whatever you decide, please believe there is lots of good advice here and I think many of us have felt we knew the right answer and many have had our eyes opened by experienced adopters here.
I would take advice from experienced clued up adopters over social workers any day!
(Ps just so you know we have a birth dd, I don't think your son should forgo the trip, for the record I'd let him and dad go for some dad/lad time. There is almost six years between my dd and ds, they do a lot together but we do try and give them separate time too!)
All the best.