The question, if I understand correctly is, can SS take the child away from the father (or family) if they sent the DD home, if the Father goes to court to try to get the return of the older 3 children?
You need to find out what orders are on the baby at the moment. If the LA share PR with the mother and father, then there is a care order, or interim care order (ICO) on the baby. This means that there is a care plan as well, so the LA an ask the court to approve an amendment to the care plan to reflect that the baby moves to foster care. The grounds for changing a care plan don't seem to be as strong as removing a child from their home, but you can use the same arguments.
If the baby is back home on a residence order, the LA would have to reapply for an ICO.
You might want to find out what behaviours the older children display which might render the baby "unsafe" if the older children return home.
Another issue will be if the older children have been away from their family for 3 years or more. After 3 years, it's very hard to get your children back because the LA use the argument that you are a stranger or partly a stranger to the child, mostly because the LA reduce the contact to a few times a year, or once.
Are there placement orders on the older children? If there are, the battle is mammoth and enduring, the only people to gain out of this battle are solicitors because they get paid regardless of if they win or loose (if thee is a winner or looser in the secret family courts).