SardineQueen - she doesn't want to stop BF! She has said she wants to BF for a year like with the other DCs. Why should this be compromised? It's what the mother wants to do and it's in the interests of the baby. Why would a good and loving father want to compromise the BF relationship?
This is, and isn't, a feminist issue. I'm absolutely a feminist but I do not believe that at 4 weeks old a mother and father are interchangeable necessarily. There may be situations in which they are, ie both have been present and taken equal responsibility for nights since birth, and the baby is FF. However, that is unusual, even with together couples, and with a father who has never spent the night with the child? That child has no relationship with him tp speak of, the bond needs to be developed over time.
What would people think if it was the MIL wanting to take the baby overnight at 4 weeks? If the non resident father can do exactly what the mother can do, why can't the MIL? Or is it only worth distressing the baby and the mother in the name of paying lip service to equality between a mother who has been with the baby 24/7 and a father who hasn't?
IMO, the ex needs to work out a way that he can come and see the baby pretty much daily for an hour or so. If it has to be after work he can look after her while OP has a bath or some dinner, and build things up that way. Otherwise he can take the baby out for a few hours, in between feeds or with a bottle of EBM if the OP can manage it.
People are absolutely right that he needs to build a bond, and has a right to. But this can't be at the expense of the baby's well-being.