Ginger in law both parents have equal rights. If it was the mother who'd taken baby nobody would bat an eyelid
Wrong.
In law it is always about the child's best interests.
If there is no need for social services to intervene then it is a private family law matter and the courts will consider many things.
As both the mother and father have PR, the father is doing nothing wrong in the eyes of the law (I know, I know fucking disgusting) and if the mother wants to assert her rights she needs to get a solicitor and make an application to the court under private family law.
I wish the MNer ph47bridge was on this thread, she is a fucking legend with the law and she would be able to really help the OP here.
I don't feel confident enough to give the OP any more advice other than GET SOME LEGAL ADVICE ASAP.
Unfortunately I know a lot about private family law from personal experience and I wish I could but I can't talk about in any kind of depth because it brings back memories of one of the worst times of my life.
I am sorry, I wish I could be more helpful but I can't risk breaking myself when I have spent the past ten years getting better and getting over what happened.