He sounds dangerous. However he is also not bothering to hide it. The police should set you up with a panic alarm & I know they have a personal one too - you probably know this but then you are on a register & if he comes anywhere near you they will get to you. Assuming you have a non mol order?
Get the house, get it covered in your own CCTV, & sorted, & put it on the market. And move away where he cant contact you.
In the meantime he will say everything & anything. You cant & wont hand your daughter over & so this all becomes 'not an option'. Hes not going to tip over the edge over the house if he hasnt your daughter. You arent going to get him off your back by giving him your house & keeping your daughter so whatever way this goes you are in this. You have done a fantastic job & now is not the time to doubt yourself.
You may think you are escalating, but that ship has sailed. Theres a trail of evidence & previous behaviour. Keep calling the police, & get recordings of him threatening you (a crime!) & then move. They do get bored & move on.