My 18 month old insists on calling me daddy! I am the one with the boobs, but no matter how much I try to convince him otherwise I am still daddy.
He sometimes calls his actual male parent daddy too but usually it means me, especially when he is all upset and howling daddy daddy daddy!
So aibu to keep telling him, Son, I'm not your father.