I would be extremely suspicious here, OP, sorry.
He only changed his story about wanting a relationship with you when you said you wanted to walk away and find something that suited you more. I don't know how long you were chatting in between.
The polyamory is a bit of a red herring, there is still overlap between you and a long term relationship with a child which even if ended, I'm not sure he'd be ready to move on from.
I'd be extra suspicious as you've never seen his house. This is a big deal to me as you've no idea of his living circumstances. Even if you do go there, the partner and child could live elsewhere so it would be difficult to know for sure he was telling the truth, especially if the distance is substantial.
If you want to keep seeing him, you need to keep your emotions tightly in check and as a matter of haste, visit his house (if in a bubble etc- UK).
I wouldn't massively trust anything he suggests about only going along with polyamory to please his ex/ partner. By going online and engaging with you, he was happy enough to actively get involved in that way of life. Not saying there's anything wrong in doing that honestly and openly, just sounds a bit like altering the narrative after you said it wasn't for you.
He could be genuine but there's a lot here that would make me hang back before getting more invested. I at least wouldn't see him/ sleep with him again until I could visit his. If there's an excuse, even covid related, that would tell me what I needed to know. If he cared, he would drive you there, even if it's some distance.