It doesn't sound like a good situation all round, to be honest.
Assuming that he wasn't trying to set up a threesome, it still sounds as though there are things going on in his life which would mean that dating him would involve various tricksy arrangements and regular disappointments.
Something which does strike me as odd is that he couldn't see why a woman whom he has never actually met would agree to coming to his address very late at night. Why would you ever want to do that?
You mention that he has children; if one of them is a daughter and if she told him as an adult that this had happened to her and that she had gone to the guy's house anyway, he would be having an absolute fit. No decent man of my acquaintance would think it reasonable for a woman to go round to the house of a guy she doesn't know late at night.
I can't say that he is a thoroughly bad person with evil intentions but I do think that he is rather odd and has poor judgement. I suspect that dates with him would show you this on several occasions.
Something I would advise you to do in the future is to always suggest that a first date is a Saturday or Sunday afternoon for coffee and cake.
That would be much safer.
If a guy won't see you in the afternoon on a weekend, look for someone else who can.
I know how awful you must be feeling tonight, as you clearly invested a lot in meeting up with this guy.
I don't know whether he will contact you again. If he does ask to meet up, only agree to meet for coffee in the afternoon.
I'm not totally sure that he will ask to see you; I suspect that he has a lot of irons in the fire, to be honest.
Is he quite a lot older than you, by the way?
Be sure to nurture yourself this weekend and don't dwell on what has happened. I know that you feel grim now, but you will feel better soon.