If it was alarm bells and red flags, OP, I'd absolutely agree with those saying follow your instincts.
And of course you should never do anything you genuinely don't want to do.
But my reading of this guy as described is someone who's woefully out of practice in the dating world, daunted by returning to it, and is being (probably too) honest about the whole thing.
Do you generally feel misgivings at the prospect of first dates, or try to talk yourself out of them? Or are these misgivings singular to this guy?
I've seen lots of OLD profiles since I took the plunge that mention number of DCs and living arrangements. It sounds like he's put it a bit mournfully, but that doesn't mean that's the only emotional mode he operates in.
Also, is there something to indicate no contact with his dd? Couldn't it just be his XW now lives in a different part of the country? That particularly jumped out of your post as something you'd constructed a story around, something you were looking for, as opposed to something he'd written.
There are also profiles that come across as charming that, when you meet the person, turn out to be quite misleading.
My advise (by no means telling you to do anything) is: if you're truly worried, for any reason: cancel. If the date itself is a major hassle in terms of logistics: cancel.
If it's local: what about using the old standby dating technique of having a friend call an hour (or 20 minutes) after you meet him, so that if he's crying in his pint and playing a hand of woe-is-me cards, you can beat a hasty retreat.
Ultimately, what feels best for you is paramount. You're the only one who can assess the risk-reward potential.