Ok. You need a crash course in Matthew Hussey. Look him up on YouTube and binge watch his videos.
Bottom line...don't give him what he hasn't earned. And I'm not talking just about sex here or using sex as some sort of weapon. Don't be investing loads in the way of time and attention if he's not at least meeting you half way. You invest a bit, he invests a bit.
The texting thing...chances are, in this day and age, that he is dating and maybe even sleeping with a few other women. So he'll be texting them and texting you based on where you are at in the stages of dating. Sorry to be harsh but it's easier to end contact with someone you've decided you don't want to see past the second or third date if you've not been that communicative throughout the process.
You seem very set on wanting to invite this man into your home. Do you really know him that well? Has he earned the right to have access to your personal space? Why the rush?
Don't let the "he's in your bit of town" guilt trip you into feeling obliged to let him in. Even if you say that you are not inviting him in for sex, he is more than likely going to try to take things further, especially if you are moving things to a more physical level anyway (you mentioned "getting a bit amorous" in your OP). What's your plan if he turns nasty or becomes forceful whilst you are alone in your home?
Look if you want to have sex with him on the second date with him that's fine and absolutely your call. But if you already know you don't, you can't give him mixed messages. How will you make it clear? And why do you want him in your home? What's the gut reason?
To me, saying "You can come back to mine and we can make out on my sofa but absolutely not have sex" is sending mixed messages. You can say, if you actually feel this way, "Listen, I'm not going to lie I am really attracted to you and, although its tempting with you I'll admit, I don't sleep with men this early on in dating. I need a bit more time to get to know someone a bit better before I feel ready to make that sort of commitment. I would love to see you again. Thank you for a lovely evening."
It communicates that there is sexual desire there but that you want to take things slowly and need more investment from him before sex is on the cards.
If sex is the only reason he was dating you anyway, he will bin you off. If he genuinely likes you as a person, he can hold off until you are ready and feel safer.
Remember, its not just about him liking you, you need to ask yourself "does this man have qualities and values that I admire in another human being?". He's already not meeting some of your expectations. For what it's worth, I think they are reasonable expectations.
I got my heart put through the blender by an ex and some wise person said, "I bet, if you look back, there was a moment really early on that you ignored or made an excuse for that showed you who he really was." And they were right. Within the first month, the first time we went out at night together, I told him that I'd struggled to find parking and was going to have to walk through a deserted area in a dangerous city on my own and that I was afraid. I asked him if he wouldn't mind coming with me and that in return I'd drop him home in another city no where near where I lived. He said I needed to take more responsibility for my own safety and the possible repercussions and that he would get a taxi home as planned. He showed no care whatsoever and was really cold given the fact I was clearly anxious. I'm ashamed to admit this but he actually said "You're a big girl and you should have thought about all this before. If you get raped it's your own fault." And yes...I went on to have a relationship with him that lasted for another year 
. Because I'm a people pleaser and in my head I felt foolish and thought he must be right and I was stupid and wrong to even ask him. I left that relationship with my self esteem in tatters from the emotional abuse he later subjected me to.
It would have cost nothing to walk you to your car. It's what any decent guy would do imo. Texting a bit more often, without pressuring you for your address or trying to guilt trip you into sex is a reasonable expectation. If he's not treating you how you like this early on then its probably not going to work long term. The early stages should be the easy part.