Nini, you don't need someone else to make plans with - make your own for you and DD. Take a picnic, go to the park, go to a museum, go to a garden centre with an animal bit (all favs with my DS when he was a littlie). Make it clear that you're doing your own thing. You've just had a couple of awful weekends when there's been stuff going on, so if it's even worse when there are no plans, then make sure you're not around. You owe it to yourself to get out of the house and not subject yourself to it if you know it's coming. I know the temptation is to to do nothing in the hope that this time he'll somehow have magically changed, but you know he wont have.
With the spending time, is it possible to just say to him at dinnertime - 'We haven't had our 10 minutes for the last few days, I've been waiting to see it you'd initiate it since I've started it every time so far. You haven't mentioned it and now I feel like it's not important to you. If it is, can you please initiate a talk tonight.' Or something like that - no discussion, just stating as fact how you feel and what you want him to do about it.
I'm definitely going to steer clear of 
. I don't drink during the week particularly, but when I'm at the house on a Saturday he plies me with it liberally - he gives me a drink around 5pm (with him having been drinking since lunchtime or mid-afternoon), and my glass is never empty after that, he makes sure of it. But I need to take responsibility for myself and not blame him for this, I'm the one happily downing it, I need to say no more thanks (or no, not at all).
Nini, I've tried to talk to him several times about the 'incident' (as I keep calling it, can't get my head round it being anything else). We argued massively for two weeks following it - he kept saying he'd said sorry that he'd upset me and didn't know what else I wanted him to say, but there was no recognition at all that how he behaved was wrong, and I'm obviously (pointlessly) still holding out for some acknowledgement. It was all about him - how bad I was making him feel by keeping talking about it, as if he was some kind of rapist (he used that word, I never went near it at any point), how bad he felt because I wouldn't go anywhere near him sexually for about 2 weeks. It obviously still upsets me if it's coming out like that when I've had a lot of few drinks.
Nothing like that incident has happened since, or had happened before. But there have been incidents before and since of me saying I didn't want to have sex (usually at night, tired, etc), and he'd go into a strop, really annoyed with me, and I'd end up agreeing just to stop him going on. And when I was 37 weeks pregnant, I told him I wanted to stop having sex because I'd had bad SPD since 10wks and it was beyond uncomfortable now, and he got annoyed with me then too, told me I was selfish 
and what about his needs. So of course we didn't stop having sex. But that's partly my own fault, I feel, for not being strong enough to draw my line in the sand and say 'no, I don't want to do this and I don't care what you say'.
And now he wants me to go to his uncle's funeral with him, as well as his sister's wedding. I just don't think we're anywhere near that point of being able to go to family things together, and I'm not sure I'm emotionally strong enough for either event. I've just texted him to say really sorry but I can't go to the funeral, but I want to support him so I'll find a different way to do it, I'll make sure he has a relaxing night that night (it's our normal Thursday date night). I hope he's not going to go off on one about how selfish I am, but I guess if he does then it tells me more than I need to know. 
Sorry for the mahooooosive post, it's just spilling out of me these days...