nonameuser - in answer to your question:
Did you ever feel like things would be ok soon even though it was never going in the right direction?
Yes! All the time. I kidded myself that things would get better. They never did, and the longer I spent with him the more difficult it was to leave. I felt I had to prove he was worth my time, even though deep down I knew he absolutely wasn't.
You talk about doing your own things, but also of spending more time together... I don't fully understand this.
I think, from the sounds of things you've drifted apart as a couple. You could try to fix it by doing some of the stuff you've suggested, but he's rejected those suggestions. So, I don't think you can salvage anything - it takes two to make a couple work and if one of you is doing all the work, while the other idly sits around watching, the one doing all the work will start to resent the other. This sounds like it's happening already.
You need to be honest with yourself and your partner - if there's no future for the relationship, is it fair on either of you to continue with it?
Like I say, I spent nearly ten years trying to make a dead relationship work. It really wasn't worth it! If I'd left him ten years sooner, I would have had a much happier life.
So that's why I'm saying you should leave sooner rather than later.