I know he isn't controlling in any other way, but he is trying to control you on this one area, in this one way.
And it's not an area you can agree to disagree on either, this is something that needs you to have some common ground. Essentially what you're asking for is security through a plan for the future?
And he's doing everything he can to shut down you asking that. To the point of maybe even manipulating you a little, complaining of your tone etc. (we can all do this sometimes, it's not always a form of abuse).
I'm assuming you want the two of you to live together, and although he initially agreed that he wants the same thing, it never goes further than a verbal agreement. Talk of actual plans are shut down?
He is happy with the way things are, you need to decide if you want to stay in the relationship under those restrictions or move on and meet someone who wants the same as you. I would maybe suggest the latter is in your own interests.
It could certainly be an attachment issue, but it's only valid if he's willing to accept and work on it, I don't know how likely that is.
Did he let you go easily? During the discussion? I have this image of him saying something like "I'm sorry I can't give you what you want"