Some people honestly just don't feel you can be "just friends" with an ex, they think on at least one persons end there's going to be some residual feelings, and in a lot of cases there are.
Others just don't like knowing theres still emotional intimacy between their partner and an ex they shared physical intimacy with, it can cause jealousy and insecurity.
I personally am like your current DP, i wouldn't be comfortable if someone i was dating was still friends with an ex. For me it's both of the above, i'd worry either my partner or the ex might still have feelings and that under the right circumstances (e.g, both being drunk, or a tragedy where one is supporting the other emotionally, or even just being the same place alone together and one admitting to feelings) and that they could end up crossing the line from friendship, to cheating.
I also admit i am a jealous person and i would hate knowing someone who has kissed, touched, slept with, and otherwise seen my partner naked and been intimate, was still in the picture. BUT maybe thats because i'm still a virgin and waiting for the one to settle down with, my last partner had been with 3 women before me and it used to make me envious and unhappy knowing he'd been with other women, and while he'd be my first/only (hoped for only but splits happen) i'd only be "number 4" to him.
I'm insecure and i would find myself comparing myself to them in every way, and im also very self critical with little self esteem so i'd probably judge them better every time.
It is ultimately up to YOU if you are friends with this man, but don't dismiss how your current partner feels, as it will only make things worse. Accept his feelings are real and not something he's chosen to feel, he can't help it, even if you don't understand them. Don't get angry with him, but try to talk to him why he feels as he does, without hostility, and maybe you can put his mind at rest. He may of course decide he can't come to terms with it, and it may end up ending your relationship.