He is a lovely genuine man he just has trust issues with my friends.
No. He doesn't have 'trust issues'. What is happening is that he is fucked up inside, damaged and broken. He is broken beyond repair and therefore he is deadly dangerous. In order to make himself feel better about his life he needs to control your access to anything other than him and his version of reality.
He NEEDS you to reinforce his fucked up, mad view of himself. He needs you to plaster over the gaping hole in his soul which he is aware of and which he cannot ever face. He will only be able to do this by making you agree with his world-view. Therefore he must isolate you and have you constantly reassuring him that he is lovely and loveable. He needs you to do this full-time to the exclusion of any wishes or desires of your own. You are already starting to do that, aren't you? Posting here asking how you can improve things?
YOU as a person do not exist to him. Your needs, your friendships, all the things that are important to you as a person do not matter to him at all. Yes, he values and even sort of loves you but only as a function - and that function is to make him feel better about himself. When you fulfill that function he will be lovely to you. Actually he will be extremely lovely to you - google 'love-bombing'. When you do not fulfill that function he will punish you by withholding affection until you toe the line again. Eventually he will become violent if you defy his wishes because you exist only to serve his neurosis. Already you are being trained, already you are anxious that he will be upset, that he will leave you and you are starting to walk on eggshells. You are actually wondering how to not see your friends. Can you not see how fucked up and wrong it is that you are even thinking about this?
If a a person loves you, really loves you, then they will be happy for you when you are happy. A real partner would encourage and support your friendships (and all your hopes and ambitions) because he would understand that they are important to you and he would wish you well. He would have your back and support you.
This man can never do that because he is so damaged. If you want to have a life where you can still recognise yourself, where you will have love and fun and support from a partner, then you need to run away from this man as fast as you can.