I am so sorry you are going through this 
I can really sympathise with how you are probably feeling.....the shock, feeling of utter betrayal, anger sadness etc.
This is something I have personal experience of, except I didn't discover what was going on until after I had left. It took me 2 years to find out the true extent of his deception, things just kept coming out of the woodwork.....exp took out high interest loans, credit cards, didn't pay council tax for over 4 years, didn't pay water chargers (ever I don't think). I started finding out about 3 weeks after I left when my mobile phone got cut off, the phone call to Orange was an eye opener....'The bill has not been paid for 3 months Miss A'...'Why hasn't anybody phoned me?!'...'We have been in communication with (EX)Mr A'...'What? Why? The contracts are in my name!!'....'He said you had given permission for us to deal with him as you work a lot'. It cost me nearly £700 just to get my phone reconnected (I was sofa surfing since I left and it was my only form of communication with the outside world). This just showed me what a master manipulator he was, he actually managed to convince a massive company to go against their own confidentiality policies and data protection act and discuss my 2 accounts with him and not me 
Within the first 6 months I discovered a computer course loan (computeach I think it was called) totaling £8000 not that he ever went near a computer , I discovered 4 CCJ's for non-payment of council tax, 3 CCJ's for non-payment of water charges, rent hadn't been paid for 6+ months.....it was a nightmare. We lived in a top floor flat and the post box was a secure box by the entry door downstairs, he would empty the post box without me knowing and just leave little bits of junk mail so I would think that was all the mail we were getting. The man was an arsehole, I worked 3 jobs, would work a 12 hour day as an HCA in A&E go straight to the pub to work a shift then onto the LD residential home to do a sleep-in, he was unemployed. We had a joint account that I didn't have any access to (he chopped up my card and it was back in the day when you had both sign for a replacement so I couldn't just order another one!). It was all too much, I went bankrupt. I didn't have the fight in me, I had been abused and down-trodden for 11 years and just wanted shot of him.
The worst thing about all of it is, I have no idea where all of the money I was earning went! If he wasn't paying anything, what was he spending it on? We certainly weren't living the life of riley, some weeks we could barely afford to eat.
I guess that epic post was me trying to show you that you have found out about this relatively early on so nip it in the bud now, before it gets any worse. Had I known about even half of what he'd obtained in my name, whilst I was still with him, I would have reported him to the police (although I think there's now a group called Action Fraud? I may be wrong). It's taken me a long time to recover 