Impresionante - we still get paper statements are they are sometimes asked for as ID (and online statements not accepted). Both my teenagers have had to produce them recently for ID for work exp in a bank and a PT job respectively. And obviously they don't get other statements, eg utilities, being teenagers still.
Having said that, we all have online banking, DC got it as soon as eligible on 16th birthdays. Who doesn't have it, if they have internet and can.
As for still sending statements to parents and opening them in private, words fail me.
Op - not sure how old you both are, but it does seem like your DH has never grown up, he has just moved from being looked after by parents financially, to being looked after by you. So many red flags here it is untrue.
I cant understand why you married him without resolving these problems, esp if you had to take loans and work extra to pay for it all. That in itself should have been a red flag, that he wasn't interested in helping you save for wedding and honeymoon.
As a pp said, there are so many life events to come, maybe illness, kids, buying a house, etc, that if you are not working together financially now, it will only get so much harder.
Please give him an ultimatum - and not just to set up a SO. Statements to your address, full transparency of finances etc. Once married, esp with DC, all money should be seen as family money, but there is no way that can happen without full transparency.