It needs to be a balance doesn’t it? Ingrained childhood habits will stick in to adulthood.
A bit of structure for just one morning a weekend will help a lot. It doesn’t have to be that onerous.
Signing the eleven year old to an (preferably physically active) Saturday morning activity he enjoys: climbing wall, dance, life saving class, badminton, kayaking, anrchery, table tennis! Anything!
You start taking him, then slowly pass on the chauffeuring to your dh , who will then have to be up and dressed by a certain time.
The class will mean that you and your dh can opt to spend some time together, or apart, while your eleven year old is active. And you all come back to a nice lunch feeling refreshed and then they can really enjoy their afternoon of loafing guilt-free while you go and see friends or enjoy something alone.
Or you can schedule family cleaning on Saturday mornings with lunch out afterwards. It needed be expensive - a picnic - some outdoor space. A change of scene!
Both of these models above worked for us when the dc were between the ages of eight and sixteen . It’s a relatively short phase. You have to tell your dh that he can’t behave like a single adult when he is a husband and father. He can lie in on Sundays fhs.
The truth is that lying in and slobbing about makes you feel more tired not better!
And no it’s not fair that you have to organise it all, but it’s not working as it is and it’s not healthy for your younger boy who will tend to mimic the same sex parent.
Ideally you could talk calmly to your dh about your frustrations and come to an agreement that you are all going out for a family bike ride on Saturday mornings from now on, but in reality, if he is reluctant, an outside booked class for his dc that’s already paid for, and his son enjoys, will motivate him more. You never know, you might get the fifteen year old to join in too! It’s three hours out of 2 x 16 hr waking hours of the w/e! It’s nothing!