As the the 'unofficial' manager are you getting paid for that work? Probably not.
So immediately, give notice that others need to start taking on some of that or get a manager in.
Is the band only doing OK financially by short changing you with that? If yes, then your answer is there to some extent - the set up is too fragile to rely on indefinitely.
If no, then why are you allowing it? (There are always reasons, many are character 'issues' that may be worth taking note of)
Loyalty is a good thing but unless you're the well known lead singer, then frankly you are replaceable, even if you've been propping up the show by doing the grunt work.
Unfortunately you sound like you probably wont really know if you're doing the right thing or not until after you've done it.
But some ideas or things to ask yourself:
Do you/they at any level rely on grant money either directly or indirectly in terms of who books/employs you? A lot of opportunities are going to get slimmer or disappear in the future which could be a reason to jump now, or one to carry on while you still can.
What's your pension/savings or S/O's situation - do you need to do something about it? Or conversely - is it too late anyway?
Are you all of the same rough age, and dependent on the continuation of one individual fronting you, or more equal in individual importance/lack of it?
If the former; how long will they be able to keep at it?
Are you schlepping individually - are you in a position to look at a tour bus with manager/driver? Would it make a difference? (either way the answer is worth noting)
Is next stage of age likely to affect playing ability (yours or frontsperson) ie embouchure going south/ mild arthritis developing in wrists fingers etc. If you got told you've got four years left that you could still play, would it change your decision?
Do you have other avenues open to play for the love of it in? (coming of the road doesn't have to mean quitting all together)
Is being a jobbing musician actually a big part of your identity? How much do you take that for granted? Being less 'FOH' may turn out to be challenging if you're used to 'commanding' an audience, and performing to people who generally turn up wanting to like you, rather than just getting on with providing a less direct service. On the other hand if you're third trombone on the brass section, perhaps less so.
Is there any chance you have ADHD - many musicians do and can get it together for a targeted task that is all they've ever done but struggle without the right impetus.
The schlepping up and down is getting you, but will being more confined to a smaller area also get you after a bit?
Is the band well known enough to be useful as a stepping stone if a new career didn't work out?
Are you in a position to create a small pool of session musicians that could cover some gigs and allow you to try out another life while keeping your toe in, still helping out on the management side to give you the 'right' as it where?