The "don't worry about it" comments are mostly to do with people worrying about the size of student debt, surely?
This isn't a debt like other debt. You may not pay it off. No one will be battering down your door to get their money back. The amount you pay back works in line with income, so you are never going to have to get swamped by ever increasing debt repayments.
If you want higher education your options are to pay for it yourself, find a family member to pay for you, or to take out a student loan.
(or to get an apprenticeship, but not possible in all subjects, and limited in number)
If you want to pay for it yourself, this is likely to require an extended period of working and saving, continuing to live with your parents (or shelling lots out in rent), during which time all your peers have been to university and finished and you will be starting as a mature student who may have lost interest in study by that point. In the case of some subjects, it's likely to involve a longer study period as you re-learn concepts you will have forgotten. Yes, it's a choice, but may be less palatable.
Parents are likely not to have the money lying about, or would prefer to give it to you as a lump sum for a house deposit.
Or you take out a student loan, and accept that you have to pay the tax. Yes, you'd probably rather not. Most people would prefer not to pay NI or Income tax either, but they get used to doing so. This is still likely to be the best option for most, and worrying about it will get you nowhere.