It's not a fact, so that'll be where you went wrong.
First of all, the tax system doesn't just mean inheritance tax. You will have to look beyond IHT in order to understand the difference that being married makes across the taxation system as a whole. Start with CGT, then look at marriage allowance. Neither have anything to do with inheritance tax.
There are other ways that your initial claim about a solicitor being able to replicate all the provisions of marriage were wrong, too. A will in your favour can be changed without you knowing, whereas if your spouse divorces you, you're going to find out. No solicitor can change any of this.
In England and Wales, an unmarried father doesn't have PR until listed on the birth certificate, whereas a married father has it immediately. In UK immigration law, if you're sponsoring an unmarried partner you usually need to show two years of cohabitation, which is more paperwork and faff, whereas this isn't a requirement with a spouse. A spouse has an easier path to make a claim on a deceased partner's estate that doesn't make adequate provision than an unmarried partner does.
On the positive side, not my area but I'd heard pretty much all pensions allow an unmarried partner to benefit these days? It's a lot better than it used to be, in that respect.
Now, not all of these things will be relevant to any given couple. Some of them will be actively undesirable. You might specifically not want your baby's father to have PR at birth, for example. You might never have been abroad and never plan to. You might want to be able to leave your estate to someone other than your partner and them find it harder to challenge that.
But this is why it's so important to understand how all this actually works, and to not generalise.