If you put £1000 in savings that earns 2% interest after a year you have £1020. If you leave it there for another year, you get 2% interest on £1020, the year after that interest on £1040.40, etc etc. That is compound interest.
If you use your £1000 to overpay a mortgage with a rate of 1.5%, you are charged interest on £1000 less, so it saves you interest of £15. Keeping things the same, the next year, you now save £15.23 in interest, so a total of £30.23 in interest saved.
Using basic maths, £30.23 is less than £40.40, so you've lost money by overpaying the debt that charges less interest than your savings.
And no, the bank doesn't always win. I've spent the last 15 years beating the banks with interest rates. At one point HSBC were lending me money at about 0.88% on my mortgage and at the same time paying me 5% in a regular savings account.
I was earning enough interest to offset over half of all the interest on my mortgage by keeping £2500 in my Nationwide current account paying 5%
It's called Stoozing, read up about it. 