Labour refused to campaign for their own candidate. There is a large Jewish population there, the tory is a Jewish convert and an ex Lib Dem - he picked up a lot of their votes.. Local Muslims are said to be unhappy with Labour's stance on Gaza and a lot of people dislike Labour's Low Traffic Neighbourhood schemes.
Labour lost votes in the election for mayor after the previous mayor resigned for being seen with Dewey, a former Labour councillor who pleaded guilty to possessing indecent images of children. So it wouldnt have been an easy campaign.
A lot in play there but refusing to campaign for your own candidate obviously does not help.