Just because it's called the "Free Speech Act" (or, to give it its full title, the "Higher Education (Freedom of Speech) Act") doesn't mean that before this act there was no free speech, and that repealing it means the end of free speech. That's just rhetorical sleight of hand.
The issues are (as usual) way more complicated than the headlines make out. They are more complicated than feminist claims that "Labour want to deplatform GC people", they are more complicated than Tory claims that "Labour want to deplatform anyone right of centre", and they are also more complicated than Bridget Phillipson's claim that "This could become a licence for antisemites".
In several cases, the speakers being uninvited/deplatformed have been uninvited by students' unions (or other societies affiliated with the SU of any given uni), which are ostensibly independent of universities, but of course can never be so entirely. A complex aspect of the Act is about defining where the responsibility should lie. It creates a requirement for complex new relationships between SUs and unis, and a large number of possibilities for conflict between the two over issues that have nothing to do with what most of us here would consider to be free speech. It also creates a huge workload for SUs, which despite their reputation in the tabloids are mostly dominated by Trotskyists, in every university and most FE colleges in . Overseeing these relationships will be the Office For Students, a body that was sufficiently critical to the previous government that its proposal for the first Chair was Toby "Actually mate, I had my dick up her arse" Young.
Have a read of this or this, which describe the realities of the implementation of the Act, and ask yourself if it sounds like a workable way of preventing deplatforming, or if it's more likely to turn into a bureaucratic nightmare, which the tabloids can then use to bash universities (one of their favourite pastimes anyway).