Yes, I think it's going to be a really bad time. My subject (Humanities) across the country is around 35% funded by EU grants, and being a Humanities subject it's not realistic to believe that any of that money will get replaced. Most of the money comes from funding streams that the UK won't be eligible for once we leave. A lot of the EU money goes to fund post-doc opportunities and PhD places, and that will make life very hard for early career scholars trying to get a position after finishing their PhDs, as if it wasn't hard enough already.
Competition for the UK funders will obviously be even more intense than it already is, which will mean fewer people with successful bids, which is likely to have a longer-term impact on how sustainable universities perceive departments to be. And if you add the lost income from EU students not coming here as much any more, that's an even bigger financial hit. And of course having international students and staff is really important to the vibrancy and prestige of the universities, and keeping them real players on the global stage. Recruiting top international talent is going to get even harder with visa restrictions, and even there are more and more stories of the Home Office preventing foreign academics coming to conferences - if that starts to affect EU academics too, we will end up totally isolated. I wouldn't be surprised if a lot of universities cut back, best case scenario by not replacing those who leave or retire, worst case by redundancy programmes.
It's bloody scary, to be honest. I think we're on a path to a slow decline, and I don't see much future in UK academia for someone ambitious and research-focused if a hard or no-deal Brexit goes ahead. I think people aren't talking about it because everyone knows it's likely to be bad, and what's the benefit in getting upset now when we're going to have to deal with it all pretty soon anyway. Within the group of academics I talk to, some are hoping that the government will see sense and go for a soft Brexit; some people are quietly applying for jobs in other countries and not telling many people until something works out for them; some are stressed enough with the day-to-day grind to have the energy to worry about it until it happens; some spend all their time venting on Twitter about it to make themselves feel better. Haven't met anyone in academia who is feeling optimistic!