Not terrified, no. What will change? Racists have always been here and always will be. Look how much airtime Farage is being given. As much if not more than the leaders of the 2 main parties. That's no coincidence.
Black people - especially those raising children here - who want to challenge racism will actually have to get up off their chair and attend meetings, groups, rallies etc. & vote, too; I know too many people who write long think pieces about politics on FB etc yet they don't even vote so, why..? Black churches can speak up and out too, for a change.
Make the effort to be visible and vocal, write letters, where it matters.
I never made England my holy grail. Once I had children myself I worked steadily towards 'this ain't it'. At this stage now I'm semi-retired, spend 4 months of the year in UK, the rest in my heritage country. Some of my friends and family did the same. My children now grown are there regularly, I made sure to take them as often as I could when they were little, so they could see another way of life and people who looked like them everywhere. & not think you can only survive in life if you live in UK or USA.
I hate racists, they hate me. Stay and fight for what, exactly? But for those here as said - you're going to have to get up.