Yes Hesta We will trade with EU countries eventually but not so easily, and not on such favourable terms. There will be loads of the dreaded red tape and It will cost the country more.
Concerns about EU immigration are all mixed up ( completely incorrectly) with fears of all sorts of things including fears of otherness, fears of terrorism. However it's illogical and ill conceived to mix those things up.
EU immigration is proven to be economically and socially needed -it is vastly beneficial. Our kids get to work in Italy, Lithuanian kids come and work here.
Having a different visa system for EU men and women won't stop some bad people coming in from anywhere (Russian poisoners for eg). It won't stop weirdos like that McGreavy being born here.
The US has a different visa system compared to us - yet bad things happen there. Saudi Arabia has a ruthlessly strict residency permit system - bad things happen there.
Trashing our great trade deal to enable us to have a different visa system for French or Polish workers isn't going to stop a weirdo arriving from there or from Asia,America,Africa. It only happens occasionally and always will- whatever system you have.