The history of 2004 accession is interesting
"In the early 1990s, when John Major was at war with his party over Europe, there was one issue on which, broadly speaking, he found common ground with the Eurosceptics. It was EU enlargement. Supporting expansion to incorporate the former communist nations of central and eastern Europe was the golden scenario.
"Wider, rather than deeper" was the catchphrase. By expanding eastwards – so the Tories believed – the European Union would become so big that political union would be impossible. A bigger EU would evolve into a looser union of free trading nation states, with weaker institutions at its centre. It was the way to put the brake on Franco-German and Benelux ambitions for ever closer union, while widening the internal market and promoting stability between east and west. The vision of Europe that Margaret Thatcher had outlined in her Bruges speech in 1988 would come into being.
In 2002, Tory MEP Roger Helmer, who went on to defect to Ukip, put it like this: "Tory policy on enlargement is clear. We are in favour of it, for three reasons. First, we owe a moral debt to the countries of central and eastern Europe, which were allowed to fall under the pall of communism after the second world war. Second, by entrenching democracy and the rule of law in eastern Europe, we ensure stability and security for the future. Third, an extra hundred million people in our single market may be a short-term liability, but long term will contribute to growth and prosperity.""
www.theguardian.com/world/2013/dec/21/tories-conservatives-eu-enlargement-bulgaria
A significant number of people from 2004 accession countries (which UK was at the forefront of advocating for) could already come to UK on the Seasonal Agricultural Workers Scheme (SAWS)