Yes, technically it should be five years' experience - as per Lynne Truss's discussion of the film Two Weeks Notice and how it should have been Two Weeks' Notice However, almost everyone is unsure and nervous about this particular aspect of apostrophe usage so if you've already written it without the apostrophe, I doubt anyone will say anything.
Whenever youre wondering whether a phrase like this needs an apostrophe, try mentally substituting one for two (or whatever other number is in the phrase youre wondering about). Two years experience and two years experience sound the same -- but one years experience and one year experience are easy to tell apart. And of course youd say one years experience. Youd also say one weeks notice and one dollars worth. Just remember, when you go back to your plural, to put the apostrophe back after the s, where it belongs on a plural possessive.
It's because it is possessive - it means '5 years of experience...' if you wrote it out in full. And the apostrophe goes after the s because 'years' is plural. It would be '1 year's experience' if that was the situation. HTH.