Dress for work and pop a change of clothes in bag? 
I guess I’m just using my experience of OD - there is nothing worse than being stuck with someone weird who looks 15 years older and 5 inches shorter than his picture, in the nearest convenient “local” pub whilst looking properly glamourous (so it’s pretty obvious you’re on a date
) Wrap style dresses can have a bit of a hypnotic effect on some men, so if he’s on the creepy side he’ll be pawing you all night 
Plus heels you can’t walk in aren’t a good idea, especially if it’s just at the “casual first meet” stage?
If you hit it off you can always wear the dress on your next or third date, and suggest something more formal, like dinner or somewhere swanky ...