Well ultimately, it's up to you but I wouldn't give any consideration to a random man who gave me his number in the street like that.
I often get approached by women when out and about; they ask about my hair (long and silver) or my clothes (feminine with pops of colour). Maybe they are PUAs too!!!
I think you probably understand their motivations are different.
Yes, there are some women on here saying message him and you've referred to them as romantics. Maybe they've just got really poor boundaries?
He knows nothing about you and, more importantly, you know nothing about him other than he is prepared to give his number to a complete stranger with a few brief words of flattery.
I'm sure you're aware of the shark cage analogy?
My friend used to tell me that, when they were at university, they would approach women they didn't know on a night out and say, "Fancy a shag." He reckoned they had a 10% success rate. This is no different.
Give your number to 100 random women in the street and maybe one of them will have poor enough boundaries to call. Do you really want to be that one?