I've known a few Fabians, mostly German or with German connections. I like it. But yes, in the UK a lot of people would think Fabian Society (not necessarily a negative, though - depends on your politics!).
It later had Atlee as a member - it was one of the movements that laid the frame work for the Labour party (I'm positive Blair and Brown where at one time members) It is now a left think tank and a socialist society affiliated with the party. The young Fabians are also well known