I've just (au pair arrived last week!) used an agency as it's our first time - I can imagine doing it myself via the web in the future when I have more of an idea of what I'm doing.
I like knowing we have the back-up of the agency in case anything goes wrong or we have questions about what's reasonable.
The agency we used (Brick Au Pairs) is very specialised - it recruits Hungarian au pairs for the north of England!
It was about £300 which saved us a mass of time as we didn't have to look through 100s of cvs or send out lots of emails that never got answered. They sent us 2 profiles of which one looked perfect and she has turned out to be just as nice as she sounded.
They justify their fee by the fact that they actually go to Hungary and interview the girls themselves, so you can talk to someone who has actually met them. They also give them an English test and check and translate references.
I also tried an agency which just has connections with lots of agencies in different countries and was not so impressed with them. I also would not touch with a bargepole any agency that asks for money upfront just to register.
You could ring up a couple of agencies for a chat and quiz them about what they're offering? It might also be that if you're quite mainstream - eg in London, where most of the ones on the web seem to want to be - it's easier to use the web than if you're in the sticks or different in some other way - when I looked on the web I could hardly see anyone who wanted to come to where we live!