Definitely it has been worth it - I went to Oxford and it was worth what I learned from how hard I had to work to get in, what I learned from the form and manner of the teaching and worth it for the opportunities it gave me and the friends I made whilst there. That's not to say other universities don't give you those things, but that Oxbridge creates a particular experience in the getting in bit and the work you do when you are there.
It has also opened doors where otherwise I might not have got in, although I'm pretty sure I stayed through some doors based on my experience and ability to come across in interview. Law firms did come courting (I read law) but that happens at other RG unis too. I have to disagree with previous posters re: the Bar and Oxbridge bias, where the postgraduate qualification was populated with a broad range of educational backgrounds and the other pupils in my chambers reflected that range. However we are talking [ahem] 20 years ago and in a discipline that even now wouldn't offer a £50-60k pupillage, so it may be that times have changed.
Other posters have given good advice on what to think about. The college is key and I'd add Mansfield and St Peter's to the list of ones to look at at Oxford if you think the older, more traditional colleges might not be right.
For many subjects at Oxford your entire degree is based on your Finals and your Finals only. The pressure that creates is enormous and that's a factor to consider. The fact that you can withstand that pressure - I sat 8 x 3 hour exams in 9 days, covering course content from the previous 6 terms - says something about you and it's definitely not for everyone.
The public school contingent were there, it didn't matter, we all rubbed along ok and didn't feel it necessary to make a thing of it. My friends from that time are a mix of public school, grammar school and comprehensive educated, and our backgrounds are similarly mixed. I loved being in an environment where I wasn't being bullied for being bright, that had a small college community on one side but the amazing facilities and academics of Oxford on the other. But I do get that not everyone has that positive experience.