Is this really a question about money...or about fitting in? If it's about money, I think that previous posters, with more recent knowledge than I should have set the OP's mind at rest. If it's about fitting in, perhaps I can add something.
I went to Oxford (admittedly quite a long time ago) from a very ordinary Northern family. I had the time of my life. I'd made up my mind that I was going to grab this experience with both hands, and wring every ounce of fun from it, and so I did. I went one of the posher colleges, and it's true, lots of my peers were from very privileged backgrounds. But you know, posh people can be quite nice too! I ended up being really good mates with a quiet, sweet, old Etonian, who would apparate from nowhere whenever I got my baking stuff out... There were some odd experiences; I remember one Summer evening lying reading on the grass in college when the Bullingdon Club appeared. I'd decided that they were probably a myth, as no one I knew had ever seen them in real life, but here they were, and there was really no mistaking what they were. They were almost tripping over me, but they completely ignored me. I felt entirely invisible. But this was such an isolated experience. On the whole, background just didn't matter at all.
There was a party, near the end of my time, which I went along to with a friend. At the far end of the room was the loudest, most eye-wateringly posh boy I'd ever encountered, spouting off about something, and wearing, if I remember correctly, red jeans. "Give him a wide berth" I thought.
Dear Reader, I married him. For as well as being very loud and very posh, he was also the kindest, funniest, most passionate person (about everything) that I'd ever met. Oh and had the sexiest hands... He likes my northern accent ;-)
So, please Fosgirl, go and have a look, and give it a try. It's not for everyone - I had friends there who had made the wrong choice and were not entirely happy, but I don't think lack of money was part of the issue for any of them.