I wouldn't describe this particular London scene as 'casual'. 'Low key' or 'undressy' perhaps but, from what I've seen (at family events and similar) it's a very considered look. The insistence that no-one notices what you wear is somewhat undermined by the references (on this thread and others) to 'out of towners', along with a photographic recall of outfits spotted in Kent, Manchester and Liverpool. Basically, I don't believe no-one cares and I can understand your anxiety.
That said, I wouldn't buy anything just for this occasion, especially when you've got other demands on your cash. However lovely these people are, you didn't keep in touch and it's quite possible you won't see them again for at least another 10 years. But if you do buy something new, I think the advice to go for upmarket versions of what you'd normally wear is good. In particular, I'd avoid a new jumpsuit because, from memory (I've not tried one this time around) they can feel a bit weird the first couple of wears and that's the last thing you need.
If it were me, I'd probably aim for a ratio of two dress-down items and one dress-up. So I might wear my favourite jeans (or it it's too warm, some ankle length, loose fitting trews or combats) with a loose fitting silk shirt, and then on top, something like one of those canvas 'workwear' jackets. Then if I realise I've misjudged and everyone's looking smart than expected, I can just take off the jacket. Or alternatively, I might wear the jeans (or whatever) with a T shirt and, over that, a vintage 50s/60s jacket, which I could take off if it felt too much, or leave on if not.
They're all things I've already got in my wardrobe and I'm not suggesting you go and buy similar but hopefully you get the idea: a 2.1 dress down/dress up ratio with layers that you can either keep on or take off once you're there and you've weighed up the scene.
Hope you have a good time.