I hear you, I used to be terrible at small talk, excruciatingly so really, so I had to teach myself. Wouldn't say I'm an expert now, can still be a bit awkward at times but nowhere near as bad. The key message I drilled into myself was not to worry too much about seeming really dull or inane, small talk with strangers or work acquaintances or tradespeople or whatever is not the time you need to be demonstrating your sparkling wit or searing insight into cultural affairs, it's really a kind of noise to fill the silence! Mainly people's perceptions of your confidence and likeability come from how you say it not what you say, so smile, open body language, make eye contact, ask questions and listen to the answers and ask follow-ups (remember people tend to really like to talk about themselves so this is a good way to not have to talk too much yourself!), titter politely at their jokes etc., and even if your chat is pretty lacking they'll likely think you're a nice person anyway. And it's fine to be a little nervous, even if it's informal obviously you know you're there to be sized up for a job you want so only an arrogant monster would be totally cool and unphased by that!
What I would have in my head as a programme for this kind of event is 2 or 3 'general' topics for when you arrive, are being walked to the lift etc and 2 or 3 'work' topics for the 'informal chat' itself. General topics would include things like my journey there, the traffic/public transport, compliments to them on their office/building/location (unless it's some kind of hideous portacabin on a A road flyover affair and you risk seeming sarky of course!), the weather, the fact it's Monday (start of a new week, so much to do!), Wednesday (nearly over the hump now!) or Friday (looking forward to the weekend!). Really dull and cliche I know but remember the above, it's about being safe and having something to say for yourself rather than being really interesting! 'Work' topics would be anything of note in industry news, asking them about any particular developments at the company or challenges (although keep it light/positive unless they themselves start off a complaint or gripe in which case nod and smile knowingly - don't bring up that PR disaster or court case they lost or their impending bankruptcy/hostile takeover unless they do first!), if they have anything interesting on their website like a blog post or news article or press release or a new product, ask about that. I think it's also fine to ask lots of questions in this kind of scenario about the job itself, the team, the culture, clients, any exciting or interesting new projects or developments on the horizon, and the process/next steps after today, like a little mini 'any questions for us' section of a proper interview?
Good luck!