Female condoms are difficult to get in the UK. (They are more easily available in Africa, where the risk of HIV is higher than here.)
A cap is a diaphragm, not a female condom. Diaphragms can also be harder to get these days - you have to get them properly fitted, and a friend found recently there's no trained nurse available do it in her GP practice. As someone pointed out upthread, there's little money to be made. If they're not training up more nurses, they will become even harder to get.
Which means condoms are the only real option as a barrier contraceptive. There are lots of different ones available these days, including ones thst are meant to be almost unnoticeable to feel.
In my dating experiences around 50, there have been a couple of men who couldn't keep it up long enough to get a condom on, but most were fine. Men of this age are very reluctant to use condoms though, which has surprised me, as we reached puberty at the height if the AIDS epidemic, and we were all going to die from it - and if I'm up for shagging one bloke on a first date, I've probably done so with others, so why don't they want to protect themselves, let alone me? I did regular STI tests anyway, I wouldn't rely on them, but the stats on STI numbers is no surprise, the way some of them behave. I asked them to wear a condom, not go out and drown kittens or gun down their parents.
So yes, just go for the condoms, OP. You can make it fun.