We've always enjoyed UK holidays with young kids (but had a bigger age gap so never 2 under 5 which obviously would be harder, but I guess is harder at home too!). We've always enjoyed a caravan with a fenced veranda for gated outside space / al fresco eating, near the sea and some family friendly days out (south Pembrokeshire, Devon and isle of wight all fit the bill). You need to lower your expectations of what constitutes a holiday, but I've never had one I regretted and kids always had happy memories. Buy pastries and variety pack cereals for breakfast, picnic stuff for lunches and then plan a mixture of takeaway, early evening pub type meal out and a few minimal cooking meals or bbqs- book a supermarket click and collect or delivery for first day to keep things easy. You can pack much more stuff if you stay in UK (or drive and ferry I guess) so you can take whatever makes life easier for you with your kids.
We initially did the sites with lots of facilities, which means you have more rainy day options, but mine wouldn't usually take part in the kids entertainment and we found we only got round to using the pool a couple of times mostly, so now we are more likely to choose a more basic site or a holiday cottage, ideally walking distance from beach, so not every outing is a big day out, but can just have a quick walk for an hour between rain showers sometimes!
We've never had the weather ruin a UK summer holiday - you accept you won't be sunbathing but who has time for that with young kids anyway?! Plan a few rainy weather activities (going out for breakfast, indoor swimming pool, zoo or other attraction which is partly undercover, amusement arcades, souvenir shopping, going out for cake, film afternoon in the accomodation, board games, sticker books etc), and get out between showers. I don't think we've ever had more then one torrential rain day on a week summer holiday.
If you do go abroad, we've only done it twice,and with older kids, but I'd say look for hotels which advertise 1 bed apartments rather then family rooms, as you usually then have a sofa bed in living area, so you can sleep there, but kids in the bedroom and be able to use the living space and balcony while they sleep. We haven't struggled to find these. I also love AI abroad as Tesco deliveries aren't an option and can't pack as much stuff on a plane, so self catering seems harder work somehow.
I think holidays with kids are easier then keeping them entertained at home, as long as you choose well and don't expect it to be 'relaxing'.