Sadly MN is full of stories about dad's ( and some mums ) who don't stick to arranged access ( I'm tired, hungover, busy, can't afford it, have to work etc etc etc ) and unfortunately you can't make them stick to it. You'd think they'd put their kids first, but alas no.... so I don't think you can make him stick to anything if he's being a dick and being irresponsible. All you can do is re-enforce that his dc will suffer if he doesn't put the effort in.
Before my dd started staying with me ( I'm an NRP ) I would take her out to museums, parks if the weather allowed, swimming, toddler play at the local sports centre, play grounds ..that sort of thing. But thankfully I get on ok with my XP so I'd also go to her place ( she'd go out ) and I'd play with dd there sometimes.
And to be honest, certainly in winter, I wouldn't want to be taking dd out all day.
You shouldn't feel you have to go along with the two of them, possibly / likely he lacks the confidence to have dc all day, you could help him with that maybe. i.e. making sure he goes out with all the stuff, spare clothes, coloured pencils and paper, picture book, sticker book, changing kit, food and drink etc.
So yes I think it is reasonable that he takes her out, and if sometimes you let them play together at yours whilst you go and do something else then good for you.
Ultimately though he needs to sort out a suitable home where he can take care of your dc, and even then he'd need to plan stuff to do.