I definitely wouldn't let my partners brother take DD out for a few hours on his own, although she is only 6 months old at the moment, but the same would apply if she was 2.
It's not that I don't particularly trust him, although he isn't the most sensible person sometimes but I know he'd look after her really well as he has a little girl of his own, who is 7 months older than my daughter.
However, she doesn't know him well and wouldn't cope well. If she hurt herself or got scared, she'd want me (that's even true for now, so I imagine even more so at 2). She would also possibly push her luck with him, might get confused with what she is and isn't allowed to do and most importantly not be very used to him, considering she only sees him a few times a year, as far as things are going at the moment.
So I really don't think it's a matter of trust I think it's more to do with how your daughter would feel. Explain to him how this would be for your daughter, not you. If he can't see that, he isn't being very respectful to your daughters needs and that's not so nice.
I really think it's the place of your partner to explain this to him though in a clear and concise manner. If he doesn't listen to his brother, he's probably unlikely to listen to you. But maybe you could explain it a bit better? It depends a lot on the relationship you have with your BIL (close, formal etc).
I second what a previous poster suggested. Go all together and perhaps he could take her for 30 minutes whilst you and your partner have a coffee in the café or go and see a different attraction. That way she is getting time with him, but you aren't far away if there is anything bothering her (or him).
If he doesn't have his own children he wont realise how daunting it can be if you have a little child really, really upset. My daughter got really hysterical yesterday and I nearly ended up crying and I am her mother and in general know what to do - so for an unexperienced man to deal with this would be quite difficult.
I also think it depends how well you know the girlfriend too. After all, if you don't know her well you wont know how well she'll deal with your daughter out and about and how well your daughter will respond to her.
I hope this can get sorted so your BIL can have a good time when he visits. :)
Sorry that was a mammoth post.