There is some really good advice and suggestions in this thread.
My mother has had dementia for 10 years. She still lives at home. She has until relatively recently enjoyed visits from my son, who has enjoyed seeing his nanny, although after the initial very short period, he does leg it into another room with his IPad - not just normal boy stuff, but she has never really been able to sustain interest and then ignores him completely. He has coped quite well with it, and has quite a sensitive caring personality. He also has a serious medical condition himself, has been in hospital a fair bit, and pretty much takes all sorts of disability and unusual behaviour (as long as it's not very violent) in his stride.
That said, her condition has recently deteriorated, her behaviour has become very unpredictable, with aggression to me on some occasions. She no longer recognises me, or (on occasion) my son. My son has started to not enjoy the visits, and has become anxious, so now I either don't take him or OH comes too and can take him off. Some of the things she comes out with are distressing too, and have affected him - thinly veiled suicide threats, and very nasty comments about us and other family members, many of which are paranoid ravings - but it is particularly upsetting for my son to hear this about his granddad who he adored when he was alive.
TBH I am not very good with dementia, and it also puts me in a foul mood for the rest of the day, so, for various reasons, we are cutting back on visits - they no longer seem to do either of us any good.
In a care home, obviously, you have different issues. However, if your father has expressed a desire to see your daughter and has good days, when he is relatively lucid, then my feeling is he would get some benefit out of it. It really does depend. From my own experience, children can bring a smile to those suffering from dementia when nothing else can. That said, obviously you have to weight it against the effect on your daughter. If your daughter gets upset, or anxieties surface, then personally, I wouldn't push it. Yes, it is good not to be over-protective, but also important to keep happier memories, and deal with scary things at a speed they are happy with (and 6 is still very young).
Meeting on neutral ground away from the care home could be a possibility if you could manage it. However, from my limited experience, this can also be frightening for the sufferer (my mother can't stand the slightest noise and becomes super-cantankerous - we spent a very short and miserable family meal with her in Pizza Express with her hands over her head and moaning loudly). A family room if they have one may be a good idea though, I could see that working, or in the garden on a fine day if they have one - there may be some less threatening place in the home you could use?
I do think taking your OH and letting your daughter go off with him after a short time, or if things don't work out, would be a good idea. Maybe take a small personal gift, or leave a photo behind? Also the suggestion of associating it with a treat or pleasant experience afterwards is a really good one.
Only you know the (changing) personalities involved. However, do think about how to manage the goodbyes. My mother now has a habit of either getting very upset when we leave, or violently telling us to clear off and slamming the door in our face. It sounds really mean, but I have given up doing the goodbye thing. I just leave her in her chair, tell her in matter of fact way I am going and let her carry on watching Jerry Springer or whatever. She has no recollection I have gone, or was ever there.
It is I think doing the best you can for the personalities involved - a parting gift maybe, or getting one of the carers to break things up with a distraction - tea, I don't know, and it may not be an issue for you, but it can be beneficial to think of "what if" scenarios in advance, and do as much as you can to make things easier.
It is one of those situations, where there is no "right" answer. If you don't visit, you may regret that, but there are real positive reasons for not doing so - remind yourself of those if the guilt starts to kick in. If you do visit, yes, it may be upsetting - again you just have to play it by ear and think about the benefits. The reality is you may find yourself torn in two ways, having to negotiate a middle path that makes sense to you.
Listen to your other relatives, but don't be swayed by them. They don't live with your father, and they don't know your daughter as well as you do. You have to live with the situation, and any fallout, and it's your decision. Whatever you decide, it will be the right decision for you.