I wouldn't put it that way exactly ^ but yeah. I might pray for someone who is an atheist that objects to it and not antagonize them with the fact, because causing them distress, especially in the name of God, is not on.
I will admit I have told someone I will pray for them when they were being really nasty and afterwards looked at myself in the mirror and thought hey hang on, that wasn't good or selfless to do that at all, I almost used that like a weapon. I'm ashamed of that, because that is not the purpose of prayer. I am quite a sensitive person about these things normally (outside aibu 😜 ) so I can see how other people would fall into that trap.
As I said before, I think if someone is antagonising you with it, and you have said "thanks but I don't want to hear about it", then they continue, well you can flat out ask them if they mean to push people away from God in tiny increments by antagonising them in his name, and then drop it rather than spiralling into an argument. I think for a real believer, that when they go away and think about this, or when they next have a proper dedicated prayer session, it is likely to resonate.
I think generally if someone prays for you it is lovely. The purpose of it is not to advertise it to you though, so I think people can respect the idea of not doing that if it causes distress.
I think like a lot of situations, if you are kind and reasonable about it you will get much better results than being scornful or PA about, and that counts for both sides of such a discussion.