My honest opinion... you're both getting into it, and he feels it's working because you seem to be reacting positively...
...but the moment is lost because his beard is irritating you.
At that point, he feels stupid. It's an irrational reaction, but it's likely what he felt. From going from 'great lover' to 'it's not working for me' isn't going to be a pleasant moment.
So he's immediately reacted by criticising your technique / movement too.
It's silly, and it's from embarrassment (I feel). If you'd continued and orgasmed over him, he would not have complained in the slightest, of that I am sure.
You need to talk to him and explain the beard thing. He needs to understand how sensitive a vulva is and it's nothing to do with him per se, but a beard rubbing against it can eventually start to hurt. There are products to soften his beard, or he can try to change technique to minimise the rubbing, or just lose the beard.
If it were me, I'd lose the beard, but it's a big deal for some men.