@mummytippy You say he could easily have afforded to pay for all four of you.
If that is the case, why do you think he didn't? He seemed determined not to do so, so what point do you think he was trying to make?
I would have thought that when you said you couldn't afford it but would have to use your credit card, at the very least he might have suggested that he pay and you pay him back next month but he was determined not to pay, even though it was embarrassing.
I would think about why he was so set on this and base my conclusions and whether I was prepared to continue on that conclusion.
Was he annoyed that you had invited your son? Maybe he thought that your son should be making a fuss of you-making you breakfast, buying you a bar of chocolate or packet of biscuits-and setting you up on the sofa to watch television or read all day.
It's possible that when he saw that your son wasn't going to make a fuss of you, he felt sorry for you and stepped in and was then flabbergasted when you asked if, the same son who wouldn't do those things, could join in on the treat he was providing.
It's possible that his mum told him that he shouldn't be paying for your son.
It's possible that, despite what you think, he was a hard up at that moment in time.
It's possible he thought he was being used by you to get him to pay for you and your son.
It's possible he is a tight man.
You must have your idea as to why he behaved like this and you will have to judge and act accordingly.