Apologies if this posts twice as seems to have disappeared so typing out again.
Name changed and a few details altered slightly.
Ds (4) was unplanned. I was on the pill, ex refused to use condoms, refused to have a vasectomy even though he was adamant he didn't want anymore kids (he had an adult dc from a one night stand, and a now adult (just turned 18) dc from his marriage). I'd already ended the relationship when I found out I was pregnant so it was a massive shock to us both. He wanted me to get an abortion, called me selfish, talked of suicide, said another child would ruin him financially, you get the idea. Ds was very much wanted by me although a huge shock at first and is a lovely addition to our family and I have other dc who adore him.
I informed my ex of the birth and sent photos at regular intervals but had no response so I gave up for a while. Then when ds was one I arranged to visit his dad and travelled the 100 miles by train, booked into a hotel and spent a few hours in his home city with him. We went our separate ways early evening and he said he'd like me to visit again so that his middle ds could meet LO. The next day I asked about meeting up again before we headed home but he had plans gaming with mates and said no. I never heard from him again.
Ds has started to ask questions about his dad and I don't know what to say. He knows he lives far away and knows his name and that he has a brother but that's as far as I've got. He's only just four so skips from topic to topic quickly (thankfully). His brothers mum (my ex's ex wife) sends presents every year but my ex had never sent so much as a birthday card. His ex wife is lovely and I appreciate her kindness towards my ds but it highlights the lack of contact from his dad. His other brother has never been in touch but has my number and other contact details. Makes me sad.
Ds asked me to phone his dad and ask him to visit us. I didn't know what to say. I know he wouldn't visit as would cite no money. That was his reason for not seeing him ever. He claimed it would be easier never to see him than only see him occasionally. Sounded like a cop out to me. I'm pretty sure he'd agree to meet up if I offered to go there but it was always me who visited him and I feel he should make the effort to see his son. I think he only met up last time as was no effort on his part except to catch a bus. And he was still nearly an hour late 
He does pay the csa rate of maintenance (probably of little relevance but posters always ask on threads about absent fathers).
I don't know what to say to my son. His father never wanted him and has never made any effort in seeing him or even keeping in touch. He didn't even acknowledge his birth ffs.
What would you say/do?