Emptyshell that isn't a sign of plankdom - it's an example of dogs not generalising very well :)
To him, "Down" means a situation as much as it means a position and 99% of dogs would do the same as he does unless trained otherwise.
I taught my dog the distance down by teaching her to go to a target and lie down.
Do you use a clicker at all? It's vastly easier if you do.
I used one of those plastic lid things for keeping beans/pet food fresh. Clicked/treated for any sort of interest. When she was reliably going away from me to it, I moved closer to it again and started asking her to "down" on/next to it. Then gradually moved away so again, she had to move away from me and lie down on it.
If your dog is really reluctant to leave you, you can always put a treat ON the marker, then click a split second before he eats it. He's still getting his treat. And then graduate to putting the treat UNDER the marker.... then graduate to no treat on the marker, but C/Ting for going to it.
When that became reliable, I started adding a "Down" cue at the moment she lay down. Say the cue too early, and "Down" comes to mean "Lie down in 3 steps' time. You want it to mean "Lie down NOW".
When it's really reliable in the house, move to the garden etc, then the local field etc, in the same way you'd proof anything else.
Gradually ask for more duration, so he has to stay down for longer before click/treat/release.
Some people gradually make the marker smaller but TBH I found that the "Down" cue became so ingrained that before long she just hit the floor when she heard it, regardless of whether she was on a marker.
It doesn't take that long, especially if, as I said, you use a clicker. It sounds more complicated than it is! Then it's just a case of practise, practise, practise. As with the recall command, try not to ask for a down unless you're 99.9% sure it's going to be obeyed. Eventually you should find it's SO ingrained that they almost can't NOT obey!