Also, there were a LOT of people between him & the crown throughout the 1470s.
EIV was hale & hearty, with an heir rapidly approaching his majority, a spare & 5 surviving & marriageable dds. Then there was Clarence, & his son Edward of Warwick.
I suspect Richard was very happy being EIV's reliable (ie. 'not that twat 'false fleeting' Clarence') in his brother's eyes. He had the North, huge wealth, was married to the surviving Warwick heiress.
Suddenly, Clarence is executed, quite possibly with Richard's complicity but to be fair he'd definitely been pushing his luck. EIV dies, & the new king is a 12yo under the control of his hostile maternal family.
Basically, Richard can wait to be swept from the board by a y7 student who is being raised to see him as a threat & to eliminate him ASAP, or he can act decisively - & he does.
If RIII had won at Bosworth - & he does appear to have been doing a fair enough job as King - we would be acknowledging that his potential rivals, including children, did tend to come to grief or go AWOL. Just like those of every other ruler in history. The PITT are only 'special' because we don't know for sure.
It's the same impulse as the interest in Jack the Ripper. Everyone loves a cold case.