Yes, I find pencils smudge, as do powders and waxes. Unless you are dripping they just about stay put but if you touch, but one careless sweep of your fringe or eye-rub and the gaps are back.
My current favourite is the Soap and Glory Archery pen, it has a waxy pencil at one end which gives the coverage and a felt pen at the other which is dark in tone but light in coverage, use it on top of the pencil and it gives it much greater staying power. It is the only felt pen one I've tried that hasn't got a reddish tinge to it.