Hi Aimsmum- Waverley will be much closer than Haymarket, but it would still be a fair walk. It's right down at the bottom of Leith Walk, about 1.5 miles? There are loads of buses that go down there. Let me know if you want more detailed instructions
I shouldn't think a taxi would cost much more than a fiver, but you could get a bus number 16 that would take you right to the end of the road (there are others, that's just the one I happen to know). The bus stops at the side of Jenners (opposite side of Princes St to Waverley). Good luck!