There was a great thread recently about things to do in Edinburgh:
www.mumsnet.com/Talk/uk/3243337-Best-things-to-see-and-do-in-Edinburgh
It went to several pages.
Edinburgh is fairly compact and the buses and trams are great so don't be put off by a hotel a little further out if it's cheaper. Just check it's close to regular buses/trams.
I've no doubt it was mentioned on the other thread but be prepared for rapidly changeable weather.
Scottish bank notes are unfamiliar in England so shops don't like to take them but it's not the case the other way around.
I recommend doing a tour of the underground streets like Mary Kings Close, there are several companies offering, mostly advertising on the High Street, can't miss them.
Have a wander all the way down from the Castle to Holyrood Palace, kinking down Victoria Street, round the Grassmarket and up Candlemaker Row, past Greyfriar's Bobby and back to the Royal Mile. There is a street market in the Grassmarket on Saturdays.
Highly recommend getting even slightly off the main tourist thoroughfares for nice food at better prices, e.g. Rose Street instead of Princes Street.
If time and strength permit, also Arthur's Seat, the Botanic Gardens, out of town Stirling castle and Linlithgow Palace are pretty easy to get to. Stirling Castle is probs a better day out than Edinburgh castle if we're honest, if you're doing all three you're getting into the realms of joining Historic Scotland and getting money off in the café.
There was loads of great suggestions on the other thread.