I live in Arbroath so in between both cities and I love a day out in either. My main suggestions for both places have already been mentioned though.
Another place you could get to from Dundee would be Kirriemuir. There's not a lot there but they've got JM Barrie's birthplace and you could take a selfie with the Bon Scott statue. There should be buses to there from Dundee.
Dundee to Arbroath is easy on public transport, the 39 and X7 buses take you from bus station to bus station, or the trains are regular as well. There's a 73 bus but it takes an hour to get through to Arbroath, you can use that if you wanted to go to Broughty Ferry or Carnoustie though! The X7 actually runs from Perth, through Dundee, Arbroath, Montrose and a bunch of other places until it gets to Aberdeen, it takes around 2.5 hours from Dundee to Aberdeen though so I'd only really recommend it if you wanted to go to Perth, Arbroath or Montrose, and my DH might actually be your driver. 
If you wanted to see St Andrews, again there are buses there from Dundee. We're a pretty good area for public transport, although you do get the odd cancellation still.
We don't have a lot in the way of shops here in Arbroath but there are some nice restaurants and pubs. If you did come here then the Old Brewhouse and Old Boatyard both do lovely food, especially if you like seafood. And the Commercial Inn has a gorgeous massive beer garden out the back if you just wanted a refreshment on a nice day. All three are in the harbour area. We also love getting chips from The Marina chippy at the harbour and sitting looking at the boats (weather permitting of course, and watch out for the seagulls!). Also at the harbour, Marcos does nice ice cream and there's a lovely wee gift shop called the Bo'sun's Cabin near the lifeboat station. There's also the Signal Tower museum which has loads of info about the Bell Rock lighthouse (which I think there are still boat trips out to, although we've never done that).
In the High St there's decent food at the Townhouse Hotel or if you want some simple pub grub then the Pageant bar is good (it's one of our regular haunts in town even if it's just for a few drinks). One thing you won't fail to find here is a pub tbh, we have a good number of them, although not as many as we had at one point.