We stayed in Beadnell with our 5mo DS last Sept and had a wonderful holiday. It is such a beautiful part of the country. I'm really jealous of you heading there!
Alnwick has a good town swimming pool (next to the Sainsburys close to A1) which we went to with DS on a rainy day. It's fairly new so has nice clean family changing rooms, a smaller kids' pool etc.
I would second the Gardens and the book shop.
The beaches are wonderful. Our favourite was Embleton Sands between Low Newton and Dunstanburgh Castle which is great for a long walk or just for pottering around. There is a pub in the square at Low Newton which does nice, simple food (local kippers and crab included). If you have a baby back carrier or can borrow one for your 6mo I would recommend it as the walks along the beach are easy and wonderful.
Bamburgh castle and beach was another favourite. Again we walked from Seahouses along the beach.
Craster to Dunstanburgh is another easy walk, this time along the grassy cliffs/slopes down to the sea rather than beach. Dunstanburgh is a beautiful castle - ruined and very atmospheric.
We did a boat trip round the Farne Islands to see the birds and seals and there were plenty of small children on that.
Good fish and chips in Seahouses - can't remember the name but it was a restaurant right on the corner of the roundabout near the Tourist Info Centre.
Holy Island is worth a trip - although check tide times when you're there for access. The Priory would be fun for your toddler to run around the ruins and the walk out to the castle and visit to the Gertrude Jekyll garden is nice too.