If it's your cup of tea there's Landguard Foret at Felixstowe (not far as the crow flies, but it's the long way round to get the other side of the river). An intact fort from Henry VIII, but beefed up throughout history. There's 2 levels of buildings, quite extensive tunnels to explore.
Took DDs 3.8 and 2yo there last week, a bit young but they loved it.
10 mins further up the A12 is Orford castle, a small but fairly intact Keep again the DDs enjoyed it (especially getting in the huge oven in a room on the roof - room for several more children *disclaimer - it hasn't been lit in a long while)
Tilbury Fort must be fairly close as it's on the Thames on the Essex side. Again a beautiful star shaped fort.
These are English Heritage (EH) sites, if there's more near your starting point might be worth joining for a year.
St Botoloph's Priory is a proper ruin in Colchester town. Avoid rush hour times for travel in Colchester center as it's a bit grim for a while). It's free which by EH standards means good for a walk round to knacker the darlings out and a picnic but not for spending hours there.
A little further afield is the 'secret nuclear bunker' at Kelvedon Hatch. There are a lot of steps but it was really good and is one of our Zombie Apocalypse plans, but that's another thread