It depends. Sanssouci and New Palais are nothing like Chatsworth, the 18th century palaces in Germany are more like a French one, very elaborate Rokkoko. Chatsworth will come over as a large manor house.
I love Frederick the Great and the story behind the castle and for DD who does German nationalism iuntil 1918 for A-level history, it was important as well.
If you prefer wandering around a town then I would skip it.
You book a timed ticket for one castle (Sanssouci) online and then a second one for New Palais when you arrive. Inbetween are several more buildings you can view, depending on your ticket.
We had nothing but friendly and polite staff, it wasn't too busy as well.
If you feel the need for a castle, you could also go to Charlottenburg which is more in Berlin itself.
If you want some downtime without going all the way out, you can alway head to Tiergarten, a huge park next to the Reichstag. No animals there, that's where you would need the Zoo for.
We were actually very surprised how quite Berlin is in many areas. DD is sensetive to noise and crowds of people and had hardly any issues apart from the DDR Museum.
Do also check the museums, some are currently closed.