I am in SA OP.
I know you will have an amazing trip.
I see you are booked for late March so this is going to fall in to the Easter holidays, so it will be a bit busier than off season, but nothing like December (which is just so crowded it can be unpleasant).
I think you do have to see all of the sights mentioned, as you really can't go to CT and not go up the mountain, go to Boulders etc. But there are some things that I never see on these threads:
Will you have a car? If so you will definitely have better access to going more places.
You can take a boat trip from Hout Bay to Duiker Island. It is a short around 45 min - 1hr minute, but this little island is home to 1000s upon 1000s of seals. You can book a trip right at the harbour there, they go out a few times a day. I can assure you, you will not see more seals in one place than here lol. It has been a while since I went, but you can choose from two, one returns via Chapmans Peak, so such a stunning view. The other one detours to see the shipwrecks on the way back. Then there are quite a few nice restaurants right by the harbour, mostly seafood which will have just been caught.
Kalk Bay is just so gorgeous. It is a small little place, but there is a restaurant there I can recommend, The Brass Bell. It is right on the water and the waves crash right up onto the windows. If you sit right inside it feels like you are sitting in the ocean. You could do that stop on the way back from Cape Point, as there you will have a view you don't want to miss. Wherever you go, pack a cozzie. There is a tidal pool next to the restaurant and you can take a dip. Fair warning, the water is kak cold in CT. (you will hear that word a fair bit i reckon )
If you have a car, you could head up the West Coast for a day trip. If you were to head to Langebaan, there is a restaurant experience which you should try. You do have to like seafood though 😃 Look up Die Strandloper. It is a set meal, but about 8 courses of all the local fish and foods from the area. And the setting is right on the beach, just stunning. You can even take a coolbox with your own wine or buy from the bar there.
Do not be hesitant to Uber. So many people Uber around to get from A to B. Also consider a weekly ticket for the MyCiti bus, and then you can plan a trip and just get off at certain stops. Including Kirstenbosch Gardens. They often have concerts there, so you might be lucky enough with your timing.
There is so much to do. You can even do amazing hikes if that is your thing too.