Great list! I'll add some hikes for people who are staying close to the capital area and maybe have small children like me that limit how adventurous you can be:
Esja - the mountain you can see across the bay north east of ReykjavÃk, just outside Mosfellsbær. A very popular hiking trail with locals and brilliant views over the capital area from the top. Úlfarsfell is another option in the area which is smaller, so easier.
Glymur - a brilliant waterfall at the head of Hvalfjörður. Easy day trip from the capital. Steep but worth it.
Heiðmörk - Very nice area for walks in the capital, just next to Garðabær. Wooded and good views. A few short easy trails suitable for younger kids, as well as longer ones.
VÃfilsstaðavatn - A lake near Garðabær. The path around it is a lovely little walk for children and stunning when the lupins are out.
Elliðaárdalur - A big park in the city with a nice little waterfall. Paved paths and easy walks. Good for children and people who want a walk but don't fancy a full on mountain hike.
Also if you hire a bike or even an electric scooter, there are so many lovely paths around, many of which join up nicely. You can go all the way from Seltjarnarnes to Elliðaárdalur or even on to Grafarvogur and Mosfellsbær (the coastal path between Grafarvogur and Mosfellsbær is wonderful).
And this is not in the capital area but I am mentioning it as my favourite hike I've done. If you are near to VÃk on the south coast and have a 4x4, I highly recommend Hafursey. Stunning views over Mýrdalsjökull, just incredible.
Honestly almost every part of Iceland is gorgeous. It's like a wonderland for people who love the outdoors.