There are some lovely beaches jsut up the coast from caister, can't remember the name, maybe scratby area. It had a nice beach and shallow water for a long way out so it great for kids, someone local might know it. Caister beach is all dunes, and there isn't much else in caister. I can remember a castle and a motor collection. I also liked takign the kids to california. Close to great yarmouth will be ideal for keeping the kids entertained, plus you are only half hour from pleasurewood hills or even Norwich (castle museum is good for kids). It isn't even too bad to get up the coast to sherringham, hunstanton etc. I'd say caister is quite central and a good choice.
I grew up in gorleston, jsut the other side of caister.