Muhammad Ali called it his "slave name" largely because that was what the Nation of Islam told their followers when converting and recruiting them. It really doesn't have any reflection on the name itself.
OP, in case you couldn't tell, I'm a fan of Cassius
I really like Cassius- a really attractive boy's name. Caleb sounds a bit try-hard/trendy/cliche to me unless you're Irish, in which case it's fine.
Of your girls' names only Adeline jumps off the page at me (in a good way), but I prefer less used names. Alice is nice and classic so that's a good choice too. Violet is lovely but becoming very popular (if you care about that!). Matilda I just don't like- not sure why, maybe it just doesn't have any prettiness about it.