I'd think very, very carefully about this.
At 24 you are young, and could be living a relatively care free life.
Being nailed down to a man just about to hit his 40s with 3 kids ...is a lot of work, and hassle. I say this is a wife of a 38 year old with 3 kids. I'd be concerned about how much responsibility you will be roped into when he has his kids, and if he's already got health issues creeping in. I'm not being mean here, but my DH is starting to fall apart already!
Where does this leave you, if you want kids? I know from DHs perspective, if we divorced tomorrow, no more children are on the horizon for him. He said 2 felt like enough, 3 is his hard limit.
I'm just thinking, you're going into a situation with 3 kids, if you want babies of your own, then there's more to unpack with him, and then he's going to hit old age 14 years before you, so by mid 50s you may end up caring for him. MIL is 14 years younger than FIL, and has been dealing with a lot of care for him since she was in her 50s. Now early 60s -FIL is 76 and he is dealing with cancer, heart failure, alzheimers and he can barely hear a word. All of her friends with husband in closer age are in much better health.
Unless this man is the absolute love of your life, and you cannot live without him, I wouldn't recommend it.