I wouldn’t leave the flexible setting you already have in a million years.
I was a SAHM until DS was 3. Career wise I never recovered from it (and never will).
Being a SAHM is not for everyone, I did a LOT of stuff with my child when I was a SAHM, that he no longer remembers. But even when I was meeting people and certainly not sitting at home all day while DS watched CBEbies, I found myself often bored, unmotivated and unchallenged. I also missed having interesting conversations with other professional people (sorry to say this but I often thought that most intelligent conversations were taken down to lower levels as soon as I mentioned I was a SAHM). The balance in my relationship also changed, some way and very slowly he stopped seeing me as an equal. As much as he loved me, I was no longer the sort of person he would have fallen in love with.
But the main damage was that when the time came for me to consider leaving my husband, I couldn’t, as I couldn’t support DS and myself on my own. It took me years to get to be in the position to leave. And now that I’m raising my son on my own, I cannot provide for him in the level I could have had if I had not decided to spend 3 years baking cakes, taking him to the park or meeting with other parents (someway he also seemed to have a much better social life when he started nursery ).
Both your kids are tiny, but in a couple of years, things will be very different. If I were you, I would hold in there rather than become a SAHM.