We went through the exact same thing. From what I was told, four months is the point where their sleep cycles start to mature. So now they're cycling through light sleep then deep dreaming sleep and in babies, each cycle lasts about an hour or so.
It's really common in normal sleep to wake up and go back to sleep between cycles and not remember it. But in babies, they haven't learned how to link up their cycles and fall back asleep.
Your baby is probably used to getting fed to sleep, or at least mine was. I had breastfed her off to sleep her entire life, so that was how she settled to sleep and when she started waking up between sleep cycles, she didn't know how to settle herself without being breastfed back to sleep.
I didn't really like the idea of leaving her to cry it out, but what worked for us was gently breaking the fed-to-sleep association gradually by changing her schedule so that I was feeding her on waking up, not right before a nap, and putting her down for a nap after playtime, not after feed time. Same with nighttime, I'd try to give her a feed about an hour before actual bedtime, so she was learning how to fall asleep without being fed to sleep.
Eventually, after a few weeks, she stopped needing a feed to get back to sleep and now at six months she's back to sleeping through, with just the occasional night feed.
The four month sleep regression is hell, for sure, but you'll get through it!