How do I get him to settle without feeding every time (which is what I've ended up doing recently)?
Move all of your feeds away from sleep time, where possible. So in the daytime, arrange your routine so he wakes up hungry and you give a full feed when he wakes from a nap. That way during his floor time (activity time) if he cries you know it isn't hunger and so go straight for a nap, without feeding at all.
Obviously nights are harder to seperate feeding and sleeping but you can help this by moving all/most his calories to the daytime. It's not unusual for calorific need to increase by 25% st 4 months. It's easier to see this increase when bottle feeding so you may not notice when breastfeeding. But if you can accommodate all of those extra calories in the daytime, then you can be confident they aren't needed at night. Then comfort and resettle, rather than feed.
It doesn't just fall out when he's in a deep sleep (this was the case up until about two weeks ago), it falls out because he's grunting or flailing around and we then have to put it back in, or it gets stuck under him which he finds uncomfortable.
This is baby getting stuck in a light sleep, not getting into a deep sleep as he was previously. (That's a normal part of this regression, nothing you've done wrong).
You need to help him a bit more, for a bit longer. The grunting and flailing around mean if he is sleeping, it's light sleep, he could also be partially awake do not aslerp at all. So through this phase keep the dummy rubs or patting going (or whatever you do to comfort) and help him keep the dummy for comfort sucks until through to a deep sleep.
Also look at the underlying cause of more light sleep and less deep sleep. It is usually yo do with either lack of calories and so hunger (see above point re calorific need increase) or cumulative sleep deprivation over the last 24/48 hours, causing restlessness. In your case could also be pain/discomfort from digestive allergies.
For the dumny going under his body, you could consider sewing a short ribbon (No longer than reaching his ear) onto the chest of his sleeping bag/swaddle. This keep a dummy handy, helps you find it in the dark and stops baby rolling on it.