This is overtired.
It's not to do with the dummy directly, although inconsistent dummy use won't be helping. You would be better fully using the dummy for all sleep times.
Dummy helps baby get into a deep sleep, which is why baby settles with it. Once baby is in a deep sleep the dummy serves no purpose, baby's mouth/jaw goes limp (as do all muscles when in a deep sleep) and the dummy can and should drop from baby's mouth.
If baby is failing to remain in a deep sleep, that's because baby is over tired. An over tired baby sleeps in a very restless manner: mostly light sleep, waking easily at the slightest thing and struggling to stay asleep. It becomes a self-perpetuating cycle - bad sleep leads to worse sleep which leads to even worse sleep.
The opposite is also true, so good sleep means baby will more easily fall asleep, will get into a deeper sleep more easily and quicker and stay in a deeper sleep for longer. So good sleep leads to better sleep.
In your case the dummy is helping baby get to sleep, which is it's purpose. But the dummy cannot overcome the physiology of an over tired baby struggling to sleep deeply.
To solve the overtiredness, start off with 3 days of "Emergency Measures". By this I mean cancel everything in your diary, let the washing pile up and plan for easy to prepare meals. Focus totally on Any Sleep, Any How.
For these three days dont think about sleep habits, concentrate on helping baby overcome sleep deprivation so that you can reverse the over-tired spiral downwards that you are in. Plan for long drives in the daytime to help baby sleep. Or walk miles, Or bounce baby perpetually in the bouncer, or hold and rock baby continuously.
Awake windows want to be never longer than double nap length and never more than 2h. So if nap length is 30 mins then no more than 60 mins to next being sleep, 45 minute naps mean 90min awake window. If nap lengths are over 1h, don't go more than 2h awake.
Awake time wants to include settling time. So if you have 30 minute naps and it takes you 20 minutes to settle baby to sleep then start this no more than 40 minutes after waking.
Calories are also important at this age. Maintain pre-weaning milk intake and also include you 3 solid meals a day. So include at least one form of calories in every awake time - either a solids meal, a milk feed or both.
Get out of the overtired cycle and baby will settle into a deeper sleep more easily at night.