Can I put containers straight into border?

[bananalover]

Have a lot of container plants that I want to put into my newly made soil border.
But the soil is rubbish, full of rocks and new build material.
Instead, I thought I would keep the plants in their pots and just dig a hole and put them in with their pots,IYSWIM. Then put down some bark stuff to repress weeds.

Will this work or am I going to fail badly?

If I have dodgy soil I did a decent size hole and whack in some multipurpose compost for the roots to grow into first and establis before hitting the soil.