What can I put in the border?

[SugarPlumpFairyCakes]

To stop all the soil running off when it rains? See pic.

Some edging.
Lots of different types available (wood, metal, plastic, stone etc).

More plants?

I'd go for edging too, it's so straightforward to put in and doesn't have to be expensive.

Yes, you need some edging that sits a little higher than the soil to stop it spilling over.