Suggestions for beautiful September flowering plant please.

[cupcakes]

I am have just miscarried my baby which was due in September and I would like to plant something beautiful to remember him or her which will flower when the baby was due (start of September).
It needs to be tough and perennial as I would like to enjoy it for as many years as possible.
I was thinking of perhaps a hydrangea but any other suggestions greatly appreciated. Are there any small trees which would fill this criteria?
Thanks in advance and sorry for the morbid tone.

I think (and I stand to be corrected) that you are more likely to get berries/fruits on trees at that time of year. Rowans have bright scarlet berries around this time and certain varieties of crab apple are spectacular with branches covered in golden fruit. I can have a look in my book for stuff like this if you are interested?

Sorry for your loss,cupcakes. I have some purple loosestrife that grows from nothing to 6 ft every yaer and flowers all September. It dies back in the winter but I don't cut it down til spring. I've had it a few years now so its quite hardy. The latin name is Lythrum and its totally different to the yellow loosestrife which is a bit of a weed.
shrubs that flower in September include Lavatera - masses of large pink flowers and potentillas - comes in lots of colours. Also hardy fushsias.

Thank you for your suggestions. We went to the garden centre yesterday and bought an apple tree ('Scrumptious') to add to our plum and pear trees. I really hope it will bear fruit this year and if not we have many more years to enjoy it in.