I come from there 'there's 2 ways to keep a horse' school of thought. You can either pay lots of money, or work bloody hard. Both ways work well with a little pony. It needn't be too expensive, but your niece HAS to be aware that there is a daily commitment to provide for this pony's needs, come rain or shine.
Here's how much my Shitland has cost me over the last 12 months;
Hay - £200
Feed - £100
Bedding £40
Feet (no shoes, trimmed every 12 weeks) - £100
Insurance (BHS 3rd Party only) - £57, but that covers any horse I own/keep at home
Vet's fees, including vaccinations and teeth - £150
Worming - £45
I have bought her a new saddlecloth because I liked it (didn't need it!) for £5, and my daughter has had a new second hand outfit which cost £22 for boots, jods, shirt, tie and jacket. I keep her at home so don't have any livery bills, although the mortgage is bigger to cover the paddock. She needs restricted grazing to prevent laminitis, so has a tiny paddock which is shortly to receive an old shed converted to make a field shelter (Hello Clucky - I'll buy you a drink for the shed .)
So in total she cost me about £730 for the year. If I was paying for livery, around here it would be £100 per month, but my cousin up north rents a 5 acre field for the same price and keeps 3 ponies on it.
So that's ponies. (In my case, even the pony was free - gifted to me by a lady who lent her to me but sadly died.) Of course, other posters are right about lessons, entry fees etc, but that all happens regardless of whether you own the pony or 'rent' it. The difference there is that if you own the pony you will have much more access to it, and your niece will want to do more with it.
Horses, OTOH, cost muuuuch more! My big horse costs me about £300 a month, and that's without having to pay livery and not competing. I was paying more like £900 a month when he was on full livery and I was competing him regularly.