South-facing is ideal! It means you should be able to get loads of light and sun.
Garden centres can be very hit-and-miss. A good one is a place of rare wonder, beauty and excitement. A bad one will flog you things grown in continental greenhouses and unsuited to soil that will die on you within weeks. Unfortunately, both tend to be rather expensive. While this is worth it for the good ones when you have an established garden, it can make the cost quite prohibitive when you have metres and metres to fill at the start.
I definitely recommend the discounters as a strong alternative (Aldi in particular, but also Lidl, Morrisons, and particularly Wilkos who do all kinds of really good practical kit cheaply. Seeds, bulbs, shrubs, perennials, climbers, and even fruit you can get very cheaply from these places - we're talking £3 for a fruit tree in Aldi. If you get there early, the plants are excellent quality. I also highly recommend the Secret Gardening Club for more unusual plants. I've bought loads of stuff from them now, and everything I've received has been tiptop.
One of the main things to find out is when to plant things, because this makes a big difference. Right now, we're at the last chance saloon for planting bare root trees - these need to go in very, very soon or in the autumn, but it's a bit too early for many hardy perennials. It's too late for spring bulbs (unless you buy them flowering, which is expensive) - these go in during the autumn (check out J Parkers for them - they're brilliant). Many summer bulbs, however, can go in now.
Sweet peas - these are usually annuals. You plant the seed now in loo rolls or little pots, they'll flower in the summer, then they'll die. They are one of those plants where there is a proper 'official' way of sowing them which is a lot of faff - many people swear by just shoving them in the earth instead! If it all sounds a bit difficult, there are perennial sweet peas which are easier and less bother - these will climb through other shrubs quite happily and come back year after year. Some are scented too.
Fruit is very, very easy to do and worth considering if you like things like raspberries, currants, rhubarb, strawberries and have the room. Maintenance is minimal, and the rewards are brilliant. Veg is another story - in many ways, it's the biggest gardening challenge to get right. However, even then there are easier things, like courgettes and pole beans that have high yield and are just fun to grow. Pumpkins are also easy can be brilliant fun for kids - especially the gigantic ones.