A placid three year old can still be a very sharp, unpredictable 4-5yr old, there's a reason they call it the 'terrible 5's'
.
If you're very confident handling and riding, and love having a long term project, then maybe a 3yr old would be fun. Bear in mind it'll be 3-4 years before they're properly doing their job. Just because you haven't had a youngster before, doesn't mean you can't do it (everyone has a first youngster) but you'll need people who are experienced with youngsters around to help and advise.
Having had a couple of very teenagery 5yr olds in years past, I was surprised initially at how placid my 4yr old was when I bought her. She'd been broken in a 3yrs, ridden on a bit and sold on at the sales in Ireland, then turned away for 9 months. I bought her from the field, and within a few weeks of work, my 11yr old son could sit on and have a short plod (with me walking alongside).
6 months on it's a different story. She's been super sharp and quite bratty at times under saddle this winter. She's now teething (lateral incisors), is very chewy and uncomfortable, so I can do very little with her as she can't settle with the bit just now. It might be weeks before I can ride her again. She's grown and filled out, and looks like a different horse to the one I bought.
I know it's only a guide, and different breeds etc are different, but a 3yr old horse is roughly the equivalent of a 9yr old child. You should really turn them away for the best part of a year, then start again. Even then, there'll be breaks for growth spurts and teething, changes in temperament, testing of boundaries and adolescent brattiness. Just as you think they're coming on, they'll grow again and be out of balance again. It'll be a couple of years before you can even begin to work them hard. I hate seeing photos of 4yr olds for sale jumping massive fences, or doing fancy collected movements - their growing joints and ligaments are being stressed enormously, and they'll likely have trouble with injuries and arthritis later on.
I love having a project, I enjoy watching horses mature and come on, but there are still days when I dearly wish I could just tack up and go for a long, fast hack with my friends, or go do some jumping.
It depends what you want out of horse ownership. Good luck with whatever you decide.