We have done it. DH has 2 children DSD (8) and DSS (11) and we have DS (2). He was a bit of a surprise and I'm not sure we'd have planned things this way, but I knew I definitely wanted children at some point.
As it happens DSD and DSS have been brilliant from day one. We were worried about issues but aside from a bit of detachment from DSS during pregnancy, it's been great. Once DS arrived his big brother and sister doted on him. We've made sure they are very much involved the whole way through.
DSS and DSD came to live with us full time a year ago and will be here until they are able to live independently. The house is chaos (you people with 5 children and a calm house need to teach me, I'm begging lol) and they have a totally normal sibling relationship. All three squabble, annoy each other and have moments of being lovely to one another.
I wouldn't change it for anything :)