Social Services would usually let family foster or adopt kids where the parents can’t care for them - which often leaves the kids who are older and with needs their families didn’t want to take on.
That’s not remotely true. I’m a child protection social worker and a parent through adoption. There are many reasons why a child might not be adored by foster carers, not least because fostering and adoption are very different things, the legal status is very different, many foster carers don’t want to adopt and the linking and matching process for adoption is more considered and complex for adoption whereas placement in foster care is often about an available place for a child in the short to medium term.
Depending on where you live and your local authority approach to permanence planning you could find yourself placed with an under 2, it’s not unusual at all.
I would give real consideration to what others have said though about the time, care and attention you have available for another child. With children who are placed very young there’s often a lot of uncertainty about their future outcomes because many of the issues are developmental and don’t show themselves for quite some time. With older children, issues are much more likely to be known.
When people are talking about stress impacting a baby in the womb, they’re talking about extreme levels usually involving high levels of poverty, violence, abuse and uncertainty - if that’s not part of your pregnancy experience I wouldn’t worry too much about being stressed in pregnancy.
The last thing I’d suggest is thinking about why you want to adopt, SW will want to explore this very fully given you have 2 birth children. (You don’t need to answer that here btw, that’s not what I’m asking) Adoption isn’t by any means the preserve of the childless but given the challenges and complexities of bringing a traumatised child into a family with birth children, they’ll want to unpick that with you.