OP, you will be forever waiting your turn to use your own garden, if you let this self-imposed, turn-taking rule live in your head any longer.
Use your garden whenever you want to, nobody thinks it’s bad manners to do so, so why restrict yourselves. There is no rota, no agreement, no unwritten rule that everybody is supposed to know about for good manners sake, so just go for it and get used to it before writing your neighbours off.
You and your DH do sound rather private/reclusive and that’s fine, but the norm is for people to be friendly, enquiring about each others lives, sharing their own experiences and finding mutual ground if there is any, on which to build a satisfactory neighbourly relationship -enough to make you feel settled at least.
If you really don’t want that kind of contact, then I don’t understand why you are upset that your current neighbour has managed to do just that with her neighbour on the other side.
You can’t make or expect your neighbours to be the same as you. ( maybe they are but you just haven’t bothered to find out yet) We are mostly all different which to my mind, makes life interesting. A good community is one where everybody feels settled, uncomplaining, not judged, and able to contribute just to the extent they are happy with.
If you are the sort of people to be polite and civil but stop at getting to know other people’s business then there’s nothing wrong with that at all. But do it with an open heart without feeling you need to please people or without feeling you have been left out or without feeling you don’t belong or without feeling guilty.
Whatever type you both are, I still feel, it’s best to not only be accepting of your more established neighbours, but to put yourself out of your comfort zone just a little bit to reach out to one or two of them in a friendly way to introduce yourselves.
You can’t really complain about them OP, if you don’t know about them and haven’t made some sort of effort and if they aren’t doing anything wrong.
You certainly can’t complain that they make you feel left out and unable to go into your garden if you are the one wanting to build barriers between you before you even try to establish a rapport.
If you are going to build a higher fence, then I think it’s right to talk to your neighbours about it first. It’s not as if they are being loud or overbearing other than that they exist and quietly use their garden. You are right that they might wonder why it’s necessary.
Being home birds is lovely OP but you are not going to be able to love your home with the attitude that your neighbours are selfish for using their garden too much and not giving you a turn to use yours.