I am sorry people are making so many assumptions, being so judgmental and hostile. It seems like there have been a few misunderstandings and so you are not getting the empathy and/or advice that you need.
Let's see if I understand correctly: You used condoms and 'pulling out' to try to avoid pregnancy when you first got together but for the last three years you have been having sex around 4 days/week without contraception, feeling safe in the knowledge that if you did become pregnant then you would both welcome it. You are now concerned that you have not fallen pregnant.
Firstly, I'm sorry for your disappointment. I understand your concern and attendant anxiety about seeing your GP and possibly recieving bad news regarding your fertility. It would indeed be a blow but only temporarily as there is so much you can do to improve your chances once you know what's what.
Secondly, some people fall pregnant very easily and sometimes wish they hadn't. Thankfully, that isn't your situation.
However, for most people they need to properly try to conceive. That means taking folic acid, ensuring that you and your partner have a generally varied and healthy wholefood diet, maybe taking other dietry supplements if needed, exercising regularly, achieving a healthy weight, abstaining from alcohol and drugs (- not saying either or any of these are an issue for you, specifically, just general issues for fertility), tracking your ovulation and having plenty of sex at the optimal time of your cycle.
If, having done all this for more than a year, you are still not pregnant, that is the time to confidently make an appointment for you to both see your GP together.
Just to reassure you, I am a mum but it did take time, planning and all of the above.
In my case, we discovered that my fallopian tubes were diseased and had to be removed and my husband's sperm had low motility. This was rectified with a change to his medication, diet and IVF.
IVF is a whole other pain as you know but can be well worth it. My lovely, healthy boy is now 13 years old.
To encourage you on your journey, the reason that I only have the one child is because I met my partner late in life, I was over 40 when I gave birth to my lovely son and by the time I felt my son was independent enough for me to consider conceiving a sibling, I had other health problems.
Having your children young is the ideal for good health all round and will give you the best chance of the family you hope for.
Like PP said, please marry first for your own security and that of your children. If your bf is committed enough to raise a child with you then marriage should be a no-brainer. You can do it 'on the cheap' and still make it special. Maybe start a new thread about that....
Wishing you all the best, OP.