It will cost you roughly 110 per year each, so 220 for the 2 adults. Children are free. That is "complimentary insurance", silly name as it is obligatory for Belgian social security payers. That covers roughly 80% of costs.
For GPs, so long as they opt in to the health system called "conventionne" expect to pay 24 to 30 euro per visit, getting 18 euro refunded in about a week. OR sign up to a single GP called "global file" and you'll pay only the difference, so if your GP charges 24 euro, you hand over just 5 euro instead.
For dentists, they tend to want to charge more, but again for conventionne ones, the majority, you'll pay upfront and get a considerable reimbursement. General children's treatment can be had for free. You get incentives to go regularly in the reimbursement system. I am lucky, my dentist never charges me in full, no globalisation for dentists, my dentist waits for me to be refunded then I pay! It's not usual but essential if on low income.
Hospitalization, it's variable and depends on choice of hospital,
There is a maternity hospital called Edith Cavell, excellent but private and charges more than any other hospital in Belgium. It is the hospital of choice for EU civil servants, as they are fully insured!!! If you want to save money, for a normal delivery, you might pay as little as 100 euro in total, even without hospitalization. Depends on the hospital and any complications. If you give birth at home, you'll actually be paid to do so.
Most Belgians choose a gynae attached to a hospital for ante natal, but you can instead choose a GP or a midwife, the choices are becoming less regimented. It's entirely up to you what you choose. There are 3 groups of midwives I know who do ante natal in Brussels in English. There are only a few hospitals where midwives deliver however. In Brussels, Edith Cavell, Erasme and St Jean I think are the only ones.
You will find health care quite cheap here, so long as you have paid your 110 euro per year basic insurance.