"If I would want to apply for a British passport however I would have to give up German nationality."
Really? I though the law was changed about a decade ago to allow both. I'm a British and German national and only got German nationality recently. I don't think it's different the other way round, is it?
"I know someone whose parents are Greek and German respectively and he had to choose which nationality he wanted to be as her couldn't be joint German nationality with another country." AFAIK that's due to restrictions imposed by the Greek government, not the Germans. The Germans are fine with joint EU British/German nationality.
"I just trust the German care system more" Very sensible too.
By the way, if you're not married by that point it absolutely makes sense to apply for the Vaterschaftsanerkennung while you're still pregnant, which would mean you can go ahead and apply for passports as soon after the birth as possible.
As a British mother, I had no problem getting British citizenship for my two DC born in Germany - but because my DC have always lived in Germany they're now British citizens by descent, and can't pass on that citizenship to their own children. In order to overcome this, I believe your children must live in the UK for at least three years before they turn 18, then they would qualify.
The baby could use either a Kinderreisepass OR a Kinderausweis to travel to the UK, so go for whichever can be issued quicker and easier by your local Bürgeramt.
"The only thing that makes me wonder now is that we are not registered with the local Einwohnermeldeamt" Can't you just register temporarily wherever you're staying while you have the baby?
Oh, and a lot of airlines have strict rules about how late they will carry pregnant women, so you'd need to plan that carefully beforehand. Obviously they want to minimise the risk that you'll go into labour in the air.