Speaking as someone married to a foreigner and living in their country (Brazil, as it happens), I think you need to think about what it means to marry this man - that is, what it means to YOU and your DD, not the Home Ofice.
If there was no immigration issue, would you still marry him? Just live together and see how it went for a few more years? Ask yourself what your level of commitment is, and what you honestly think his level of commitment is.
Because one VERY likely scenario here is that you end up living in Brazil, either for a few years, or permanently. Now, Brazil is great, and I reckon you'd love it here, and your daughter is a good age to pick up Portuguese. But is that what you're signing up for?
Now me, I'd let him go home to Brazil and get turned down by the Home Office when he applies for the spouse visa (because that is VERY likely). If he still wants you and DD to come over and marry him and live happily ever after somewhere warm and sunny with runaway economic growth, you go for it!!
And yes, obviously, you need to see an immigration lawyer!
Boa sorte.