I've been doing freelance work for 14 years now and it's almost 8 years since DD was born. In my experience, it is hard going, you will be jiggered a fair bit of the time, you have to be prepared for long days and little or no downtime, but the upside, which continues once DC are in school, is that you have much more flexibility in planning how you use your time, you can be there for school pick-ups, school sports etc and make the most of the school holidays without needing some else's permission, and, in my case at least, I can make a good hourly rate doing the type of interesting and satisfying work that I wouldn't necessarily be able to do if I was working for someone else.
I do think that you really need a mixture of reliable paid childcare and some sort of support network - DH/DP and/or family etc - to make things work. You also have to be very realistic and disciplined about the types of work you take on - for example, if you can't do out-of-hours meetings because you don't have childcare then, don't take on projects which require those. You need to be sure that you can to do your work to a high standard and on time because clients won't make allowances for childcare problems on an ongoing basis and you are setting yourself up for stress and hassle. And the point above about being able to concentrate and having a separation between children and work is absolutely spot-on!
I understand that your DP is under the pressure of being freelance too but, especially if extra paid childcare isn't an option, he should be doing his fair share of looking after your DC. I would suggest getting into some sort of routine agreed by both of you where you have paid childcare at certain times and other set times when you know that your DP will be looking after the baby. This will let you plan your workload and also plan in meetings etc, and things should instantly become so much more manageable. This will also let your DP know exactly when he is and isn't available to work, and let him plan his time.
Because my DH is working away so much at the moment and I am therefore very reliant on paid childcare at set times, I'm in a similar situation to you in terms of wondering whether I can keep going on a freelance basis, from a financial and practical point of view. But I know that, if I had DH at home and helping, things would work fine as this was the case in the past, and I am pretty sure that going back into the jobs market would be a retrograde step both financially and from the point of view of flexibility re spending time with the children. Time for a serious discussion with your DP, I think, and then plan the next steps after that.