Please speak to Women's Aid as well. Your feelings are perfectly normal and expected. They have seen this all before a million times, and may be able to organise counselling for you or just reassure you. You're going to run the whole gamut of emotions, anger, fear, sadness - for you, for him, for the relationship that you saw in the beginning, happiness, confusion, wanting to go back, not wanting to go back.
In short, nothing is wrong to feel but please trust your lovely friends and family who are being supportive, and the police, and WA, and your head. Whatever he says or promises, (and, oh, he will promise you the world, including therapy, anger management, etc. Tell him to do it solidly for 2 years+ and then and only then can you make tentative plans. Most of them are full of air. They promise but they don't last two weeks.) it is never going to be a good idea to take him back. Mexican is right, your DD is so little that she will be unsettled for a short time (as she'll have picked up on the tension, change of routine etc) but with you as a constant presence she will be absolutely fine. It's much harder to leave with an older child, as they tend to pick up behaviour problems due to the upheaval and also the disordered behaviour they have been witness to prior to leaving.
You have done the right thing, you are strong, all you need to do now is take it one day at a time and be there for DD, even when you don't feel strong. Lean on your family, they love you and want to help.