Hmm.... I don't think you are either mad or delusional, but as others have said you would need to go into this with eyes wide open to the sacrifices you (and your family) would be making.
The first part, the 5 years of medical school, would be expensive but utterly do-able - you'd have the motivation and maturity and would likely sail through. You'd get weekends, holidays and no nights.
Then you'd hit F1. You'd apply for this via a national application scheme, so could end up anywhere. And even if you are super-duper and score incredible well, the reality is you would still be applying to a deanery, which can be a massive area, rather than a specific hospital. Then, depending on the deanery, you could be sent to 3 hospitals miles and miles away from each other in your F1 year. And then maybe another 3 hospitals in your F2 year. And each time you moved you'd probably get your rota a week, or maybe 2, before you start. And should you try and get it earlier as you need to plan childcare, you may be accused of being 'unprofessional' and 'expecting special treatment because you have children'. Once you do have the rota, you may discover that you are working 1 weekend in 4, doing nights for a period every month, and routinely doing shifts that finish at 8 or 9pm with the odd 'compensatory' day off, meaning you'd need to have a robust childcare plan.
Once you are through F1 and F2 you'd want to apply for GP training. At this point life would likely get (slightly) better: GP training is 3 years long, and 1.5 years of that is likely to be in GPland, meaning more predictable (although still long) working patterns. However, 6 months of it may be in A&E, working 2 weekends in 3 and having approximately 3.2 days of normal shifts in the entire 6 months. Plus you need to fit revising for your postgraduate exams around working. And find the over £2000 these exams cost.
Then you'd be a GP. Which is tough, but in the area I'm in a salaried session (=half a day a week for a year) pays around £8000, and I know many GPs who are mums who are salaried and do 6 sessions a week = £48000 for 3 days work - but these days are often 8am til 8pm .
This has ended up being rather long but in summary, the med school bit (when your daughter would be 2 to 7) would be fine, but then the junior doctor bit (when your daughter would be 7 to 12) would be very challenging and not well renumerated, and in fact could end up costing you money once childcare is taken into account. Then GP-dom could be relatively well paid and rewarding and give a better work life balance.
Disclaimers: mum of 2 small people, current hospital doctor, only did GP for 4 months as an F2 many years ago.