Absolutely do it, if you are sure it's what you want. Mid 30s is no age, you have probably 30 years or more of working life to go which is a long time to be in a job that isn't very rewarding or fulfilling your potential. I have friends who have respectively retrained as a doctor, a public health professional, a counsellor and a teacher in their late 20s or 30s and whilst it hasn't exactly been plain sailing all the way I think they are all glad they did it.
However, Law is a tricky area to get into in today's climate, you are looking at several years university study which is a big drain on your finances and time as a family, and then there is big competition for training contracts, many of which will demand long hours for not as great pay as you'd imagine, and then depending on what specialism you'd be hoping to get into, things can still be difficult, e.g. family and crime work can be rewarding/fascinating but poorly paid, corporate better paid but ludicrous hours and not always very interesting etc etc. With the rise in tribunal fees, cuts in legal aid and increased automation 'bread and butter' stuff for high street solicitors like conveyancing, employment, family, crime etc. is on the downturn.
Don't want to put you off your passion, just make sure you've done your research before taking the plunge, e.g. do you know any lawyers who might be willing to chat to you about their work and career over coffee, could you look at getting some work experience/admin work in a solicitors office to see it first hand? Or if you can pinpoint what it is about law (is it the academic, legal, detailed, forensic side of the law or more the helping people and getting an insight into their lives, or the public speaking/argumentation/persuasion of a barrister?) that really interests/excites you, you could look at whether any other careers which don't require quite the same input and risk in terms of money/time would press the same buttons for you? Lots of different ways to skin a cat, I'm sure there are many different routes/jobs you would also be brilliant at...