I don't think it's a terrible idea and could have a lot of positives compared to the alternative of buying in care from a care agency/provider (I would recommend this over just employing a single private carer for various reasons, happy to expand if you like).
Some agency/private carers are absolutely fantastic but my experience with various relatives has been that even with a great agency, the quality of care can be variable, unless you are prepared to shell out £££ for a very high end agency, which for ordinary people is beyond their reach. Problems can be: you don't choose your carer and you seldom consistently get the same carer every time, due to staff turnover and rota needs etc so the person doesn't get to build up good relationships with them, you don't get to dictate their hours (so e.g. if the person gets a morning call to get them up and an evening call to put them to bed, the morning call can be anywhere between 7am-10am and evening call between 8pm and 10pm, regardless of when the person actually wants to get up and go to bed). It can be unreliable with the carer not turning up when expected (or at all in some cases!).
So a committed reliable family carer will avoid a lot of these problems. However to make it work it will depend a bit on the personalities of those involved and needs some careful working through and understanding/matching of expectations on all sides before you enter into it - both you, your MIL and the rest of MIL's family who may also have opinions/wishes about her care.
My brother was a carer for my disabled mum for a short time whilst in between jobs and it worked out OK - he got Carers allowance which isn't much really and my mum paid him a small amount on top and she provided full board and food in exchange for an hour or 2 a day of cooking, cleaning, errands, taking her to appointments etc. It did slightly blur the lines of what he was being paid for, what he would always have done as a good son anyway and what he would have had to do for himself if he was living alone, plus I sometimes wasn't clear how much I ought to be helping (was doing a lot before he moved in and obviously this then lessened) but we all trust one another as a family and no-one felt put upon, and anyone unhappy would have been able to voice it, even so I'm not sure it would have worked as a long-term arrangement but there was lots else going on there anyway.
Just a few things to think about, what kind of care does your MIL need, is it mainly household help/chores/errands etc, or is it personal care too? How does your MIL feel about you providing care, I find with a lot of my elderly relatives they may be adamant about wanting family care and recoil from having 'strangers' involved, but when/if it comes as far as them needing really intimate care like help showering and toileting then actually for some of them it can start to feel quite awkward/demeaning to have their children helping them, and they find it easier to detach from someone who is being paid to provide a service - also a big part of a skilled and experienced carer is knowing how to maintain a person's dignity and minimise the awkwardness for the person receiving care - not saying you couldn't do this for your MIL of course but sometimes the fact of it being a 'professional' just makes it easier, of course there is also the impact on you to consider and how it will change your relationship with your MIL, don't underestimate how much the dynamic can change there...
Also I would sound out how the rest of the family will react to this - you don't want them feeling like you are taking 'advantage' of MIL in any way or (conversely) that because you are being paid that makes you her lackey or go-to person 24/7 for all her needs. You'd want (as far as possible) there to be consensus from both MIL and family about how many hours you will do, what you will be paid and what you can realistically do in those hours. Again with my relatives I have found once they have (reluctantly!) started paying for a service (and what is a pittance to you e.g. £10 an hour can seem like a fortune to them), they have huge expectations that the carer can do everything round the house, a week's shopping, laundry, cooking and take them for all their appointments in something like 2 hours a week! Also, what help will your DH and any siblings continue to provide and who will 'cover' if you are unwell or on holiday (this is one thing you wouldn't have to worry about if you got agency carers in).
Good luck, it works out for you...