What country are you in? For now I will assume you are in England. Your country can make a difference to taxes and to any funding eligibility.
Would I be paying their taxes?
No. You agree a gross wage just the same as your employer has agreed a gross salary with you. From that gross wage you deduct your nanny's Income Tax, National Insurance and Pension contribution and anything else like Student Loan.
The costs you have as an employer are:
Payroll Admin, Employers National Insurance, Employers pension contribution.
You will also have other non-payroll costs: activities, mileage (if nanny uses their car to transport your child) or public transport.
How much would they charge?
It varies depending on where you are. As a nanny in Surrey I am paid between £11.50 and £14 gross an hour.
Are they entitled to holidays, etc.?
Yes, it is the same as any other employee. GovUk, ACAS and Citizens advice has information about workers rights and the nanny payroll companies provide a lot of information on their websites.
A nanny working 2 days per week, assuming the days are the same length, would be entitled to a minimum of 11.2 days annual holiday. This is known as Statutory Holiday Entitlement. You can offer more holiday but not less.
Should I go through an agency?
You should certainly use a nanny payroll company, they will make it easier for you in terms of the payroll setup and management, plus can provide help with creating a contract of employment.
Nanny agencies for finding someone are a bit hit and miss. As a nanny 10 years ago I found a job via an agency. Last year agencies did not produce any jobs for me, I found two jobs via www.Childcare.co.uk
My biggest criteria is finding someone long-term, who DS can form an attachment with and who'll feel like a member of the family.
It is hard to say how long someone will stay in a job. I was in a job for over 9 years. In the past I was in for 3 years. Currently I have been in one of my part-time jobs for a year and the other for 10 months and I have no intention of leaving either of them.