Think you maybe do need to spell it out to her, give her some guidance on what you are expecting her to do with the kids after school. Homework, playing, helping get dinner on etc.
For comparison, when i was an a-level student i had a half term job looking after two kids, a girl, 12 and a boy about 9 i think, full time 9-5 for the week. I discussed with my mum whether to do it as I had m own school work i needed to get done that week. My mum encouraged me to do it, her take was kids that age would / should be able to play and entertain themselves a bit and I'd be able to put on a film on for them a couple of afternoons and get my reading done so i was prepped to write my essay.
But it just didn't pan out like that. the girl in particular wanted my attention every hour of every day and didn't appear to have any homework of her own (although the idea was we'd look at that together). Didn't want to watch films, read a book etc etc. encouraged by my mum one afternoon when her brother had wandered off to amuse himself i pushed this a bit, said we'd have a bit of quiet time / reading. I remember her not being able to do this and me eventually telling her i had reading i needed to do.
I wasn't asked back!
I can see from the Mum's point of view, now, that she probably thought it was totally off me needing / expecting to get a bit of my own stuff done during that time. On the other hand, part of her expectation was I'd help, the older girl with studying and study habits. The idea of doing some school work together seemed like a good one beforehand.
And crucially, I was following the advice of my mum who encouraged me to expect / demand the 12 year old entertained herself occasionally.
Anyway, the point of all this is, maybe talk to her first. Different families and different cultures have very different takes on what constitutes 'looking after children' from being present in the house to providing a non-stop entertainment service. And she's young, probably never done this before. Maybe babysitting at home did involve being in the house letting the kids watch tv while she emailed her friend etc, and that was acceptable.