We went to Sri Lanka this summer (2 adults) but would have worked very well with children as well. Direct flights to Sri Lanka with Sri Lankan.
We got a private driver which is normal in Sri Lanka - we got it through Sri Lanka Car and Driver website. That was US$650 and included mineral water bottles each day for both of us and he also bought us fresh fruit like coconuts and mango cut in front of us and gave us some Sri Lankan sweets. We paid in LKR and also gave a tip. He was the most careful driver which you definitely need with elephants and tuk tuks on the road. He stayed in accommodation he got and you just buy a lunch for him if you are out for the day. Only pay once you arrive so risk free.
Its monsoon season down south then so we just stopped quickly one night at Kahanda Kanda weather was actually perfect, over 12s only. £155 for B&B had place to ourselves and swimming pool / tropical gardens to ourselves.
https://www.kkcollection.com/kahanda-kanda/ Visited tea plantation a few pounds each and included cake, seeing tea, lots of fruit and veg. At hotel went to tropical gardens, swam in pool, most amazing food and drinks and wildlife on grounds.
Then Yala for 2 nights at Mailagama Cinnamon Residence which is by block 5 of Yala which is quieter. This place was amazing and run by some of the kindest people in the world. Only about £30 a night including breakfast plus about £5 a meal each for 2 courses. They took us round their rice fields, all their fruit and veg and their cinnamon on day arrived, then lovely Sri Lankan evening meal, gave us free drinks on arrival. Then arranged for a tuk tuk to pick us up next morning and take us to a bird sanctuary which was free and then afternoon we had a Yala safari - about £50 for jeep they arranged and we saw lots of wildlife including leopard.
Then out to another national park Kunama and 2 more safaris with a BBC photographer and conservationist - there was an elephant on the road enroute. Stayed at Wild Lodge Panama - think it was about £50 a night including breakfast. Safaris were per vehicle and it was a bit extra to have conservationist and that was about £320 for the two I think. Excellent meals. Quite rural out there and lots of pilgrims about, saw a temple and school children with umbrellas in uniform.
On to Ella, stayed at Tantra Ella, this only has 2 rooms, about £30 a room per night, the most amazing breakfast for £5 ish each including fresh fruit, pancakes, toast, eggs etc and fresh fruit juice. Giant squirrel in hisbiscuis tree with its babies there. Train from Demodara about £10 for 2, visited tea plantations maybe £10. Ate there pizza about £4 each for giant pizzas and drinks.
On to by hotel with pool by Lions Rock, climbed Lions Rock maybe £25 each or so and you start very early, monkeys there. Dambulla caves. Hotel was about £90 a night there.
Then stayed in a villa for final place with a swimming pool and tropical gardens just for us, was called Dark Green Resort, about £60 a night for giant villa and big swimming pool just for us. Had breakfast and candlelit dinner there, extra but very cheap, about £8 each for 2 courses and superb.