I had a similar experience OP- I wasn't at fault, fairly minor damage (bumper, reversing sensor broken and reverse light). The other driver was very apologetic and said he'd rather pay himself then claim on his insurance.
I wasn't sure what to do at the time so I asked for his details anyway which he did give to me.
I called my insurance and asked for their advice and they said that it was ok for the other driver to pay. They did make a note of the details though and said that I could go back to them if necessary.
Other driver asked me to get 3 quotes for the repairs- fair enough but this took me some time and effort especially as most busy garages don't seem to want to quote for minor repairs.
He was surprised by the costs of the quotes and did try to suggest that I was trying to get a full resoray on a van which had less than perfect exterior paint before the accident. I wasn't claiming for any paintwork so that was easily clarified but I guess that kind of thing can be tricky whereas an assessor will decide what needs doing if you use insurance . He did eventually agree to the cheapest quote after a final quibble about needing a working reverse sensor- I drive a van with a solid bulkhead so have no rear view mirror and the sensor is very much needed!
I also needed to use a garage that would provide a courtesy van, and which would not take too long over the repairs as I rely on having a van to be able to work safely.
I had to persuade the other driver that I needed a courtesy van.
I guess what I'm saying is that going via your / their insurance can make sure that things are done in a more convenient way etc.. It also removes any quibbles about costs, extent of repairs, and the worry about hand over of payment (I asked other driver to pay garage in advance).
It might be best to correspond with the other driver by email if possible so that you have an audit trail.