I wouldn't point out that they take 5 weeks - will justs get their backs up. However, I would mention about your holiday.
Something along the lines of:
" I appreciate that you require a speedy reply on this matter, however, as 'Mr. ex-rickman' is aware, I will be going away on holiday shortly and will therefore be unable to respond within the requested two week period. Obviously, once I return, I will give the matter my urgent attention.
I trust that this is satisfactory, but please contact me if this is likely to cause any difficulties."