It could make a difference in a private sale, but not so much on a part ex. Has the dealer already given you a price, is it all confirmed/signed? Was the part ex value on the condition of providing SH?
Personally I'd just hand the keys over, pick up the new one and drive off. The dealer is probably more concerned about the deal on your new car, rather than if the service book is in the glove box - they might be sending it straight to auction anyway.