Seconding the Matlock and Bakewell suggestions — the Derwent Valley between Matlock Bath and Cromford is brilliant with little ones that age. Cromford in particular is worth a look as a base: the canal towpath is flat and pushchair-friendly, Scarthin Books has a lovely little café upstairs, and High Peak Junction is a short walk along the canal where there are old railway wagons the three-year-old will love clambering on. Cromford Meadows has a shallow paddling bit in summer too.
For a crawling baby and an energetic toddler the single biggest thing I'd look for in a cottage is an enclosed garden — ideally with a gate you can latch rather than an open-plan lawn that backs onto a lane. It makes such a difference when you can just open the back door and let the three-year-old run around while you have a coffee. Worth also asking owners directly whether they have a stair gate or a travel cot, as not all listings mention it.
Chatsworth is obviously the headline day out but the farmyard and adventure playground there are genuinely excellent for that age — the under-fives section of the playground is fenced. Crich Tramway Village is another good one nearby (about 15 minutes from Cromford), and Gulliver's Kingdom in Matlock Bath is specifically designed for under-nines so your three-year-old would be right in the sweet spot.
At May half term a three-bed cottage in the Peak District does get booked up, so it is worth comparing across a few different sites rather than just one. I found holidaycottages.ai useful for that — it pulls together listings from several agencies so you can filter by number of bedrooms, dates, and things like enclosed gardens without having to check each site individually: holidaycottages.ai/cottages
Hope you find somewhere lovely — the Peak District is a gorgeous part of the world for a family break.