Go with your own instincts. Your hv is trying to help but if you don't want to do it, then don't.
OTOH, your ds was at nursery before so you have left him with other carers - not sure what the difference is between leaving him at nursery and leaving him at pre-school? Were you unhappy with him being at nursery too?
It isn't necessary but I did feel both were ready for it , certainly by 2 1/2 - 3. It is worth having the break just so you can get some one-to-one with your baby and do the chores without a toddler constantly in tow. That way the tiem you do spend wiht him is less about just getting through the day and mroe focussed on him. Perhaps with all the classes/groups you are trying a bit too hard and this may relieve the pressure on you. From his pov he can explore activities he may not do at home and become more independent(a double edged sword !). btw he doesn't have to go to school at 3 1/2 if he isn't ready , he could stay at preschool and go into Reception the next year.
our pre school takes them from two and half but ds wasnt potty trained so i had to go in and change nappies if he pooed! they seemed to enjoy it but were a bit over two and half. two sessions sounds about right