I went "Cold Turkey" from BF to FF, and I wouldn't recommend it! Its very painful and uncomfortable,I was lucky I didn't have any blocked Ducts or Mastitis,and the emotional side of it was....Aaarrrgh! I cried buckets.Just because you switch to FF, it will not be "Miracle Cure". You have a very young baby who is going to have "growth spurts" and sniffles, which will interfere with their normal routine,and leave you feeling tired and frustrated,whichever way you feed.
I would personally persevere,from what you have said there are "better times" and worse times,so its not all bad. I would disregard what the H.V says,and find someone more skilled,or listen to the MN'tters,who will share their experiences and help you to come through this "bad patch".
In another 24 hours,things could feel very different.IMO you are nearly "over the worst". The discomfort should be subsiding,and LO will settle into a "routine" you can cope with.There are times when every baby hardly puts on any weight,and sometimes even loses,but this is normal too,some BF seem to be very "bonny", and so sometimes it evens things out a little if their weight gain isnt huge some weeks.
I'll let the experts take over,as my experience with 3 DC's have been very different. But all I can say is never go Cold Turkey,ease down gradually if you must,even try a bottle of FF at 11pm as I did,if they'll sleep a little longer,but continue if you can,you will be glad you did later I promise. HTH