This is just my own personal experience of course!
I went from DD1 not bfing at all on day 2 (or even bottle feeding - she had to be force fed
) due to jaundice, I expressed 12 times a day with a hospital grade pump for 8 weeks which provided most but not all her milk so she had formula as well and it was far more formula than ebm in the early weeks.
She did latch on on occasions but wasn't very good at it so she had a lot of bottles. At 8 weeks she caught on and we gave up bottles entirely - this was helped by me going on fenugreek and actually having enough milk for the first time.
She was ebf then to 26wk when she started solids and carried on bfing until she was 3y 6m (she self weaned due to jaw development) - I was 8m pg with DD3 so she had bf through DD2's pg and tandem nursed for about 21m with her sister.
A rough start doesn't have to mean the end of bfing but equally there are no guarantees in life.
There are a few things that you might want to try to get your DS back to bfing (you may already be doing them/have tried them though!):-
Skin to skin when possible - as you have a toddler that may mean wearing DS in a sling (wrap or ring kind) while you are topless (wearing a cardigan or loose top over both of you if you feel more comfortable that way) so that he gets used to breasts as being a lovely comforting place to be - it also makes it easier for his cues to be responded to immediately.
Co-Bathing - one to try after the toddler is in bed and your OH is on hand to help - this can help trigger feeding reflexes in babies and the angle you lie in a bath is ideal for Biological Nurturing.
Nipple shields - not the best solution in the world BUT if they mean your baby latches on to you for longer and is happy to feed then they are great (shields can be weaned from later if needed/wanted).
Trying paced bottle feeding so that there isn't so much instant gratification with the bottle - and perhaps trying to finish a feed on the breast (feeding a bit from the bottle so baby isn't ravenous then offering the breast to end the feed). This can work well as then baby begins to associate the breast with satiety and repletion rather than the bottle.
I hope you find a way forward that works for you, what ever it entails 