I originally joined a gym to strengthen my back on the advice of physio following bad back spasm which landed me in hospital.
By the time I fell pregnant the gym was a way of life as advised by op, bur had also become more than that to me.
I have slowed down now, but try to go two or three times a week to do spin and body balance. However, the change in weather combined with the ever expanding bump (25+6) I twinged my left side of my back on Sunday night.
It has meant taking a couple of days off work and employing the use of my mat pillow, but it has been completly manageable with just a few paracetemol.
I believe this is down to keeping excercise going and I plan to keep doing something even after I get too big to heave up on to spin bikes :)
I also went to a physio drop in session for pregnant ladies and was actually shown how to do pelvic floor excercises which I do daily and I also believe they have helped maintain and build my core strength.
Mainly, listen to your body and rest when it tells you to
I don't know if this is any help to you, but wanted to share my experience as I share your initial fear.