I swam in the first trimester, but decided to stop as it was leaving me too shattered afterwards. I am a good swimmer, so was doing about 1200m without really thinking about it, and that was on top of the 30 minute walk to the pool and home again. In hindsight, it was a bit much.
Now I workout for maybe 30-40 minutes, 3-5 times a week at home (so I can go to the loo whenever I need to, etc). Normally I do the same workout videos as before, but modify them - so I step instead of jumping, do weights when they do abs, etc.
At the beginning I found all the pregnancy workout videos really frustrating, as they never got my heart rate up at all. I think I've found a good balance now between making pregnancy stuff harder, or non pregnancy stuff easier.
I'm 32 weeks now and will just keep doing as I have been. We do quite a bit of walking as well, but I don't count that as exercise as I can't walk very fast any more.
I don't really consider pregnancy yoga to be exercise though - as a pp said, it's not very cardiovascular - more stretchy stretchy breathy breathy.