Hello, just noticed this thread.
Applesandcider is right, the Olympic Museum can be covered in 1/2 day. We went there with the DC and spent a couple of hours, but a lot of that was wandering around the gardens. You could spend more time there if you were going to look at things in more detail.
The restaurant in the museum is lovely. We only had coffee and cake there, but the menu looked great. It overlooks the lake, with the mountains beyond.
Lausanne is built on a big hill. The city centre is at the top of the hill, the lake where the museum is quite a distance away. It is all very well connected though.
I think you would probably struggle to spend the whole weekend in Lausanne, depending on your other interests.
Geneva is definitely worth a visit. I would base myself there and take the train (or perhaps even the boat) to Lausanne. That will depend on when you travel, as the boat to Lausanne only runs in Summer, I think. Or you take the train there, and boat back.
There is a lot to do in Geneva. Museums, galleries, bit of window shopping. There are often concerts or fêtes going on. Or farmers markets.
What time of year were you planning to go?