Not at all. We've been to several venues and people have arrived late at everyone. We were at the Velodrome on Saturday. That stipulates you have to arrive 75 mins before the session start time because they let people in in batches to keep help maintain the inside temperature. Loads of people came in after the start time and the couple next to us arrived an hour late (and had to shift people from their seats!).
The stadium is about a 15 minute walk from Statford tube/DLR.
No problems with turning up and you'll only be a little bit late!
However, when you get to your seats, be mindful of what event is going on at the time. Was very annoying today when people stood up blocking our view of the start of the 100m - they could have waited 30 secs!