I go past this on the train on the way into Paddington when working in London! All trains stop at Oxford, so they’ll only be going slowly anyway given the proximity of the station. It is a busy line, as it has trains going off in different directions - GWR, Cross Country and Chiltern Line.
Port Meadow is lovely for a wander, but as some PP have said, I’d check the flood risk given you’ve also got the canal on the other side (although canals will rarely flood due to lock and water management).
I’d say, despite the railway, it’s a fabulous location with open space close by, but still walkable into the city centre too. Looks like you have parking too which is a big plus. However, do look at the new Oxford congestion charge to see if that’s likely to affect you.