Definitely not too old or unfit for yoga.
I've got a chronic health problem which has always been exacerbated by any kind of sport or exercise. I went to my first yoga class not able to forward fold and get fingertips to the floor but with regular practice and classes I've seen a major improvement.
For the first few weeks I found myself looking around thinking I'll never be able to do that pose, and it took me ages to switch that competitive side of my brain off and just focus on me and how I felt that day.
My first few months of vinyasa yoga was super hard and I struggled to keep up but just did what I could when I could.
Honestly, letting go of worry about what I couldn't do made such a difference, it made yoga much more enjoyable and I soon found I'd surprise myself with what I could suddenly do.
Maybe vinyasa isn't the right fit at the moment, so it's worth trying other styles and other teachers to see what fits. Ultimately my advice is to not try to compete with others or yourself and to give it a good 3 months of regular practice and you will see even subtle differences as your strength and flexibility improves.
Also, yoga by Adrienne is really good and quite gentle to practise at home; and take time to practise simple poses like child's pose and downward dog at home, focusing on your breathing regularly and you will find your body changes, softens and becomes stronger quite quickly.