What a wonderful subject to explore! I studied art history years ago and have found it hugely enjoyable and enriching.
Ernst Gombrich’s book “The Story of Art” is a good starting point, a refreshingly opinionated and enthusiastic overview from cave paintings to the 20th Century (it was originally published about 50 years ago I think and Gombrich died in 2001 so you won’t find the latest contemporary art in it, but there’s a reason it’s an absolute classic, it’s a great read).
If you’re interested in the Dutch masters check out the Rijksmuseum’s excellent website, it includes many resources, all available in English afaik. They also have a YouTube channel.
And yes, I agree seeing the actual works in galleries is best if you can get there - ideally with a guided tour or an audio guide to provide some context. But also just looking...
If you’re in London (or can get there easily), Kenwood House In Hampstead Heath has one of my favourite works by Rembrandt (one of his self portraits), and also a beautiful Vermeer. All free to see.
London is blessed with so many fantastic art galleries, lots of them free if you go to see the permanent collection rather than a special exhibition.
I’m less familiar with the galleries elsewhere but it’s worth checking out what is available locally to you.
The art.uk website is a great starting point for finding out about museums and galleries all over the UK. They also have interesting stories where they explore a certain subject and link to relevant works of art from across different collections.
Enjoy!