If you use a mail client (that connects via the IMAP protocol) then you will automatically download a mirror copy of each email to your computer for you to read, as well as leaving the copy that is stored with the hosting provider. You can do that on your desktop computer, laptop and (for example) phone, so the same email is stored three times.
Then periodically you can duplicate the emails to a local folder on your desktop computer(only) and delete them from the server.
If your hosting provider loses its data you still have the copies that were automatically made locally.
If you delete an email on one computer then it will automatically be deleted from the other copies of your mailbox (phone, laptop, whatever) next time you collect your mail with one of those other computers.
So you have an archive of your historic emails on your desktop, and three copies of each recent email to refer to whenever you need.