To worry about history being lost because everythingis digital

[Trainbother]

So much of what we know about the past, especially about ordinary people, is because someone came across old letters or diaries. How does that happen when everything is on old or obsolete devices that no-one's looking at?

I recently came across a carrier bag of photos from late DH's military service in the 1990s. Very interesting, but they'd have been lost forever if they were on a phone or PC I didn't know about or couldn't access.

How will future generations learn the history of ordinary people? I know there'll be official records of the big stuff, but what about the rest?

That's my point though, no-one knows if it's important, if no one knows it's there.

The article I linked to was about a national BBC project to recreate the Domesday Book with school children in the 1980s

They had a lot of data - and they decided to do it so it could only be played on a Domesday Machine. And there were only 1000 made.

Domesday Project reborn online after 25 years - BBC News

It's been transferred to the internet

I've worried about this for years!

Sometimes I worry about the opposite. That we artificially store past information in real life or digitally beyond the scope of human memory. For thousands of years humans couldn't do this, they kept stories alive through speech and legend. I worry we weigh ourselves down with it and want to live in a museum as a country. Having said that I enjoy history and visiting sights etc as much as anyone. Not sure what conclusion I am coming to.

Isn't this why things have evolved to use "common" file protocols, i.e. PDF and JPEG etc., that are literally readable on any device and using readily available/free software available under most operating systems??

It's surely just a matter of everyone making sure they save the most important stuff as pdfs or jpegs (and other readily transferrable/readable protocols)??

I think users need to be better educated and more aware of the potential risks of data/document loss when all they use is proprietory software/systems that may not stand the test of time or where the software firm may make it unavailable either after a certain amount of time or the subscription is stopped being paid for.

I always tell my clients to ensure that they take pdfs and save things like online invoices, online bank statements, because after a couple of years they often become no longer available to download. That doesn't have to be paper copies, I tell clients to save them to their own computer, pen drives, online backups, - the options for data saving are endless.

Yes, but Joe Public won't have this training and will just let their private family archive fall by the wayside.

Although to be fair, many working class families in the 19th and 20th centuries will have had the same experience albeit for different reasons: lack of money, lack of secure housing. I have an unusually large number of heirlooms from one branch of middle class Victorian ancestors which I really value - but very little from the other branches. Most families won't have much at all.

This is an interesting discussion thank you. Just to say that there are still letter writers and journal keepers out there ( diarists??) Obviously not to the extent that it used to be but people do still use pen and paper. I bought my mum one of those Grandmother books in the hopes that she would write down some of the family history but she didn’t do it. Too much work apparently which is a shame. Hopefully there will be families who do write them or print out their photos so I dare say we will maintain some continuity for future generations

Storage systems often rely on a weird mix of insanely precise physical hardware and custom software. If someone handed you an IBM 1316 disk pack from the 1960s then first you'd have to find the specs of the drive to put it in. Which would be tricky. Then you'd have to build the drive. There'll be very few off-the-shelf parts you could use for that. Nearly everything would have to be custom made.
You then get a string of ones and zeros off that drive. Now the research project starts to decode those ones and zeros into useful data. You might get lucky and it's in an easily-readable format. You might get unlucky and find out it's a database file from an application that no longer exists and without which the data makes no sense whatsoever. If you read in the data and it's a series of entries like "MOON, HOWARD, 32, 12633, GH, 0" then you can guess the first two fields are someone's name but the rest? Who knows?

That's interesting. What is considered best practice for the very-long-term storage of digital content? An organisation I used to work at had a regulatory requirement to keep certain records for several decades. They ended up opting to print everything out on paper or microfiche as redundancy for the digital content as nobody could confidently guarantee that the hardware and software required to read the media and file types would still be available in 30 or 40 years time.

I think things are a lot more standardised these days. More and more software makes it easy to take PDFs of documents, some automatically save them as PDFs. Lots of different bespoke software packages use proprietary database software behind the scenes. That means that even without the "front end" being available in years to come, the data behind it could probably be read by using the proprietary software. But, I think we're talking more about "documents" here when taking about comparing agianst physical photos and diaries, so we're not really looking at long lists of 0s and 1s, we're looking at the format in which documents/photos have been saved, which is usually pdf and jpeg.

When you go back 30-40-50 years, a lot of software developers would write their own code from the ground up, even their own operating systems, but over the decades, there's been a lot of standardisation. Most systems these days (particularly domestic and small business ones) seem to provide for data import/export so there must be common file types baked in, even if it's just as a simple csv for data.

For example, I had hundreds/thousands of holiday photos uploaded to the truprint app. I never felt secure about them being stored by a third party in the cloud and was relieved to see that it was relatively easy to download them as JPEGS in zip files, which I did and have now unzipped them and downloaded them onto a portable hard drive and uploaded to the cloud in the form of dropbox, so now have them in three different places (in fact, four because I have them on my laptop too!). My family know my password, so hopefully if anyone is bothered, they could find them after my demise. Obviously, if they don't, then future grandchildren etc won't find them either. But that's the same as if my next of kin just chucked all the photo albums in the skip when clearing our house when we've gone!

I think the answer has to be for people to be more aware of where they store their documents, how they're backed up, making sure their family know where they're stored with passwords, etc. A kind of "will" for the world of modern data.

I remember reading something very worrying about how much data/info we have lost already from material recorded on systems that are obsolete - I think awareness of this has improved but it's still a concern.

I've been concerned about this for years. For that reason I do keep a physical written diary and we print off a selection of photos after holidays/events.
In the past we transferred videotapes onto CD as the technology became obsolete but as technology moves on the same thing will happen again.

Perhaps we are among the last generation, those who have experienced the world pre Internet, who will have these concerns and storing physical copies of personal data will just end with us.

It's also not just about having things in a digital archive because even properly archived stuff needs maintenance and management. I work the area of developing data archives (for scientific data). And I think there is an assumption that if something is digitised, then it's there forever unless someone deletes the file. But there is an enormous amount of work that needs to go into updating file formats, cleaning data, updating the infrastructure that supports the data storage, transferring data into new file formats/data storage locations when things become obsolete, etc. It also uses a huge amount of energy, meaning it makes pollution, waste, etc.

That said, I don't know that we're more or less disadvantaged because of it. So much more of our data is publicly available than ever before. For example, I don't have a single letter, diary, artefact from my grandparents. I know they existed. I remember as a child sitting in their attic reading their love letters to each other written during the war. As far as I can tell, those no longer exist. I think my mum either binned them or some distant family member took them. If they'd had Facebook back them, there might still be some record of it somewhere, possibly.

That being said, my dc have boxes of stuff I've saved for them and continue to save.

Basically (and I'm not a huge expert on it, give me paper and parchment any day!), you keep 3 copies, at least 1 in a different location. You fixity check them every so often (to make sure they are readable/uncorrupted), and replace with another copy if so. You keep an eye on file types and software, to make sure they are usable and you migrate to other formats if required. In a way it's similar to how we look after old records - assess risks and mitigate them. Printing out can be good in some cases - but with some files (video, databases), it's either not possible or you lose functionality. I think one of the biggest problems with digital is the sheer quantity. We don't need to keep everything and actually getting rid of what we don't want to keep makes it easier to save and access the rest.

I definitely print off photos and we have them around the house and in albums. I don't even have a clear photo of one of my great grandparents, and I'd say 200 years ago the majority of my ancestor's couldn't write, so lack of records about ordinary people isn't a new thing, it's just different now.

There is a lot of work going on with community archives, to make sure ordinary people are represented. So much that has survived in the past is by accident (apart from official records), so I'd say it's an improvement.

I sometimes think you will have sort of "digital archaeologists" in a couple of hundred years, going through old servers and storage and finding all sorts of stuff that hasn't been "archived" properly!