Eternal Data Preservation – is it possible?
I was making my monthly backup recently when it occurred to me that long term data storage does not yet really exist. Can you believe it? There is currently no well known technological means of digitally storing data for a few hundred years. I decided to dive in and investigate and here is what I discovered.
First of all, sorting out personal needs from corporate and governmental requirements I found out that the term ‘long term data storage’ currently refers to a period of 20 to 30 years in a ‘controlled environment’. What is a ‘controlled environment’? Well, this phrase refers to the absence of ultra violet rays (basically sunlight) and heat – both of which decrease the lifespan of current data storage media. It is hard to believe that an industry has come to the conclusion that 30 years is ‘long term’ but setting that aside, let’s address personal data storage alternatives first. There are CD-R discs and CD-RW discs available as well as DVD-R, DVD+R and DVD+RW options in most electronics stores today. If you want to store your data for a few years, do not rely on RW discs – they are suitable for temporary storage or data transfer purposes only. Technically speaking, the coating is volatile and will not last more than a few years.
Next, let’s explore the coatings used on CDs and DVDs. As I put these words onto digital paper, the most durable of the CD-R coatings is Phthalocyanine. When combined with gold reflective layers and stored in a cool, dry, dark environment, experts tell us that the data ‘should last’ up to 100 years on these discs. Reality however, is that no one has been able to confirm the manufacturer’s claims given than CD and DVD technology has only been available to the masses for the past 20 years (one fifth of the claimed lifespan). The other issue is that CD and DVD readers / writers are constantly changing and also improving by increasing data storage capacity and improving read / write tasks keeping pace with Moore’s law. Thus, the conclusion is that although you may be able (under ideal conditions) to maintain integrity of some discs for up to 100 years, the chances are good that there will not be a reader available for those discs 50 years from now. Storage is apparently one thing and data retrieval an entirely different beast.
On the corporate and government fronts, DVDs and CDs are not only too expensive to archive data but also too difficult to handle given their current capacity limitations. Tape systems have worked in these environments for decades but, tape is a medium designed for 5 to 10 years at the most before the data is no longer retrievable. The other issue is quantity or volume of data that needs to be stored for more than 20 years. Imagine the vast libraries of books that are being converted into digital format, add to this collection another library of artwork and photos that are only available digitally and quintuple that amount of data with a library of films, movies and documentaries and you quickly become overwhelmed with the task faced by people with the responsibility of preserving history for future generations to appreciate.
The facts are fairly straight forward, hard drives are mechanical and will fail in under 5 years with a probability better than 70%, top quality compact discs can last 15-20 years before the coatings, dyes or composite materials will begin to break down and disintegrate.
What can we use to preserve our digital assets for eternity before it’s too late?
One possibility, Microfiche. If this is an option, it would be a case of back to the future IMHO. The basics are that data as we know it today would be printed on the microfiche using a color laser. The Microfiche could be stored for hundreds of years in a fraction of space required by other media and, when needed, the Microfiche could be scanned back into a PC using the then current scanners which would likely be much better than today’s resolution thus delivering a technology independent method of long term data archiving.
I wonder what companies would be the first to order if such a machine were to exist?