EEVblog Electronics Community Forum
General => General Technical Chat => Topic started by: Halcyon on May 12, 2017, 07:49:44 am
-
I'm basically just mucking around here, but also being half-serious.
I have a some data files which are fairly small (<1 megabyte) but contain a raft of critical data such as an encrypted container full of passwords, copies of birth certificates and other vital documents. I currently back these up to various different locations/media (including physically printing the data out (in encrypted form) using a program called PAPERBACK (http://ollydbg.de/Paperbak/).
But as we know, optical media can degrade and be easily damaged, normal NAND flash (like what's in USB sticks or SD Cards) isn't overly reliable in the long-term and hard disks can die unexpectedly.
I was thinking of backing vital data up to EEPROM (like the 1MB Microchip 24AA1026-I/P (http://au.element14.com/microchip/24aa1026-i-p/serial-eeprom-1mbit-400khz-dip/dp/1971892)) which can then be stored off-site and is small and durable.
I'm just thinking, if the worst was to happen and house burnt down, all I'd need is a ROM reader to recover vital data. Thoughts?
-
standard storage media + regular refresh of the media is the way to go. copy to NEW media in regular intervals (few years) and shred the old media.
-
Maxim iButton? If the capacity is enough.
Although, nowadays I would just place the info into into a password protected file and send myself in an email.
-
Yes ... I'm not a fan of any solution where a sense of "set and forget" can creep in.
Regular refreshing has a number of benefits - from ensuring the archives are readable to resetting the aging clock.
What the best solution is - I don't know.
-
There are DVDs and Blurays that are rated to last 1000 years:
http://www.mdisc.com/corporate/ (http://www.mdisc.com/corporate/)
They're not even that expensive (3$ per DVD and 5$ per Bluray)
Greetings
-
There are DVDs and Blurays that are rated to last 1000 years:
Nothing guarantees, that you could buy a Bluray reader in 10 years. Imagine buying a VHS, Laserdisk, HD-DVD or a Zip drive today. Also, you cannot just make a EEPROM USB drive, because maybe USB will not be around, and the Flash in the controller chip might die, and the programming toolchain might be gone.
-
There are DVDs and Blurays that are rated to last 1000 years:
Yeah... no. We use good quality DVD and Bluray discs at work for archival purposes and even now we are worried about longevity. We face media which is 10 years old and worry about it. The first thing we do before anything else is image it because we just don't know how reliable it is.
Nothing guarantees, that you could buy a Bluray reader in 10 years. Imagine buying a VHS, Laserdisk, HD-DVD or a Zip drive today. Also, you cannot just make a EEPROM USB drive, because maybe USB will not be around, and the Flash in the controller chip might die, and the programming toolchain might be gone.
Correct! I have a hard disk sitting on my desk right now where the data is just totally lost. The USB controller on-board has failed and there is nothing I can do (with the tools I have at home) to recover anything.
Yes ... I'm not a fan of any solution where a sense of "set and forget" can creep in.
Regular refreshing has a number of benefits - from ensuring the archives are readable to resetting the aging clock.
Sure, but when stored off-site, regular refreshing is a pain in the ass. For something like a birth certificate or copy of a driving licence, that image won't change in 10 years (or ever). For things that are updated more often (like a password DB), that sort of system would work I suppose.
Although, nowadays I would just place the info into into a password protected file and send myself in an email.
That doesn't work for me. I don't actually "know" my e-mail password. It's a very long string on purpose. Without my password DB, I couldn't get into it.
-
If most of the size is photos, and you only have a few pages of text i might say go the analog approach, get them printed to archive grade negatives or microfilm, then store them in a small sealable container,
In 50 years when bluray is only something grandpa talks about, you can still get out a magnifying glass.
-
EEPROM, FLASH, etc. are not any more viable than magnetic media, and maybe worse. They will lose the cell charge over time and lose the contents.
OTP ROM may be a possibility. I am talking true OTP, not just UV erasable EPROM packaged in a plastic non-windowed package which is what most "OTP" ROMs are. True OTP would use a series of fusable links that are blown out or kept during programming. This seems like an immutable, physical change type of programming that can't occur or revert spontaneously. It should last. You may need to build something to read out the contents of your ROM chip in 25+ years, but that will always be possible.
I have a pile of TDK archival DVD+R discs for backup of small data. They are rated 100 years, and they are coated with the same ultra-hard coating that blu-ray discs use to prevent scratches. Obviously it is impossible to truly test the data retention, but so far so good. I don't need them to last forever, but longer than the 'cheapest I can find' CD-Rs I bought back in the early days of optical disc burning, would be nice.
Maybe your best policy is to not depend upon longevity of your chosen medium media, but to refresh the media (whatever they are) every N years, possibly updating the data while you are at it.
-
Maxim iButton? If the capacity is enough.
Although, nowadays I would just place the info into into a password protected file and send myself in an email.
What happens when your hdd dies and you loose all your e-mail?
-
Maxim iButton? If the capacity is enough.
Although, nowadays I would just place the info into into a password protected file and send myself in an email.
What happens when your hdd dies and you loose all your e-mail?
You mean when google looses all my emails? I just send a request to NSA, surely they have a copy of it.
-
Use base64/OCR friendly font or downright binary / QR schemes and A LOT of redundancy (error correction) to print the data on archival paper then store the paper. You can OCR and use the error recovery information if needed to recover the information.
You could also look at MRAM chips, apparently they have a data retention of 20 years at 125c (for some versions) and they're very easy to work with:
https://www.digikey.com/product-detail/en/everspin-technologies-inc/MR25H40CDF/819-1040-ND/3471117 (https://www.digikey.com/product-detail/en/everspin-technologies-inc/MR25H40CDF/819-1040-ND/3471117)
You could store a few of these at random locations and maybe once a year read back the data, check them for errors, write the chip again (if needed to "refresh" the memory cells) and put it back in the archive.
-
standard storage media + regular refresh of the media is the way to go. copy to NEW media in regular intervals (few years) and shred the old media.
+1
To this I'd add that you add error correcting using something like MultiPar.
-
If you truly trust your encryption, you can store them in multiple public-ish places (various cloud drives (google, microsoft, take your pick)), attach them as forum posts, email them, facebook, paint the Internet with them. If the Internet goes away permanently, you have bigger problems.
The most durable non-electronic medium would probably be granite+PaperBack+engraver.
The most durable electronic medium would be as was said antifuse PROM memories, or mask programmed memories.
Or, if you are looking for obscure stuff: https://www.theverge.com/2016/2/16/11018018/5d-data-storage-glass (https://www.theverge.com/2016/2/16/11018018/5d-data-storage-glass)
:)
-
Yes ... I'm not a fan of any solution where a sense of "set and forget" can creep in.
That was my first thought as well. My current strategy: If you want to keep data, then make it part of your active data set which gets backed up and transferred to new media during an upgrade.
-
For small volumes of data, paper tape was always a very durable medium. It had the advantage of being human repairable and readable (in small chunks anyway). Maybe we need an updated version with higher data density due to modern technology (smaller optical sensors, laser punching?). Maybe a more durable tape material too, although tight wound paper is extremely resistant to most threats (including water and fire), plastics might actually suffer more long term degradation.
There's a big attraction to a media format that is just so simple that it simply cannot go obsolete - if you discovered it and wanted to read it in the far distant future, then it would be a simple matter to constuct a suitable reader from scratch. It's the sort of media you might want to put in a deep space probe!
-
OTP ROM may be a possibility. I am talking true OTP, not just UV erasable EPROM packaged in a plastic non-windowed package which is what most "OTP" ROMs are. True OTP would use a series of fusable links that are blown out or kept during programming. This seems like an immutable, physical change type of programming that can't occur or revert spontaneously. It should last. You may need to build something to read out the contents of your ROM chip in 25+ years, but that will always be possible.
I like this option. It's totally weird, and sounds fun!
Looks like the good kind, with the titanium-tungsten fuses, is rare but available. Here is 63S141: http://www.ebay.com/itm/222365236136 (http://www.ebay.com/itm/222365236136)
edited to add: alright, so only 8 kB is available, hahaha. Text files only!
-
OTP ROM may be a possibility. I am talking true OTP, not just UV erasable EPROM packaged in a plastic non-windowed package which is what most "OTP" ROMs are. True OTP would use a series of fusable links that are blown out or kept during programming. This seems like an immutable, physical change type of programming that can't occur or revert spontaneously. It should last. You may need to build something to read out the contents of your ROM chip in 25+ years, but that will always be possible.
I like this option. It's totally weird, and sounds fun!
Looks like the good kind, with the titanium-tungsten fuses, is rare but available. Here is 63S141: http://www.ebay.com/itm/222365236136 (http://www.ebay.com/itm/222365236136)
edited to add: alright, so only 8 kB is available, hahaha. Text files only!
8 kB ? What are you looking at. The one you linked is 256x4, that is 4 bits wide, 256 locations. In other words... 128 bytes. Enough for one password maybe. :-DD
-
Oops, I meant kilobit, because there are 8 available!
Here, this is much better: only 1.5c / byte! http://www.ebay.com/itm/322224601177 (http://www.ebay.com/itm/322224601177) :-DD
-
OTP ROM may be a possibility. I am talking true OTP, not just UV erasable EPROM packaged in a plastic non-windowed package which is what most "OTP" ROMs are. True OTP would use a series of fusable links that are blown out or kept during programming. This seems like an immutable, physical change type of programming that can't occur or revert spontaneously. It should last. You may need to build something to read out the contents of your ROM chip in 25+ years, but that will always be possible.
I like this option. It's totally weird, and sounds fun!
Looks like the good kind, with the titanium-tungsten fuses, is rare but available. Here is 63S141: http://www.ebay.com/itm/222365236136 (http://www.ebay.com/itm/222365236136)
For utter coolness you could put a huge amount of SMT fuses on a board and some simple decoding logic to program and read them.
-
At what point would it be easier to just get some thin steel sheet, paint the sheet with a layer of black paint (if it helps) and then use a laser engraver to make dots in it ?
or maybe make some steel tape, a few bits per line .. some holes on the side to advance the tape and you should have a pretty reliable medium
Or maybe we could revert back to wire memory, since apparently you can still hear music and voice recorded decades ago like it was recorded today ?
(jump to ~ 8:20)
https://youtu.be/K_haRMbKZ1M
-
standard storage media + regular refresh of the media is the way to go. copy to NEW media in regular intervals (few years) and shred the old media.
This is basically what I do. I don't have too much stuff, right now it fits on less than 2 TB. I have it all stored on a microserver running FreeNAS (2 drives mirrored). I have a backup copy on another small computer running WHS2011. I update that backup about once a month and the rest of the time that computer is turned off. I keep a third copy, for just in case, on a external drive connected to yet another computer.
-
There are really only a few types of storage media with demonstrated long term reliability.
Cut letters in stone. Demonstrated thousands of years. Low density. Expensive.
Pressed letters in baked clay. Demonstrated thousands of years. Low density. Relatively cheap.
Ink on paper. Demonstrated reliability of a couple hundred years. Modest density. Cheap.
At the beginning of the PC era there was a technology based on printing pixels on paper. Assuming 300 dpi both directions a normal sheet of paper would be good for something like 5 megabits allowing some overhead for alignment and error correction. Decoding the result shouldn't be overly hard either since some form of camera/scanner is likely to be around for a long time.
-
Maxim iButton? If the capacity is enough.
Although, nowadays I would just place the info into into a password protected file and send myself in an email.
What happens when your hdd dies and you loose all your e-mail?
You mean when google looses all my emails? I just send a request to NSA, surely they have a copy of it.
Many years ago I had a hotmail account. Then Microsoft bought Hotmail and one day I signed in and found only one message that was a 'sorry we lost your e-mail' message. So it can happen.
-
Even if the emails are not lost, the software to recover them may become difficult or impossible to obtain. I have many megabytes of archived email from one system that I was using a couple of decades ago. I don't own the software currently, it is expensive and is likely to become impossible to get in the near future as it lost the war many years ago and has been hanging on by fingertips since. Another demonstration of why a rotating refresh is necessary if you really want to save the data. I obviously didn't want any of this data that badly.
-
if you are seriously in need of really long term storage, and you trust your encryption immensely, just make a shit load of different copies in different forms of media.
Magnetic storage like tapes tend to be a good format to use, but they can degrade over time. If you really have some important stuff, get multiple hard copies, and store it in a few bank safe deposit boxes.
Build a few metal boxes and bury it in a field somewhere.
Throw it on every cloud storage solution on the internet.
Just make sure that the files are impossibly well encrypted and that only you can get to them.
-
Unless you know your box is incredibly well sealed and inert i would say do not bury it in a feild, google is full of too many examples of time capsules, even well made ones seeping and turning into corroded mush after even 5 years.
-
Print on archival rated acid free paper, using a laser printer. place between 2 more acid free pages, and heat seal into a 1mm thick HDPE pocket, with a sachet of desiccant material in as well. Then place this in a separate laminated aluminised HDPE package for extra moisture resistance, then seal in a laser welded 0.8mm stainless steel box, and enclose that in pitch 50mm thick, and finally surround with 500mm of hydraulic concrete with a 30MPa strength, reinforced with glass fibre and carbon fibre.
Should last a couple of decades under water, or at least till it is subducted and crushed, so store in an old salt mine.
-
High density floating gate storage like NAND Flash has a horrible retention time so it would have to be continuously powered to allow scrubbing to take place. The problem is knowing which products do idle time scrubbing or scrubbing on read; even if they say they do, they might not.
Hard drives are better than NAND Flash but are still poor compared to alternatives unless you have a lot of data.
Low density floating gate storage like older NOR Flash, EPROM, and EEPROM will last 10s of years if not mistreated. 100 years is a real possibility.
Archival optical media can last 100 years and CD and DVD drives are going to be around for a long time. Store spare drives also. This is probably the best inexpensive option. I prefer DVDs because the media layer is protected on both sides. Store the disks in jewel cases and do not store anything with them which will outgas fumes which could attack polycarbonate.
Magneto optical media should last indefinitely but the availability of magneto optical drives is a problem.
If you have power, an internet connection, and NNTP access, then encrypt and upload to NNTP. Periodically download, compare, and upload again in a cycle that depends on NNTP retention. This cycle could be 6 months or more in some cases. As a bonus, now you and only you can access the data from anywhere in the world at any time.
In all cases use PAR2 files so recovery from small errors is possible.