Backup SSD - how to keep data

[golden_labels]

I am reading that last post and I am not sure, if we’re on the same page.

We are talking about ensuring data is not damaged. How is erasing blocks as a part of housekeeping relevant? Firmware may not do this, it likely will not, but how does that even matter in this topic? Improving future performance is not a part of ensuring already written data remain intact.

Yes, probably only manufacturer knows. Yet you made a statement, which suggests that you know too. Hence my question.

Your assessment regarding doing data-retention housekeeping is right. It’s unlikely firmware will do it. But the opposite scenario is the user doing the same thing actively. You’re trying to tell me, that doing something 100% of time (active) is wasting more time than doing something unknown percentage of time (passive housekeeping). This may only be true if firmware is also doing this around⁽¹⁾ 100% of time, but then you also say that “only manufacturer knows”. Pardon?

⁽¹⁾ Does not need to be exactly 100%: see my note above on why active reading is suboptimal.

[wraper]

I am reading that last post and I am not sure, if we’re on the same page.

We are talking about ensuring data is not damaged. How is erasing blocks as a part of housekeeping relevant? Firmware may not do this, it likely will not, but how does that even matter in this topic? Improving future performance is not a part of ensuring already written data remain intact.

It's relevant in a way that data is refreshed by writing it to another place. And SSD does it's internal data shuffling when idle. How firmware does not do this? Unlike with HDD, system does not really know what is stored where. SSD does all of it's data shuffling like wear leveling and garbage collection internally. Data allocation that system sees has nothing to do with what actually happens inside SSD.

[David Hess]

Restore from EXT4 Media:sudo dd if=/mnt/usbhdd/ssd.img of=/dev/sdb bs=10MB conv=noerror,sync status=progress - again, mostly limited by HDD transfer speed to ~88MB/s

I actually got better performance backing up an SSD to a modern hard drive rather than to an identical SSD.  The Crucial MX500 SSDs that I use, and the Samsung EVO 270 that I tested, drop to about 120 MB/s during continuous writes while the hard drives that I have manage almost 200 MB/s.  I may end up using my workstation RAID to backup my portable SSDs because it is so much faster.

[David Hess]

You should take into account that while reading SSD you waste its time it could spend on housekeeping. So if you read it all and then turn off, it's worse than just leaving it powered and doing nothing for the same amount of time.

Finally an answer. However, that response suggests that the firmware is doing refreshing 100% of the time, if the SSD is unused. Is this true?

Unless the manufacturer says otherwise, I would assume that the only idle-time housekeeping performed is flushing SLC buffers to multi-level cell storage and erasing empty blocks.  The idea behind reading the entire contents is to force scrub-on-read operations.

[wraper]

Unless the manufacturer says otherwise, I would assume that the only idle-time housekeeping performed is flushing SLC buffers to multi-level cell storage and erasing empty blocks.  The idea behind reading the entire contents is to force scrub-on-read operations.

It does not work like this. Phisical block size is like 1MiB - 24MiB which is way larger than logical block. Without shuffling the data, SSD can run out of memory very fast on some use cases.

[harerod]

In my book, the experiment is the base for science. Since I don't have any manufacturer information regarding the internals of the SSD, I observe a black box with tools available to me. I thank you all for the hints you shared. Let me in return share my findings:

Copy SSD to an image file: sudo pv /dev/sda > /mnt/usbhdd/ssd_c_2310081711_pv.img
Calculate md5 of that image: time sudo md5sum /mnt/usbhdd/ssd_c_2310081711_pv.img 
Result -> ef82749208b7a95f36517c980fa95555
Read blocksizes of the devices involved, i.e.: stat /dev/sdb
Write image back to the SSD: sudo dd if=/mnt/usbhdd/ssd_c_2310090820_pv.img of=/dev/sdb status=progress bs=4096
Calculate md5 of the SSD: time sudo md5sum /dev/sdb
md5 of SSD and image match.

[DiTBho]

bah, it doesn't make sense to hash the entire disk image: if it matches, ok, otherwise you don't know "WHAT?" got corrupted!

It makes more sense to develop software that hashes every single file, and then checks every single file by mounting the image in loopback.

This is how it works the software I developed for my NAS, and that's because I can't afford filesystems { ZFS-*, BTrfs, ...} that do the same at a low level

I have implemented it in order to prevent the risk of "bit flips", as, talking about HDDs, beyond physical damages of { plates, flying read/write hears, motor, pcb, connector, air-filter, ...}, there’s another threat to the files stored on hard disks: small, silent bit flip errors often called data corruption or "bit rot".

"Bit rot errors" occur when individual bits in a stream of data in files change from one state to another (positive or negative, 0 to 1, and vice versa).

I talked about HDDs, but these errors can also happen to flash storage systems (SSD) at rest, or be introduced as a file is copied from one hard drive to another.

[David Hess]

Unless the manufacturer says otherwise, I would assume that the only idle-time housekeeping performed is flushing SLC buffers to multi-level cell storage and erasing empty blocks.  The idea behind reading the entire contents is to force scrub-on-read operations.

It does not work like this. Phisical block size is like 1MiB - 24MiB which is way larger than logical block. Without shuffling the data, SSD can run out of memory very fast on some use cases.

The largest ones I could gain access to now use 1MB blocks with 128 pages.  The datasheets for anything larger are behind NDAs.

[David Hess]

It makes more sense to develop software that hashes every single file, and then checks every single file by mounting the image in loopback.

That is what I do now.  So far the only SSD I have used which produced bad data was a Samsung EVO 970 or 980.

More often I get data corruption from network cards until I disable their hardware checksum acceleration.  I think my old non-ECC Intel 870 system is also generating errors, but I just retired it.

[golden_labels]

In my book, the experiment is the base for science. Since I don't have any manufacturer information regarding the internals of the SSD, I observe a black box with tools available to me. I thank you all for the hints you shared. Let me in return share my findings:

IMO you did everything right,⁽¹⁾ though note that the experiment does not confirm this.

It's relevant in a way that data is refreshed by writing it to another place.

I do not like being taken for and idiot. Neither I like wasting my time on silly arguing like this. But here people’s data safety is at stake.

I can’t read your mind, so perhaps either you are expressing your ideas poorly or I am completely misreading it, but from my subjective perspective the situation looks as follows.

In the other thread you made a statement, nature of which requires both deeper knowledge on firmware’s operation and providing a specific technial information. Despite being asked to clarify, you never shown you know either.⁽²⁾ In this thread you deliver an even narrower version of the same statement, which is one step from plunging into the abyss of absurdity. Instead of explaining you engage in delivering claims, that are likely true in separation, but disconnected from what they are mean to support.

So, could you please stay in the context of your own statement and this thread? And deliver explanation that is relevant to what you claim? Because so far my impression of the situation is very bad.

⁽¹⁾ Of course I should whine about using dd once again, but one person can’t safe the world. I count removal of the data-destroying conv a success.
⁽²⁾ And I did not ask for authoritative information. Personal experience and reports from other people are fine too, as long as the reader has no doubts about the answer being honest. This condition can no longer be met, but in the other thread it was different.

[wraper]

What isn't clear in what I said? SSD can only refresh data by writing it to another free location. It's not an effing DRAM or something. Moving the data and erasing free space for it's internal purposes generally happens when it's idle to not degrade performance.

[DiTBho]

That is what I do now

yup, that's the right way.
What do you use? Custom software, or an already existing solution?

More often I get data corruption from network cards until I disable their hardware checksum acceleration

Ummm, it sounds kernel bug. I had a similar issue with Jumbo frames & chk offloading on a Tulip quad-ethernet 10/100Mbps network card made by Digital in 2001.

"jumbo frames" have NON-standard length and the circuit that generated the checksum had not been reconfigured correctly, so a couple of bytes were eaten every now and then, the payload passed intact, but the checksum every now and then was done on a smaller block short, so it was wrong, no problem for UDP/IP, but for TCP/IP that was a random discard, go-back n, and the "error rate" was very high.

5 lines of C. Problem solved. 15 bloody weeks to figure it out!

At first I thought it was faulty hardware (as it's second hand, ex corporate stuff), so I bought 4 other network cards, which I assure you are not small, they are very long and take along the entire length of the PCI slot.

All 4 showed the same defect. At that point I started looking in the kernel 

[DiTBho]

SSD can only refresh data by writing it to another free location

yup, I can confirm this, and add that "Bit rot errors" can occur.
It doesn't necessarily happen, but it can happen, and it would be best to keep this in mind.

[David Hess]

That is what I do now

yup, that's the right way.
What do you use? Custom software, or an already existing solution?

I mostly run Windows so use the Hashcheck extension to File Explorer.  It can generate hash files for entire directory structures or an entire volume.  On SSDs it will hash multiple files at once and completely saturate the CPU.  Unfortunately it does not do this with RAID which is not reported as an SSD because Windows' SSD detection is broken, has always been broken, and based on history, will remain broken.  I think this is because Microsoft prefers that users rely on their Cloud storage services, so they deliberately cripple their OS to make local storage perform worse.

More often I get data corruption from network cards until I disable their hardware checksum acceleration

Ummm, it sounds kernel bug. I had a similar issue with Jumbo frames & chk offloading on a Tulip quad-ethernet 10/100Mbps network card made by Digital in 2001.

It comes up sometimes with Realtek, especially their older hardware, and always with ASIX (USB to Ethernet), which is just broken.

"jumbo frames" have NON-standard length and the circuit that generated the checksum had not been reconfigured correctly, so a couple of bytes were eaten every now and then, the payload passed intact, but the checksum every now and then was done on a smaller block short, so it was wrong, no problem for UDP/IP, but for TCP/IP that was a random discard, go-back n, and the "error rate" was very high.

I stopped using Jumbo Frames.  My hardware cannot saturate my network anyway, although it comes close, and Jumbo Frames does not provide any measurable advantage for me.

5 lines of C. Problem solved. 15 bloody weeks to figure it out!

At first I thought it was faulty hardware (as it's second hand, ex corporate stuff), so I bought 4 other network cards, which I assure you are not small, they are very long and take along the entire length of the PCI slot.

All 4 showed the same defect. At that point I started looking in the kernel 

Reading through the FreeBSD notes about network cards show that hardware bugs are often fixed with better drivers, but sometimes it is necessarily to forgo problematical features.  Hardware checksum offloading does not provide much performance advantage anymore, at least at 1 gigabit speeds.

My old Intel 870 system managed to create a hash file showing corruption before the data was transferred over the network, so I am sure something is wrong with it.  The problem could have been a bad disk, but the storage volumes were either RAID or Windows Storage Spaces.

[evaadms]

Link your external SSD or hard drive to your computer so that it can be used as a backup. By right-clicking on the internal disk you want to backup in the Navigation Pane on the left and choosing Properties, you can confirm that your external HDD or SSD has the same capacity as or more than the internal drive.