"Extensive test" option during hard disk surface refresh

[ThomasBk]

Hello,

I want to use the function “Read + WRITE + read test (refresh data area)” in Hard Disk Sentinel Professional, to refresh my backup hard disks (external USB drives) to prevent data degradation (“bit rot”).

I have to decide if I should use the “extensive test” of which I have read in the help section (“an additional write cycle is used before writing back the contents to improve the efficiency of the error correction (drive regeneration)”).

If I understood correctly the extensive test is like this: read + write1 + write2 + read.
Write1 writes some kind of pattern, while write2 writes the original data from the first read operation.
Am I right?

What does exactly mean “to improve the efficiency of the error correction (drive regeneration)”? Does it mean that like this the drive will have more chance to detect a bad sector, with its internal bad/weak sector detection procedures?

Do you think that the extensive test increases the chance to corrupt the data on the hard disk, compared to the normal read+write+read test?
I'm thinking that in case of a problem (e.g. power loss) while the extensive test is doing its additional write (“write1” from above), then the current sector could become “wrong” (corrupted).
Instead if there is no additional write, the normal write (of the original data) would not cause a sector corruption, even if it has not yet completed writing the sector (because the remaining part of the sector to be written, would be already OK).
Does it make sense?

I have mostly 2 terabyte USB3.0 hard disks, some of which I turn on every 2 days, some others every 3-4 months, others every 2 years (don't know if this is of some interest).

Please help me to decide if I should enable the “extensive test” option or not, for refreshing my hard disks.

(Note: I didn't use Hard Disk Sentinel yet, but I'm planning to buy Hard Disk Sentinel Professional soon.)

Thanks.

[hdsentinel]

> If I understood correctly the extensive test is like this: read + write1 + write2 + read.
> Write1 writes some kind of pattern, while write2 writes the original data from the first read operation.
> Am I right?

Yes, you are 100% correct.

> What does exactly mean “to improve the efficiency of the error correction (drive regeneration)”?
> Does it mean that like this the drive will have more chance to detect a bad sector, with its internal bad/weak sector detection procedures?

Generally, to detect if a sector is bad / damaged / weak and treat them as should (for example to reallcate it to prevent its further use), the hard disk has special detection methods. For example the sector may be considered as bad if the sector can't be read correctly, the CRC check fails etc.
The extra write step helps as this way the stored data changed: so instead of simply writing back the original data, we first write something different - then write back the original data.
This would help the drive to re-calculate the CRC value, so refreshes not only the actual data, but also otherwise unaccessible area which stores CRC itself for the corresponding sector.
Before starting the test, it is possible to select the appropriate overwrite pattern: if you select Disk menu -> Surface test -> Read + Write + Read test (refresh data area), select the Configuration page in this window and there it is possible to select the "Write pattern" to be used in this step.


> Do you think that the extensive test increases the chance to corrupt the data on the hard disk, compared to the normal read+write+read test?

In theory yes, as changing the sector contents and re-calculating / re-writing the CRC may help the hard disk to recognise the possible weak / bad sector sooner, perform reallocation (if required) and then the actual data will be written back to the safe, spare area.

Apart from hard disks, I must tell that it may be even more important for flash based storage, where this "bit rot" effect (and re-writing the sector contents) are much more serious than for hard disks.


> I'm thinking that in case of a problem (e.g. power loss) while the extensive test is doing its additional write (“write1” from above),
> then the current sector could become “wrong” (corrupted).

Yes, this is absolutely true.
This is what the test description suggests: in most cases the test is safe, but on power loss, sudden disconnect, reset etc... the data may be corrupted.


> Instead if there is no additional write, the normal write (of the original data) would not cause a sector corruption,
> even if it has not yet completed writing the sector (because the remaining part of the sector to be written, would be already OK).
> Does it make sense?

Yes, it is true.


> Please help me to decide if I should enable the “extensive test” option or not, for refreshing my hard disks.

Yes, it may be enabled if you have protection against power loss (eg. use a UPS).
This would increase the test time with approx. 33% due to the extra write step performed.


If you are interested in further details about this Read + Write + Read surface test, I recommend to check this topic too:

viewtopic.php?f=32&t=10331

[ThomasBk]

The extra write step helps as this way the stored data changed: so instead of simply writing back the original data, we first write something different - then write back the original data.
This would help the drive to re-calculate the CRC value, so refreshes not only the actual data, but also otherwise unaccessible area which stores CRC itself for the corresponding sector.

There is something which is not clear to me. Why shouldn't just the writing back of the original data (without the "extensive test" option) not cause a re-calculation and re-writing of the CRC data? I don't think the hard disk knows that the exact same data is being written, and thus decides to not re-calculate and not re-write the CRC codes/data (but I'm no expert, I could be wrong). But perhaps there is some other advantage with the "extensive test" option, which helps to keep the CRC data in good "health"?

Anyway you made me think how important it is to make sure also the CRC gets refreshed/rewritten, because if the CRC-codes area of a sector gets corrupted, then also the actual data on that sector will be lost!
I don't have a UPS or any other protection against power loss, but if the extensive test is really useful for refreshing also the CRC area of the sectors, I think I will do this all the same. I have lots of backups so there wouldn't be a big problem (and anyway here the power outages are rare).

Which "write pattern" option (for the extensive test) do you suggest me for my purpose, which is to refresh all hard disk sectors, to prevent data degradation ("bit rot")?

Thanks.

[hdsentinel]

> There is something which is not clear to me. Why shouldn't just the writing back of the original
> data (without the "extensive test" option) not cause a re-calculation and re-writing of the CRC data?

If that additional step is missing, it is no problem: the test still causes re-calculation and re-writing the CRC data, the SAME data and values.

The idea of the additional step in the "extensive test" is to write back something different, to refresh the sectors. Not only the stored data, but their capability to hold different kind of data for a possible later re-use, re-format.

> Which "write pattern" option (for the extensive test) do you suggest me for my purpose, which
> is to refresh all hard disk sectors, to prevent data degradation ("bit rot")?

The default option (empty sectors) is fine, but you can use the random data option too.

[ThomasBk]

Sorry for asking again, but I didn't yet understand clearly what the additional write operation in the “extensive test” is useful for.
Is it useful to force a better detection (and reallocation) of bad/defective sectors? I see written “drive regeneration” in the help section of your web site, with which it seems to me you mean “reallocation of bad sectors” (if I understood correctly).

In case I'm not interested in detecting/reallocating bad sectors, but just in refreshing the data on the entire hard disk surface, do you think that the “Read + WRITE + read test (refresh data area)” WITHOUT the “extensive test” option is a good choice?

Thanks.

[hdsentinel]

The additional write operation changes the state of the stored bits, so not only the actual data will be rewritten (= refreshed) but also it forces the drive to change the contents, change the state of the bits storing the actual data.

This way the condition of the sectors remain high: we actually change the contents which may be important if you'd ever need to store other data.
This is where this additional write operation helps - in addition to the better detection and fixing of problems related to the sectors.

Yes, of course, if you do not prefer and simply want to refresh the stored data, you can use the test without the additional write operation.

[ThomasBk]

Hello,

You wrote: “The idea of the additional step in the "extensive test" is to write back something different, to refresh the sectors. Not only the stored data, but their capability to hold different kind of data for a possible later re-use, re-format.”

You also wrote: “This way the condition of the sectors remain high: we actually change the contents which may be important if you'd ever need to store other data. This is where this additional write operation helps [...]”

Do you mean that if a sector (on an hard disk) doesn't get written with different data for a long time, then this sector could start malfunctioning, and will have problems with storing data (and thus the written data could get corrupt)?

[hdsentinel]

> Do you mean that if a sector (on an hard disk) doesn't get written with different data for a
> long time, then this sector could start malfunctioning, and will have problems with storing
> data (and thus the written data could get corrupt)?

Absolutely NOT for hard disk.

I mean this is true for flash based storage (SD/CF memory cards, pendrives, SSDs, etc.)
This test originally designed for those devices: to not only refresh the actual data but also keep the SSD memory cells in good condition (to make sure they can be successfully altered / reprogrammed).

Sorry for the possible confusion.