okay fedi so I need some help - I now have set up my NAS, got all the drives installed, TrueNAS is running and the S.M.A.R.T.
-
okay fedi so I need some help - I now have set up my NAS, got all the drives installed, TrueNAS is running and the S.M.A.R.T. tests all passed. Now I want to go the extra step to test for write errors. Obviously
badblocks
is there to test for that, but I'd rather want to use this more modern solution. The problem being, when I try to runcryptsetup
on TrueNAS I get an error thatdm_mod
does not exist so that won't work.
I found this forums thread that concludes to basically eitherdd
-ing the drive with zeros andcmp
-ing that to/dev/zero/
or just running a single pattern ofbadblocks
.
Do you have some advice for me here?
For what is the cryptography layer even applied in the Alternative in archwiki? -
-
okay fedi so I need some help - I now have set up my NAS, got all the drives installed, TrueNAS is running and the S.M.A.R.T. tests all passed. Now I want to go the extra step to test for write errors. Obviously
badblocks
is there to test for that, but I'd rather want to use this more modern solution. The problem being, when I try to runcryptsetup
on TrueNAS I get an error thatdm_mod
does not exist so that won't work.
I found this forums thread that concludes to basically eitherdd
-ing the drive with zeros andcmp
-ing that to/dev/zero/
or just running a single pattern ofbadblocks
.
Do you have some advice for me here?
For what is the cryptography layer even applied in the Alternative in archwiki?@MagicLike i think the crypto layer is two reasons
1) it encrpyts the entire drive thus writing it fully
2) write errors show up as 'not zeroes' in the decrypted data -
@MagicLike i think the crypto layer is two reasons
1) it encrpyts the entire drive thus writing it fully
2) write errors show up as 'not zeroes' in the decrypted data@MagicLike it's a clever hack basically -
@MagicLike it's a clever hack basically
@aetios mhm - problem being that I can't run cryptsetup on TrueNAS so I would have to do it via live boot which means loosing remote access which is what I need qwq
-
okay fedi so I need some help - I now have set up my NAS, got all the drives installed, TrueNAS is running and the S.M.A.R.T. tests all passed. Now I want to go the extra step to test for write errors. Obviously
badblocks
is there to test for that, but I'd rather want to use this more modern solution. The problem being, when I try to runcryptsetup
on TrueNAS I get an error thatdm_mod
does not exist so that won't work.
I found this forums thread that concludes to basically eitherdd
-ing the drive with zeros andcmp
-ing that to/dev/zero/
or just running a single pattern ofbadblocks
.
Do you have some advice for me here?
For what is the cryptography layer even applied in the Alternative in archwiki?@MagicLike you mentioned truenas, do you plan to use zfs and a zraid with redundancy? (zraid1, zraid2, or the draid versions) if so, the best way would simply be to use the storage space, either by creating a large file with /dev/urandom, or copying your data. write errors will be detected on read or during zfs scrubs, automatically fixed by the file system, telling you which disk has an issue in `zpool status` or the web interface.
disks usually handle write errors on their own, reallocating the defective sector, which should be reported in SMART. if you do opt to use dd or cryptsetup to write data linearly to the entire disk, make sure to also check if SMART values have changed afterwards. in those cases, i recommend to keep watching how it evolves over time - reallocated sectors can happen even in new good disks, but they should be rare. (also, SMART values are often very cryptic in a manufacturer-specific way)
the reason cryptsetup is mentioned, is that it will turn a large amount of 0-bytes into a large amount of pseudo-random bytes, that can be predictably checked later without having to store a copy of the entire sequence. it's a fun relationship between seeded pseudo-random number generators (= you can generate many numbers from one 'seed' initial value), and encryption algorithms.
the reason pseudo-random sequences are used, is that otherwise we cannot know whether the disk failed to read the value back and reported a bunch of 0's, or the same value as the previous disk sector, etc -
@MagicLike you mentioned truenas, do you plan to use zfs and a zraid with redundancy? (zraid1, zraid2, or the draid versions) if so, the best way would simply be to use the storage space, either by creating a large file with /dev/urandom, or copying your data. write errors will be detected on read or during zfs scrubs, automatically fixed by the file system, telling you which disk has an issue in `zpool status` or the web interface.
disks usually handle write errors on their own, reallocating the defective sector, which should be reported in SMART. if you do opt to use dd or cryptsetup to write data linearly to the entire disk, make sure to also check if SMART values have changed afterwards. in those cases, i recommend to keep watching how it evolves over time - reallocated sectors can happen even in new good disks, but they should be rare. (also, SMART values are often very cryptic in a manufacturer-specific way)
the reason cryptsetup is mentioned, is that it will turn a large amount of 0-bytes into a large amount of pseudo-random bytes, that can be predictably checked later without having to store a copy of the entire sequence. it's a fun relationship between seeded pseudo-random number generators (= you can generate many numbers from one 'seed' initial value), and encryption algorithms.
the reason pseudo-random sequences are used, is that otherwise we cannot know whether the disk failed to read the value back and reported a bunch of 0's, or the same value as the previous disk sector, etc@alice okay so first of all thank you for the extensive answer
yes I am planning on running my 4x 8TB drives in RAID-Z1, but I am not comfortable with putting my actual data on there with the (relatively small) risk of loosing them before I made a burn-in test. (I will be moving a lot of data actually so I want to be sure)
Do you know how about long would it take to create a 24TB file using urandom?
As cryptsetup is not an option here because TrueNAS simply dropped support for the required dm_mod and obviously won't let me modify the system that easy, I won't probably use it as if I would go for that I would have to live boot or something and thus loosing remote access which is crucial in this case. (and the risk of the thumb drive failing is also there when it is running for basically a week straight) After your explanation why cryptsetup is used I probably won't use dd as there won't be a way to tell if it is a zero I wrote or that the drive gives back. -
@alice okay so first of all thank you for the extensive answer
yes I am planning on running my 4x 8TB drives in RAID-Z1, but I am not comfortable with putting my actual data on there with the (relatively small) risk of loosing them before I made a burn-in test. (I will be moving a lot of data actually so I want to be sure)
Do you know how about long would it take to create a 24TB file using urandom?
As cryptsetup is not an option here because TrueNAS simply dropped support for the required dm_mod and obviously won't let me modify the system that easy, I won't probably use it as if I would go for that I would have to live boot or something and thus loosing remote access which is crucial in this case. (and the risk of the thumb drive failing is also there when it is running for basically a week straight) After your explanation why cryptsetup is used I probably won't use dd as there won't be a way to tell if it is a zero I wrote or that the drive gives back.@alice the other option would ofc be to resort back to badblocks
-
@alice okay so first of all thank you for the extensive answer
yes I am planning on running my 4x 8TB drives in RAID-Z1, but I am not comfortable with putting my actual data on there with the (relatively small) risk of loosing them before I made a burn-in test. (I will be moving a lot of data actually so I want to be sure)
Do you know how about long would it take to create a 24TB file using urandom?
As cryptsetup is not an option here because TrueNAS simply dropped support for the required dm_mod and obviously won't let me modify the system that easy, I won't probably use it as if I would go for that I would have to live boot or something and thus loosing remote access which is crucial in this case. (and the risk of the thumb drive failing is also there when it is running for basically a week straight) After your explanation why cryptsetup is used I probably won't use dd as there won't be a way to tell if it is a zero I wrote or that the drive gives back.@MagicLike the time it takes to create the large file will heavily depend on the drives and the rest of your system, but it's fair to assume it will take a very long time. at least it should be somewhat distributed over the disks by the raid z1. but there won't be any faster way if you must write to every sector.
personally (i don't know of any unambiguous consensus either way) i would not recommend doing a full-capacity-write burn-in test - it is needless stress for the drives. i would copy the data directly, always keeping a backup which are important in all situations, and fill the rest of the space with a large random file, then run a zfs scrub and check smart values. it will have verified the entire capacity is writable and re-readable, in half the time. -
@MagicLike the time it takes to create the large file will heavily depend on the drives and the rest of your system, but it's fair to assume it will take a very long time. at least it should be somewhat distributed over the disks by the raid z1. but there won't be any faster way if you must write to every sector.
personally (i don't know of any unambiguous consensus either way) i would not recommend doing a full-capacity-write burn-in test - it is needless stress for the drives. i would copy the data directly, always keeping a backup which are important in all situations, and fill the rest of the space with a large random file, then run a zfs scrub and check smart values. it will have verified the entire capacity is writable and re-readable, in half the time.@alice mhm, I have the problem that I am running out of disk space everywhere and this NAS is going to be my archive and backup space. I know that I won't follow any backup rules with this necessarily and not even 3-2-1 for sure but this is what I have available right now.