Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse

NodeBB

  1. Home
  2. uncategorized
  3. okay fedi so I need some help - I now have set up my NAS, got all the drives installed, TrueNAS is running and the S.M.A.R.T.

okay fedi so I need some help - I now have set up my NAS, got all the drives installed, TrueNAS is running and the S.M.A.R.T.

Scheduled Pinned Locked Moved uncategorized
9 Posts 3 Posters 0 Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • magiclike@soc.sekundenklebertransportverbot.deM This user is from outside of this forum
    magiclike@soc.sekundenklebertransportverbot.deM This user is from outside of this forum
    magiclike@soc.sekundenklebertransportverbot.de
    wrote last edited by magiclike@soc.sekundenklebertransportverbot.de
    #1

    okay fedi so I need some help - I now have set up my NAS, got all the drives installed, TrueNAS is running and the S.M.A.R.T. tests all passed. Now I want to go the extra step to test for write errors. Obviously badblocks is there to test for that, but I'd rather want to use this more modern solution. The problem being, when I try to run cryptsetup on TrueNAS I get an error that dm_mod does not exist so that won't work.
    I found this forums thread that concludes to basically either dd-ing the drive with zeros and cmp-ing that to /dev/zero/ or just running a single pattern of badblocks.
    Do you have some advice for me here?
    For what is the cryptography layer even applied in the Alternative in archwiki?

    aetios@sns.minovsky.spaceA ? 2 Replies Last reply
    2
    0
    • magiclike@soc.sekundenklebertransportverbot.deM magiclike@soc.sekundenklebertransportverbot.de

      okay fedi so I need some help - I now have set up my NAS, got all the drives installed, TrueNAS is running and the S.M.A.R.T. tests all passed. Now I want to go the extra step to test for write errors. Obviously badblocks is there to test for that, but I'd rather want to use this more modern solution. The problem being, when I try to run cryptsetup on TrueNAS I get an error that dm_mod does not exist so that won't work.
      I found this forums thread that concludes to basically either dd-ing the drive with zeros and cmp-ing that to /dev/zero/ or just running a single pattern of badblocks.
      Do you have some advice for me here?
      For what is the cryptography layer even applied in the Alternative in archwiki?

      aetios@sns.minovsky.spaceA This user is from outside of this forum
      aetios@sns.minovsky.spaceA This user is from outside of this forum
      aetios@sns.minovsky.space
      wrote last edited by
      #2
      @MagicLike i think the crypto layer is two reasons
      1) it encrpyts the entire drive thus writing it fully
      2) write errors show up as 'not zeroes' in the decrypted data
      aetios@sns.minovsky.spaceA 1 Reply Last reply
      0
      • aetios@sns.minovsky.spaceA aetios@sns.minovsky.space
        @MagicLike i think the crypto layer is two reasons
        1) it encrpyts the entire drive thus writing it fully
        2) write errors show up as 'not zeroes' in the decrypted data
        aetios@sns.minovsky.spaceA This user is from outside of this forum
        aetios@sns.minovsky.spaceA This user is from outside of this forum
        aetios@sns.minovsky.space
        wrote last edited by
        #3
        @MagicLike it's a clever hack basically
        magiclike@soc.sekundenklebertransportverbot.deM 1 Reply Last reply
        0
        • aetios@sns.minovsky.spaceA aetios@sns.minovsky.space
          @MagicLike it's a clever hack basically
          magiclike@soc.sekundenklebertransportverbot.deM This user is from outside of this forum
          magiclike@soc.sekundenklebertransportverbot.deM This user is from outside of this forum
          magiclike@soc.sekundenklebertransportverbot.de
          wrote last edited by
          #4

          @aetios mhm - problem being that I can't run cryptsetup on TrueNAS so I would have to do it via live boot which means loosing remote access which is what I need qwq

          1 Reply Last reply
          0
          • magiclike@soc.sekundenklebertransportverbot.deM magiclike@soc.sekundenklebertransportverbot.de

            okay fedi so I need some help - I now have set up my NAS, got all the drives installed, TrueNAS is running and the S.M.A.R.T. tests all passed. Now I want to go the extra step to test for write errors. Obviously badblocks is there to test for that, but I'd rather want to use this more modern solution. The problem being, when I try to run cryptsetup on TrueNAS I get an error that dm_mod does not exist so that won't work.
            I found this forums thread that concludes to basically either dd-ing the drive with zeros and cmp-ing that to /dev/zero/ or just running a single pattern of badblocks.
            Do you have some advice for me here?
            For what is the cryptography layer even applied in the Alternative in archwiki?

            ? Offline
            ? Offline
            Guest
            wrote last edited by
            #5

            @MagicLike you mentioned truenas, do you plan to use zfs and a zraid with redundancy? (zraid1, zraid2, or the draid versions) if so, the best way would simply be to use the storage space, either by creating a large file with /dev/urandom, or copying your data. write errors will be detected on read or during zfs scrubs, automatically fixed by the file system, telling you which disk has an issue in `zpool status` or the web interface.

            disks usually handle write errors on their own, reallocating the defective sector, which should be reported in SMART. if you do opt to use dd or cryptsetup to write data linearly to the entire disk, make sure to also check if SMART values have changed afterwards. in those cases, i recommend to keep watching how it evolves over time - reallocated sectors can happen even in new good disks, but they should be rare. (also, SMART values are often very cryptic in a manufacturer-specific way)

            the reason cryptsetup is mentioned, is that it will turn a large amount of 0-bytes into a large amount of pseudo-random bytes, that can be predictably checked later without having to store a copy of the entire sequence. it's a fun relationship between seeded pseudo-random number generators (= you can generate many numbers from one 'seed' initial value), and encryption algorithms.
            the reason pseudo-random sequences are used, is that otherwise we cannot know whether the disk failed to read the value back and reported a bunch of 0's, or the same value as the previous disk sector, etc

            magiclike@soc.sekundenklebertransportverbot.deM 1 Reply Last reply
            0
            • ? Guest

              @MagicLike you mentioned truenas, do you plan to use zfs and a zraid with redundancy? (zraid1, zraid2, or the draid versions) if so, the best way would simply be to use the storage space, either by creating a large file with /dev/urandom, or copying your data. write errors will be detected on read or during zfs scrubs, automatically fixed by the file system, telling you which disk has an issue in `zpool status` or the web interface.

              disks usually handle write errors on their own, reallocating the defective sector, which should be reported in SMART. if you do opt to use dd or cryptsetup to write data linearly to the entire disk, make sure to also check if SMART values have changed afterwards. in those cases, i recommend to keep watching how it evolves over time - reallocated sectors can happen even in new good disks, but they should be rare. (also, SMART values are often very cryptic in a manufacturer-specific way)

              the reason cryptsetup is mentioned, is that it will turn a large amount of 0-bytes into a large amount of pseudo-random bytes, that can be predictably checked later without having to store a copy of the entire sequence. it's a fun relationship between seeded pseudo-random number generators (= you can generate many numbers from one 'seed' initial value), and encryption algorithms.
              the reason pseudo-random sequences are used, is that otherwise we cannot know whether the disk failed to read the value back and reported a bunch of 0's, or the same value as the previous disk sector, etc

              magiclike@soc.sekundenklebertransportverbot.deM This user is from outside of this forum
              magiclike@soc.sekundenklebertransportverbot.deM This user is from outside of this forum
              magiclike@soc.sekundenklebertransportverbot.de
              wrote last edited by
              #6

              @alice okay so first of all thank you for the extensive answer ❤

              yes I am planning on running my 4x 8TB drives in RAID-Z1, but I am not comfortable with putting my actual data on there with the (relatively small) risk of loosing them before I made a burn-in test. (I will be moving a lot of data actually so I want to be sure)
              Do you know how about long would it take to create a 24TB file using urandom?
              As cryptsetup is not an option here because TrueNAS simply dropped support for the required dm_mod and obviously won't let me modify the system that easy, I won't probably use it as if I would go for that I would have to live boot or something and thus loosing remote access which is crucial in this case. (and the risk of the thumb drive failing is also there when it is running for basically a week straight) After your explanation why cryptsetup is used I probably won't use dd as there won't be a way to tell if it is a zero I wrote or that the drive gives back.

              magiclike@soc.sekundenklebertransportverbot.deM ? 2 Replies Last reply
              0
              • magiclike@soc.sekundenklebertransportverbot.deM magiclike@soc.sekundenklebertransportverbot.de

                @alice okay so first of all thank you for the extensive answer ❤

                yes I am planning on running my 4x 8TB drives in RAID-Z1, but I am not comfortable with putting my actual data on there with the (relatively small) risk of loosing them before I made a burn-in test. (I will be moving a lot of data actually so I want to be sure)
                Do you know how about long would it take to create a 24TB file using urandom?
                As cryptsetup is not an option here because TrueNAS simply dropped support for the required dm_mod and obviously won't let me modify the system that easy, I won't probably use it as if I would go for that I would have to live boot or something and thus loosing remote access which is crucial in this case. (and the risk of the thumb drive failing is also there when it is running for basically a week straight) After your explanation why cryptsetup is used I probably won't use dd as there won't be a way to tell if it is a zero I wrote or that the drive gives back.

                magiclike@soc.sekundenklebertransportverbot.deM This user is from outside of this forum
                magiclike@soc.sekundenklebertransportverbot.deM This user is from outside of this forum
                magiclike@soc.sekundenklebertransportverbot.de
                wrote last edited by
                #7

                @alice the other option would ofc be to resort back to badblocks

                1 Reply Last reply
                0
                • magiclike@soc.sekundenklebertransportverbot.deM magiclike@soc.sekundenklebertransportverbot.de

                  @alice okay so first of all thank you for the extensive answer ❤

                  yes I am planning on running my 4x 8TB drives in RAID-Z1, but I am not comfortable with putting my actual data on there with the (relatively small) risk of loosing them before I made a burn-in test. (I will be moving a lot of data actually so I want to be sure)
                  Do you know how about long would it take to create a 24TB file using urandom?
                  As cryptsetup is not an option here because TrueNAS simply dropped support for the required dm_mod and obviously won't let me modify the system that easy, I won't probably use it as if I would go for that I would have to live boot or something and thus loosing remote access which is crucial in this case. (and the risk of the thumb drive failing is also there when it is running for basically a week straight) After your explanation why cryptsetup is used I probably won't use dd as there won't be a way to tell if it is a zero I wrote or that the drive gives back.

                  ? Offline
                  ? Offline
                  Guest
                  wrote last edited by
                  #8

                  @MagicLike the time it takes to create the large file will heavily depend on the drives and the rest of your system, but it's fair to assume it will take a very long time. at least it should be somewhat distributed over the disks by the raid z1. but there won't be any faster way if you must write to every sector.

                  personally (i don't know of any unambiguous consensus either way) i would not recommend doing a full-capacity-write burn-in test - it is needless stress for the drives. i would copy the data directly, always keeping a backup which are important in all situations, and fill the rest of the space with a large random file, then run a zfs scrub and check smart values. it will have verified the entire capacity is writable and re-readable, in half the time.

                  magiclike@soc.sekundenklebertransportverbot.deM 1 Reply Last reply
                  0
                  • ? Guest

                    @MagicLike the time it takes to create the large file will heavily depend on the drives and the rest of your system, but it's fair to assume it will take a very long time. at least it should be somewhat distributed over the disks by the raid z1. but there won't be any faster way if you must write to every sector.

                    personally (i don't know of any unambiguous consensus either way) i would not recommend doing a full-capacity-write burn-in test - it is needless stress for the drives. i would copy the data directly, always keeping a backup which are important in all situations, and fill the rest of the space with a large random file, then run a zfs scrub and check smart values. it will have verified the entire capacity is writable and re-readable, in half the time.

                    magiclike@soc.sekundenklebertransportverbot.deM This user is from outside of this forum
                    magiclike@soc.sekundenklebertransportverbot.deM This user is from outside of this forum
                    magiclike@soc.sekundenklebertransportverbot.de
                    wrote last edited by
                    #9

                    @alice mhm, I have the problem that I am running out of disk space everywhere and this NAS is going to be my archive and backup space. I know that I won't follow any backup rules with this necessarily and not even 3-2-1 for sure but this is what I have available right now.

                    1 Reply Last reply
                    0
                    Reply
                    • Reply as topic
                    Log in to reply
                    • Oldest to Newest
                    • Newest to Oldest
                    • Most Votes


                    • Login

                    • Login or register to search.
                    Powered by NodeBB Contributors
                    • First post
                      Last post
                    0
                    • Categories
                    • Recent
                    • Tags
                    • Popular
                    • World
                    • Users
                    • Groups