Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • World
  • Users
  • Groups
Skins
  • Light
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse

NodeBB

  1. Home
  2. uncategorized
  3. okay, duckduckfedi, new question

okay, duckduckfedi, new question

Scheduled Pinned Locked Moved uncategorized
19 Posts 4 Posters 0 Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • alexia@starlightnet.workA This user is from outside of this forum
    alexia@starlightnet.workA This user is from outside of this forum
    alexia@starlightnet.work
    wrote last edited by
    #1
    okay, duckduckfedi, new question

    turns out it's
    NOT mostly Minecraft taking up storage on our VPS, it's actually Iceshrimp.NET

    Anyone got sometips on cleaning that up? The thing runs with postgres so tips on cleaning up postgres databases would be helpful, same for maybe configuration options to give iceshrimp.NET to make it take less storage

    EDIT: yeah setting media retention to 0 days is actually a good idea ❤

    thank you!!
    ​​ ​​
    kopper@wetdry.worldK ? 2 Replies Last reply
    0
    • alexia@starlightnet.workA alexia@starlightnet.work
      okay, duckduckfedi, new question

      turns out it's
      NOT mostly Minecraft taking up storage on our VPS, it's actually Iceshrimp.NET

      Anyone got sometips on cleaning that up? The thing runs with postgres so tips on cleaning up postgres databases would be helpful, same for maybe configuration options to give iceshrimp.NET to make it take less storage

      EDIT: yeah setting media retention to 0 days is actually a good idea ❤

      thank you!!
      ​​ ​​
      kopper@wetdry.worldK This user is from outside of this forum
      kopper@wetdry.worldK This user is from outside of this forum
      kopper@wetdry.world
      wrote last edited by
      #2

      @alexia is it media or the database that's causing you trouble? not caching remote media can save quite a bit of storage very easily

      alexia@starlightnet.workA 1 Reply Last reply
      0
      • alexia@starlightnet.workA alexia@starlightnet.work
        okay, duckduckfedi, new question

        turns out it's
        NOT mostly Minecraft taking up storage on our VPS, it's actually Iceshrimp.NET

        Anyone got sometips on cleaning that up? The thing runs with postgres so tips on cleaning up postgres databases would be helpful, same for maybe configuration options to give iceshrimp.NET to make it take less storage

        EDIT: yeah setting media retention to 0 days is actually a good idea ❤

        thank you!!
        ​​ ​​
        ? Offline
        ? Offline
        Guest
        wrote last edited by
        #3

        @alexia are you using filesystem compression

        alexia@starlightnet.workA kopper@wetdry.worldK 2 Replies Last reply
        0
        • ? Guest

          @alexia are you using filesystem compression

          alexia@starlightnet.workA This user is from outside of this forum
          alexia@starlightnet.workA This user is from outside of this forum
          alexia@starlightnet.work
          wrote last edited by
          #4
          @soop No, unfortunately our VPS uses ext4 so there is no option for transparent filesystem compression

          I know there is in-place conversion from ext4 to btrfs, but we'd rather not risk bricking
          everything we host which includes both our fedi and our XMPP which are very important to us

          Maybe with a future upgrade we'll remember to do btrfs, then use zstd compression there.
          ? sqaaakoi@wetdry.worldS 2 Replies Last reply
          0
          • alexia@starlightnet.workA alexia@starlightnet.work
            @soop No, unfortunately our VPS uses ext4 so there is no option for transparent filesystem compression

            I know there is in-place conversion from ext4 to btrfs, but we'd rather not risk bricking
            everything we host which includes both our fedi and our XMPP which are very important to us

            Maybe with a future upgrade we'll remember to do btrfs, then use zstd compression there.
            ? Offline
            ? Offline
            Guest
            wrote last edited by
            #5

            @alexia owie (and that's why you don't opt for ext4 in 2025)

            alexia@starlightnet.workA 1 Reply Last reply
            0
            • kopper@wetdry.worldK kopper@wetdry.world

              @alexia is it media or the database that's causing you trouble? not caching remote media can save quite a bit of storage very easily

              alexia@starlightnet.workA This user is from outside of this forum
              alexia@starlightnet.workA This user is from outside of this forum
              alexia@starlightnet.work
              wrote last edited by
              #6
              @kopper honest to god I just checked with the du command what was taking up so much storage and got flashbanged with 51G /srv/iceshrimp and didn't bother checking any further than that

              but yeah good point we should probably turn that off
              kopper@wetdry.worldK 1 Reply Last reply
              0
              • ? Guest

                @alexia owie (and that's why you don't opt for ext4 in 2025)

                alexia@starlightnet.workA This user is from outside of this forum
                alexia@starlightnet.workA This user is from outside of this forum
                alexia@starlightnet.work
                wrote last edited by
                #7
                @soop ext4 is solid still, we actually both wanted btrfs but didn't think of it at the time we set everything up.
                ? 1 Reply Last reply
                0
                • alexia@starlightnet.workA alexia@starlightnet.work
                  @kopper honest to god I just checked with the du command what was taking up so much storage and got flashbanged with 51G /srv/iceshrimp and didn't bother checking any further than that

                  but yeah good point we should probably turn that off
                  kopper@wetdry.worldK This user is from outside of this forum
                  kopper@wetdry.worldK This user is from outside of this forum
                  kopper@wetdry.world
                  wrote last edited by
                  #8

                  @alexia yeah that'll be media then. the database is harder to clean up but will show up under postgres (assuming you don't have both under the same directory)

                  kopper@wetdry.worldK 1 Reply Last reply
                  0
                  • kopper@wetdry.worldK kopper@wetdry.world

                    @alexia yeah that'll be media then. the database is harder to clean up but will show up under postgres (assuming you don't have both under the same directory)

                    kopper@wetdry.worldK This user is from outside of this forum
                    kopper@wetdry.worldK This user is from outside of this forum
                    kopper@wetdry.world
                    wrote last edited by
                    #9

                    @alexia https://iceshrimp.dev/iceshrimp/Iceshrimp.NET/src/branch/dev/Iceshrimp.Backend/configuration.ini#L175-L179

                    set these to 0 and send an API call to /api/iceshrimp/admin/drive/prune-expired-media (or in the dev versions there should be a button on the admin panel for it, unsure where though)

                    alexia@starlightnet.workA 1 Reply Last reply
                    0
                    • alexia@starlightnet.workA alexia@starlightnet.work
                      @soop ext4 is solid still, we actually both wanted btrfs but didn't think of it at the time we set everything up.
                      ? Offline
                      ? Offline
                      Guest
                      wrote last edited by
                      #10

                      @alexia if you just want optimal compression bcachefs is very efficent at that, but the current implementation is still a bit unstable and shaky performance wise so you shouldn't be using it yet, but it's something to note

                      alexia@starlightnet.workA 1 Reply Last reply
                      0
                      • ? Guest

                        @alexia if you just want optimal compression bcachefs is very efficent at that, but the current implementation is still a bit unstable and shaky performance wise so you shouldn't be using it yet, but it's something to note

                        alexia@starlightnet.workA This user is from outside of this forum
                        alexia@starlightnet.workA This user is from outside of this forum
                        alexia@starlightnet.work
                        wrote last edited by
                        #11
                        @soop btrfs with zstd:4 would be more than enough for us, and is considered much more stable.

                        Just ask
                        @niko what zstd:3 did to her server
                        ? 1 Reply Last reply
                        0
                        • kopper@wetdry.worldK kopper@wetdry.world

                          @alexia https://iceshrimp.dev/iceshrimp/Iceshrimp.NET/src/branch/dev/Iceshrimp.Backend/configuration.ini#L175-L179

                          set these to 0 and send an API call to /api/iceshrimp/admin/drive/prune-expired-media (or in the dev versions there should be a button on the admin panel for it, unsure where though)

                          alexia@starlightnet.workA This user is from outside of this forum
                          alexia@starlightnet.workA This user is from outside of this forum
                          alexia@starlightnet.work
                          wrote last edited by
                          #12
                          @kopper I can just /swagger that shit to hit the endpoint no problem

                          thank you! ❤
                          alexia@starlightnet.workA 1 Reply Last reply
                          0
                          • alexia@starlightnet.workA alexia@starlightnet.work
                            @soop btrfs with zstd:4 would be more than enough for us, and is considered much more stable.

                            Just ask
                            @niko what zstd:3 did to her server
                            ? Offline
                            ? Offline
                            Guest
                            wrote last edited by
                            #13

                            @alexia @niko I use zstd:9 on my desktop :3 and performance is pretty good still

                            ? 1 Reply Last reply
                            0
                            • ? Guest

                              @alexia are you using filesystem compression

                              kopper@wetdry.worldK This user is from outside of this forum
                              kopper@wetdry.worldK This user is from outside of this forum
                              kopper@wetdry.world
                              wrote last edited by
                              #14

                              @soop @alexia don't think compression would help a ton here tbh. media files already tend to be really compressed, and postgres also handles some compression (at least for text) on it's own

                              kopper@wetdry.worldK 1 Reply Last reply
                              0
                              • alexia@starlightnet.workA alexia@starlightnet.work
                                @soop No, unfortunately our VPS uses ext4 so there is no option for transparent filesystem compression

                                I know there is in-place conversion from ext4 to btrfs, but we'd rather not risk bricking
                                everything we host which includes both our fedi and our XMPP which are very important to us

                                Maybe with a future upgrade we'll remember to do btrfs, then use zstd compression there.
                                sqaaakoi@wetdry.worldS This user is from outside of this forum
                                sqaaakoi@wetdry.worldS This user is from outside of this forum
                                sqaaakoi@wetdry.world
                                wrote last edited by
                                #15

                                @alexia @soop btrfs-convert has failed and "ate my data" multiple times (i converted a disk image in place and also tried to make a new filesystem using the --rootdir flag and both times it failed)

                                1 Reply Last reply
                                0
                                • ? Guest

                                  @alexia @niko I use zstd:9 on my desktop :3 and performance is pretty good still

                                  ? Offline
                                  ? Offline
                                  Guest
                                  wrote last edited by
                                  #16

                                  @alexia @niko fyi once you do get set up with btrfs you'll want to disable the minecraft server\s builtin compression of region files

                                  alexia@starlightnet.workA 1 Reply Last reply
                                  0
                                  • ? Guest

                                    @alexia @niko fyi once you do get set up with btrfs you'll want to disable the minecraft server\s builtin compression of region files

                                    alexia@starlightnet.workA This user is from outside of this forum
                                    alexia@starlightnet.workA This user is from outside of this forum
                                    alexia@starlightnet.work
                                    wrote last edited by
                                    #17
                                    @soop Yes, I am aware.
                                    1 Reply Last reply
                                    0
                                    • kopper@wetdry.worldK kopper@wetdry.world

                                      @soop @alexia don't think compression would help a ton here tbh. media files already tend to be really compressed, and postgres also handles some compression (at least for text) on it's own

                                      kopper@wetdry.worldK This user is from outside of this forum
                                      kopper@wetdry.worldK This user is from outside of this forum
                                      kopper@wetdry.world
                                      wrote last edited by
                                      #18

                                      @soop @alexia looking into it a bit more, it seems like may actually help postgres in terms of disk space, but i'm unsure how much the perf regressions would be worth it

                                      1 Reply Last reply
                                      0
                                      • alexia@starlightnet.workA alexia@starlightnet.work
                                        @kopper I can just /swagger that shit to hit the endpoint no problem

                                        thank you! ❤
                                        alexia@starlightnet.workA This user is from outside of this forum
                                        alexia@starlightnet.workA This user is from outside of this forum
                                        alexia@starlightnet.work
                                        wrote last edited by
                                        #19
                                        @kopper it is now slowly deleting 50G of data

                                        and that would be most of our space reclaimed then
                                        ​​
                                        1 Reply Last reply
                                        0
                                        • magiclike@soc.sekundenklebertransportverbot.deM magiclike@soc.sekundenklebertransportverbot.de shared this topic
                                        Reply
                                        • Reply as topic
                                        Log in to reply
                                        • Oldest to Newest
                                        • Newest to Oldest
                                        • Most Votes


                                        • Login

                                        • Login or register to search.
                                        Powered by NodeBB Contributors
                                        • First post
                                          Last post
                                        0
                                        • Categories
                                        • Recent
                                        • Tags
                                        • Popular
                                        • World
                                        • Users
                                        • Groups