[arvados] updated: 2.7.0-5388-g2e88c72ca4

git repository hosting git at public.arvados.org
Wed Nov 22 19:41:05 UTC 2023


Summary of changes:
 doc/admin/keep-faster-gc-s3.html.textile.liquid | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)

       via  2e88c72ca46a0912c8fb580ca3d63c4f57911c22 (commit)
      from  3fa9678a0a6caef757209f8666de66fb0896c953 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.


commit 2e88c72ca46a0912c8fb580ca3d63c4f57911c22
Author: Tom Clegg <tom at curii.com>
Date:   Wed Nov 22 14:39:55 2023 -0500

    21126: Update wording as suggested.
    
    https://dev.arvados.org/issues/21126#note-20
    
    Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom at curii.com>

diff --git a/doc/admin/keep-faster-gc-s3.html.textile.liquid b/doc/admin/keep-faster-gc-s3.html.textile.liquid
index 7f7cdd921f..2230ac2491 100644
--- a/doc/admin/keep-faster-gc-s3.html.textile.liquid
+++ b/doc/admin/keep-faster-gc-s3.html.textile.liquid
@@ -17,13 +17,25 @@ The multi-step trash/delete process can be short-circuited by setting @BlobTrash
 The following steps can be used to temporarily disable writes on an S3 bucket to enable faster garbage collection without data loss or service interruption. Note that garbage collection on other S3 volumes will be temporarily disabled during this procedure.
 # Create a new S3 bucket and configure it as an additional volume (this step may be skipped if the configuration already has enough writable volumes that clients will still be able to write blocks while the target volume is read-only). We recommend using @PrefixLength: 3@ for the new volume because this results in a much higher rate limit for I/O and garbage collection operations compared to the default @PrefixLength: 0 at . If the target volume configuration specifies @StorageClasses@, use the same values for the new volume.
 # Shut down the @keep-balance@ service.
-# Update the target volume configuration with @ReadOnly: true@, @AllowTrashWhileReadOnly: true@, and @UnsafeDelete: true at .
-# In the @Collections@ configuration section, set @BlobTrashLifetime: 0 at . Note this will effectively disable garbage collection on other S3-backed volumes, and garbage collection will delete blocks outright (bypassing the recoverable trash phase) on any non-S3 volumes.
-# In the @Collections@ configuration section, set @BalancePullLimit: 0 at .
+# Update your configuration as follows: <notextile><pre>
+  Collections:
+    BlobTrashLifetime: 0
+    BalancePullLimit: 0
+  [...]
+  Volumes:
+    <span class="userinput">target-volume-uuid</span>:
+      ReadOnly: true
+      AllowTrashWhileReadOnly: true
+      DriverParameters:
+        UnsafeDelete: true
+</pre></notextile> Note that @BlobTrashLifetime: 0@ instructs keepstore to delete unneeded blocks outright (bypassing the recoverable trash phase); however, in this mode it will normally not trash any blocks at all on an S3 volume due to the safety issue mentioned above, unless the volume is configured with @UnsafeDelete: true at .
 # Restart all @keepstore@ services with the updated configuration.
 # Start the @keep-balance@ service.
-# Garbage collection should now proceed faster on the target volume. Monitor progress by watching @keep-balance@ logs and metrics. When garbage collection is complete, keep-balance logs will show an empty changeset: <notextile><pre><code>zzzzz-bi6l4-0123456789abcdef (keep0.zzzzz.arvadosapi.com:25107, disk): ChangeSet{Pulls:0, Trashes:0}</code></pre></notextile>
-# Revert @BlobTrashLifetime@ to its previous value (or the default value, @336h@) and set @UnsafeDelete: false@ on the target volume.
-# Optionally, revert to @ReadOnly: false@ and @AllowTrashWhileReadOnly: false@ on the target volume and/or set @ReadOnly: true@ and @AllowTrashWhileReadOnly: true@ on the new volume, depending on where you intend to store new data in future. If the target volume has @PrefixLength: 0@ and the new volume has @PrefixLength: 3@, you can skip this step: new data will be stored on the new volume, some existing data will be moved automatically to other volumes, and some will be left on the target volume as long as it's needed.
+# Objects will be deleted immediately instead of being first copied to trash on the S3 volume, which should significantly speed up cleanup of trashed objects. Monitor progress by watching @keep-balance@ logs and metrics. When garbage collection is complete, keep-balance logs will show an empty changeset: <notextile><pre><code>zzzzz-bi6l4-0123456789abcdef (keep0.zzzzz.arvadosapi.com:25107, disk): ChangeSet{Pulls:0, Trashes:0}</code></pre></notextile>
+# Remove the @UnsafeDelete@ configuration entry on the target volume.
+# Remove the @BlobTrashLifetime@ configuration entry (or restore it to its previous value).
+# If the target volume has @PrefixLength: 0@ and the new volume has @PrefixLength: 3@, skip the next two steps: new data will be stored on the new volume, some existing data will be moved automatically to other volumes, and some will be left on the target volume as long as it's needed.
+# If you want to resume writing new data to the target volume, revert to @ReadOnly: false@ and @AllowTrashWhileReadOnly: false@ on the target volume.
+# If you want to stop writing new data to the newly created volume, set @ReadOnly: true@ and @AllowTrashWhileReadOnly: true@ on the new volume.
 # Remove the @BalancePullLimit@ configuration entry (or restore its previous value), and restart @keep-balance at .
 # Restart all @keepstore@ services with the updated configuration.

-----------------------------------------------------------------------


hooks/post-receive
-- 




More information about the arvados-commits mailing list