[arvados] created: 2.7.0-6066-ge0724a2454
git repository hosting
git at public.arvados.org
Mon Feb 26 16:58:09 UTC 2024
at e0724a24547a1e22377a4ec2fa077b7c3597dbfd (commit)
commit e0724a24547a1e22377a4ec2fa077b7c3597dbfd
Author: Brett Smith <brett.smith at curii.com>
Date: Mon Feb 26 11:57:33 2024 -0500
21504: Add a table of contents to the arv-mount reference
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>
diff --git a/doc/sdk/fuse/options.html.textile.liquid b/doc/sdk/fuse/options.html.textile.liquid
index 6f3aebdb35..1ebfa242a5 100644
--- a/doc/sdk/fuse/options.html.textile.liquid
+++ b/doc/sdk/fuse/options.html.textile.liquid
@@ -12,6 +12,18 @@ SPDX-License-Identifier: CC-BY-SA-3.0
This page documents all available @arv-mount@ options with some usage examples.
+# "Mount contents":#contents
+# "Mount custom layout and filtering":#layout
+## "@--filters@ usage and limitations":#filters
+# "Mount access and permissions":#access
+# "Mount lifecycle management":#lifecycle
+# "Mount logging and statistics":#logging
+# "Mount local cache setup":#cache
+# "Mount interactions with Arvados and Linux":#plumbing
+# "Examples":#examples
+## "Using @--exec@":#exec
+## "Running arv-mount as a systemd service":#systemd
+
h2(#contents). Mount contents
table(table table-bordered table-condensed).
commit f3cc90ee39862a7d455ddf15bca5da5966700adb
Author: Brett Smith <brett.smith at curii.com>
Date: Mon Feb 26 11:54:04 2024 -0500
21504: Add an example arv-mount systemd service definition
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>
diff --git a/doc/sdk/fuse/options.html.textile.liquid b/doc/sdk/fuse/options.html.textile.liquid
index c4f5a60826..6f3aebdb35 100644
--- a/doc/sdk/fuse/options.html.textile.liquid
+++ b/doc/sdk/fuse/options.html.textile.liquid
@@ -140,3 +140,42 @@ For example, this generates a recursive listing of all the projects and collecti
</notextile>
The first @ArvadosHome@ is a path argument to @find at . The second is the mount point argument to @arv-mount at .
+
+h3(#systemd). Running arv-mount as a systemd service
+
+If you want to run @arv-mount@ as a long-running service, it's easy to write a systemd service definition for it. We do not publish one because the entire definition tends to be site-specific, but you can start from this template. You must change the @ExecStart@ path. Comments detail other changes you might want to make.
+
+<notextile>
+<pre><code>[Unit]
+Description=Arvados FUSE mount
+Documentation={{ site.baseurl }}/sdk/fuse/options.html
+
+[Service]
+Type=simple
+CacheDirectory=arvados/keep
+CacheDirectoryMode=0700
+
+# This unit makes the mount available as `Arvados` under the runtime directory root.
+# If this is a system service installed under /etc/systemd/system,
+# the mount will be at /run/Arvados.
+# If this is a user service installed under ~/.config/systemd/user,
+# the mount will be at $XDG_RUNTIME_DIR/Arvados.
+# If you want to mount at another location on the filesystem, remove RuntimeDirectory
+# and replace both instances of %t/Arvados with your desired path.
+RuntimeDirectory=Arvados
+# The arv-mount path must be the absolute path where you installed the command.
+# If you installed from a distribution package, make this /usr/bin/arv-mount.
+# If you installed from pip, replace ... with the path to your virtualenv.
+# You can add options to select what gets mounted, access permissions,
+# cache size, log level, etc.
+ExecStart=<span class="userinput">...</span>/bin/arv-mount --foreground --disk-cache-dir %C/arvados/keep %t/Arvados
+ExecStop=/usr/bin/fusermount -u %t/Arvados
+
+# This unit assumes the running user has a ~/.config/arvados/settings.conf
+# with ARVADOS_API_HOST and ARVADOS_API_TOKEN defined.
+# If not, you can write those in a separate file
+# and set its path as EnvironmentFile.
+# Make sure that file is owned and only readable by the running user (mode 0600).
+#EnvironmentFile=...
+</code></pre>
+</notextile>
commit 4c33c80836296cd7f92b70a61844b02f350cdeb4
Author: Brett Smith <brett.smith at curii.com>
Date: Mon Feb 26 11:16:48 2024 -0500
21504: Add an `arv-mount --exec` example
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>
diff --git a/doc/sdk/fuse/options.html.textile.liquid b/doc/sdk/fuse/options.html.textile.liquid
index 0f16e3d688..c4f5a60826 100644
--- a/doc/sdk/fuse/options.html.textile.liquid
+++ b/doc/sdk/fuse/options.html.textile.liquid
@@ -122,3 +122,21 @@ table(table table-bordered table-condensed).
|@--encoding ENCODING@|Filesystem character encoding (default 'utf-8'; specify a name from the "Python codec registry":https://docs.python.org/3/library/codecs.html#standard-encodings)|
|@--retries RETRIES@|Maximum number of times to retry server requests that encounter temporary failures (e.g., server down). Default 10.|
|@--storage-classes CLASSES@|Comma-separated list of storage classes to request for new collections|
+
+h2(#examples). Examples
+
+h3(#exec). Using @--exec@
+
+There are a couple of details that are important to understand when you use @--exec@:
+
+* @--exec@ reads all remaining options as the command to run, so it must be the last option you specify. Either end your command arguments (and other options) with a @--@ argument, or specify @--exec@ after your mount point.
+* The command you specify runs from the same directory that you started @arv-mount@ from. To access data inside the mount, you will generally need to pass the path to the mount as an argument.
+
+For example, this generates a recursive listing of all the projects and collections under your home project:
+
+<notextile>
+<pre><code>$ <span class="userinput">arv-mount --home --exec find -type d ArvadosHome -- ArvadosHome</span>
+</code></pre>
+</notextile>
+
+The first @ArvadosHome@ is a path argument to @find at . The second is the mount point argument to @arv-mount at .
commit 4393e4edc2054e0d039517ddec507eaf339fb96f
Author: Brett Smith <brett.smith at curii.com>
Date: Mon Feb 26 11:11:32 2024 -0500
21504: Start arv-mount reference
Mostly generated from `arv-mount --help`. The filters section is new.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>
diff --git a/doc/_config.yml b/doc/_config.yml
index 2b78eadd42..053922a24a 100644
--- a/doc/_config.yml
+++ b/doc/_config.yml
@@ -90,6 +90,7 @@ navbar:
- sdk/cli/subcommands.html.textile.liquid
- FUSE Driver:
- sdk/fuse/install.html.textile.liquid
+ - sdk/fuse/options.html.textile.liquid
- Go:
- sdk/go/index.html.textile.liquid
- sdk/go/example.html.textile.liquid
diff --git a/doc/sdk/fuse/options.html.textile.liquid b/doc/sdk/fuse/options.html.textile.liquid
new file mode 100644
index 0000000000..0f16e3d688
--- /dev/null
+++ b/doc/sdk/fuse/options.html.textile.liquid
@@ -0,0 +1,124 @@
+---
+layout: default
+navsection: sdk
+navmenu: FUSE Driver
+title: arv-mount options
+...
+{% comment %}
+Copyright (C) The Arvados Authors. All rights reserved.
+
+SPDX-License-Identifier: CC-BY-SA-3.0
+{% endcomment %}
+
+This page documents all available @arv-mount@ options with some usage examples.
+
+h2(#contents). Mount contents
+
+table(table table-bordered table-condensed).
+|_. Option(s)|_. Description|
+|@--all@|Mount a subdirectory for each mode: @home@, @shared@, @by_id@, and @by_tag@ (default if no @--mount-*@ options are given)|
+|@--custom@|Mount a subdirectory for each mode specified by a @--mount-*@ option (default if any @--mount-*@ options are given; see "Mount custom layout and filtering":#layout section)|
+|@--collection UUID_OR_PDH@|Mount the specified collection|
+|@--home@|Mount your home project|
+|@--project UUID@|Mount the specified project|
+|@--shared@|Mount a subdirectory for each project shared with you|
+|@--by-id@|Mount a magic directory where collections and projects are accessible through subdirectories named after their UUID or portable data hash|
+|@--by-pdh@|Mount a magic directory where collections are accessible through subdirectories named after their portable data hash|
+|@--by-tag@|Mount a subdirectory for each tag attached to a collection or project|
+
+h2(#layout). Mount custom layout and filtering
+
+table(table table-bordered table-condensed).
+|_. Option(s)|_. Description|
+|@--filters FILTERS@|Filters to apply to all project, shared, and tag directory contents. Pass filters as either a JSON string or a path to a JSON file. The JSON object should be a list of filters in "Arvados API list filter syntax":{{ site.baseurl }}/api/methods.html#filters. See the "example filters":#filters.|
+|@--mount-home PATH@|Make your home project available under the mount at @PATH@|
+|@--mount-shared PATH@|Make projects shared with you available under the mount at @PATH@|
+|@--mount-tmp PATH@|Make a new temporary writable collection available under the mount at @PATH at . This collection is deleted when the mount is unmounted.|
+|@--mount-by-id PATH@|Make a magic directory available under the mount at @PATH@ where collections and projects are accessible through subdirectories named after their UUID or portable data hash|
+|@--mount-by-pdh PATH@|Make a magic directory available under the mount at @PATH@ where collections are accessible through subdirectories named after portable data hash|
+|@--mount-by-tag PATH@|Make a subdirectory for each tag attached to a collection or project available under the mount at @PATH@|
+
+h3(#filters). @--filters@ usage and limitations
+
+Your argument to @--filters@ should be a JSON list of filters in "Arvados API list filter syntax":{{ site.baseurl }}/api/methods.html#filters. If your filter checks any field besides @uuid@, you should prefix it with the @<resource type>.@ Taken together, here's an example that mounts your home directory excluding filter groups, workflow intermediate output collections, and workflow log collections:
+
+<notextile>
+<pre><code>$ arv-mount --home <span class="userinput">--filters '[["groups.group_class", "!=", "filter"], ["collections.properties.type", "not in", ["intermediate", "log"]]]'</span> ...
+</code></pre>
+</notextile>
+
+Because filters can be awkward to write on the command line, you can also write them in a file, and pass that file path to the @--filters@ option. This example does the same filtering:
+
+<notextile>
+<pre><code>$ <span class="userinput">cat >~/arv-mount-filters.json <<EOF
+[
+ [
+ "groups.group_class",
+ "!=",
+ "filter"
+ ],
+ [
+ "collections.properties.type",
+ "not in",
+ [
+ "intermediate",
+ "log"
+ ]
+ ]
+]
+EOF</span>
+$ arv-mount --home <span class="userinput">--filters ~/arv-mount-filters.json</span> ...
+</code></pre>
+</notextile>
+
+The current implementation of @--filters@ has a few limitations. These may be lifted in a future release:
+
+* You can always access any project or collection by UUID or portable data hash under a magic directory. If you access a project this way, your filters _will_ apply to the project contents.
+* Tag directory listings are generated by querying tags alone. Only filters that apply to @links@ will affect these listings.
+
+h2(#access). Mount access and permissions
+
+table(table table-bordered table-condensed).
+|_. Option(s)|_. Description|
+|@--allow-other@|Let other users on this system read mounted data (default false)|
+|@--read-only@|Mounted data cannot be modified from the mount (default)|
+|@--read-write@|Mounted data can be modified from the mount|
+
+h2(#lifecycle). Mount lifecycle management
+
+table(table table-bordered table-condensed).
+|_. Option(s)|_. Description|
+|@--exec ...@|Mount data, run the specified command, then unmount and exit. @--exec@ reads all remaining options as the command to run, so it must be the last option you specify. Either end your command arguments (and other options) with a @--@ argument, or specify @--exec@ after your mount point.|
+|@--foreground@|Run mount process in the foreground instead of daemonizing (default false)|
+|@--subtype SUBTYPE@|Set mounted filesystem type to @fuse.SUBTYPE@ (default is just @fuse@)|
+|@--replace@|If a FUSE mount is already mounted at the given directory, unmount it before mounting the requested data. If @--subtype@ is specified, unmount only if the mount has that subtype. WARNING: This command can affect any kind of FUSE mount, not just arv-mount.|
+|@--unmount@|If a FUSE mount is already mounted at the given directory, unmount it and exit. If @--subtype@ is specified, unmount only if the mount has that subtype. WARNING: This command can affect any kind of FUSE mount, not just arv-mount.|
+|@--unmount-all@|Unmount all FUSE mounts at or below the given directory, then exit. If @--subtype@ is specified, unmount only if the mount has that subtype. WARNING: This command can affect any kind of FUSE mount, not just arv-mount.|
+|@--unmount-timeout SECONDS@|The number of seconds to wait for a clean unmount after an @--exec@ command has exited (default 2.0). After this time, the mount will be forcefully unmounted.|
+
+h2(#logging). Mount logging and statistics
+
+table(table table-bordered table-condensed).
+|_. Option(s)|_. Description|
+|@--crunchstat-interval SECONDS@|Write stats to stderr every N seconds (default disabled)|
+|@--debug@|Log debug information|
+|@--logfile LOGFILE@|Write debug logs and errors to the specified file (default stderr)|
+
+h2(#cache). Mount local cache setup
+
+table(table table-bordered table-condensed).
+|_. Option(s)|_. Description|
+|@--disk-cache@|Cache data on the local filesystem (default)|
+|@--ram-cache@|Cache data in memory|
+|@--disk-cache-dir DIRECTORY@|Filesystem cache location (default @~/.cache/arvados/keep@)|
+|@--directory-cache BYTES@|Size of directory data cache in bytes (default 128 MiB)|
+|@--file-cache BYTES@|Size of file data cache in bytes (default 8 GiB for filesystem cache, 256 MiB for memory cache)|
+
+h2(#plumbing). Mount interactions with Arvados and Linux
+
+table(table table-bordered table-condensed).
+|_. Option(s)|_. Description|
+|@--disable-event-listening@|Don't subscribe to events on the API server to update mount contents|
+|@--encoding ENCODING@|Filesystem character encoding (default 'utf-8'; specify a name from the "Python codec registry":https://docs.python.org/3/library/codecs.html#standard-encodings)|
+|@--retries RETRIES@|Maximum number of times to retry server requests that encounter temporary failures (e.g., server down). Default 10.|
+|@--storage-classes CLASSES@|Comma-separated list of storage classes to request for new collections|
commit 4a1d81633cd0633d70cd2d250f82776062c30a5a
Author: Brett Smith <brett.smith at curii.com>
Date: Mon Feb 26 10:39:29 2024 -0500
21504: Consistently use backtick markup in arv-mount help
Mostly to make it easier to generate reference documentation, but it's
nice as-is too.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>
diff --git a/services/fuse/arvados_fuse/command.py b/services/fuse/arvados_fuse/command.py
index 53e8c26fc7..719ec7ee95 100644
--- a/services/fuse/arvados_fuse/command.py
+++ b/services/fuse/arvados_fuse/command.py
@@ -50,8 +50,8 @@ class ArgumentParser(argparse.ArgumentParser):
const='all',
dest='mode',
help="""
-Mount a subdirectory for each mode: home, shared, by_id, and by_tag
-(default if no --mount-* options are given)
+Mount a subdirectory for each mode: `home`, `shared`, `by_id`, and `by_tag`
+(default if no `--mount-*` options are given)
""",
)
mode.add_argument(
@@ -60,8 +60,8 @@ Mount a subdirectory for each mode: home, shared, by_id, and by_tag
const=None,
dest='mode',
help="""
-Mount a subdirectory for each mode specified by a --mount-* option
-(default if any --mount-* options are given;
+Mount a subdirectory for each mode specified by a `--mount-*` option
+(default if any `--mount-*` options are given;
see "Mount custom layout and filtering" section)
""",
)
@@ -132,14 +132,14 @@ The JSON object should be a list of filters in Arvados API list filter syntax.
metavar='PATH',
action='append',
default=[],
- help="Make your home project available under the mount at PATH",
+ help="Make your home project available under the mount at `PATH`",
)
mounts.add_argument(
'--mount-shared',
metavar='PATH',
action='append',
default=[],
- help="Make projects shared with you available under the mount at PATH",
+ help="Make projects shared with you available under the mount at `PATH`",
)
mounts.add_argument(
'--mount-tmp',
@@ -147,7 +147,7 @@ The JSON object should be a list of filters in Arvados API list filter syntax.
action='append',
default=[],
help="""
-Make a new temporary writable collection available under the mount at PATH.
+Make a new temporary writable collection available under the mount at `PATH`.
This collection is deleted when the mount is unmounted.
""",
)
@@ -157,7 +157,7 @@ This collection is deleted when the mount is unmounted.
action='append',
default=[],
help="""
-Make a magic directory available under the mount at PATH where collections and
+Make a magic directory available under the mount at `PATH` where collections and
projects are accessible through subdirectories named after their UUID or
portable data hash
""",
@@ -168,7 +168,7 @@ portable data hash
action='append',
default=[],
help="""
-Make a magic directory available under the mount at PATH where collections
+Make a magic directory available under the mount at `PATH` where collections
are accessible through subdirectories named after portable data hash
""",
)
@@ -179,7 +179,7 @@ are accessible through subdirectories named after portable data hash
default=[],
help="""
Make a subdirectory for each tag attached to a collection or project available
-under the mount at PATH
+under the mount at `PATH`
""" ,
)
@@ -211,10 +211,10 @@ under the mount at PATH
dest="exec_args",
help="""
Mount data, run the specified command, then unmount and exit.
---exec reads all remaining options as the command to run,
+`--exec` reads all remaining options as the command to run,
so it must be the last option you specify.
Either end your command arguments (and other options) with a `--` argument,
-or specify --exec after your mount point.
+or specify `--exec` after your mount point.
""",
)
lifecycle.add_argument(
@@ -235,7 +235,7 @@ or specify --exec after your mount point.
help="""
If a FUSE mount is already mounted at the given directory,
unmount it before mounting the requested data.
-If --subtype is specified, unmount only if the mount has that subtype.
+If `--subtype` is specified, unmount only if the mount has that subtype.
WARNING: This command can affect any kind of FUSE mount, not just arv-mount.
""",
)
@@ -245,7 +245,7 @@ WARNING: This command can affect any kind of FUSE mount, not just arv-mount.
default=False,
help="""
If a FUSE mount is already mounted at the given directory, unmount it and exit.
-If --subtype is specified, unmount only if the mount has that subtype.
+If `--subtype` is specified, unmount only if the mount has that subtype.
WARNING: This command can affect any kind of FUSE mount, not just arv-mount.
""",
)
@@ -255,7 +255,7 @@ WARNING: This command can affect any kind of FUSE mount, not just arv-mount.
default=False,
help="""
Unmount all FUSE mounts at or below the given directory, then exit.
-If --subtype is specified, unmount only if the mount has that subtype.
+If `--subtype` is specified, unmount only if the mount has that subtype.
WARNING: This command can affect any kind of FUSE mount, not just arv-mount.
""",
)
@@ -265,7 +265,7 @@ WARNING: This command can affect any kind of FUSE mount, not just arv-mount.
default=2.0,
metavar='SECONDS',
help="""
-The number of seconds to wait for a clean unmount after an --exec command has
+The number of seconds to wait for a clean unmount after an `--exec` command has
exited (default %(default).01f).
After this time, the mount will be forcefully unmounted.
""",
@@ -308,7 +308,7 @@ After this time, the mount will be forcefully unmounted.
cache.add_argument(
'--disk-cache-dir',
metavar="DIRECTORY",
- help="Filesystem cache location (default ~/.cache/arvados/keep)",
+ help="Filesystem cache location (default `~/.cache/arvados/keep`)",
)
cache.add_argument(
'--directory-cache',
commit 7e9d451769e84b18faf05dd93caf91c440ba8308
Author: Brett Smith <brett.smith at curii.com>
Date: Mon Feb 26 10:32:31 2024 -0500
21504: Reorder arv-mount --mount-* options
For consistency with the previous section.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>
diff --git a/services/fuse/arvados_fuse/command.py b/services/fuse/arvados_fuse/command.py
index c12e45255b..53e8c26fc7 100644
--- a/services/fuse/arvados_fuse/command.py
+++ b/services/fuse/arvados_fuse/command.py
@@ -134,6 +134,23 @@ The JSON object should be a list of filters in Arvados API list filter syntax.
default=[],
help="Make your home project available under the mount at PATH",
)
+ mounts.add_argument(
+ '--mount-shared',
+ metavar='PATH',
+ action='append',
+ default=[],
+ help="Make projects shared with you available under the mount at PATH",
+ )
+ mounts.add_argument(
+ '--mount-tmp',
+ metavar='PATH',
+ action='append',
+ default=[],
+ help="""
+Make a new temporary writable collection available under the mount at PATH.
+This collection is deleted when the mount is unmounted.
+""",
+ )
mounts.add_argument(
'--mount-by-id',
metavar='PATH',
@@ -155,13 +172,6 @@ Make a magic directory available under the mount at PATH where collections
are accessible through subdirectories named after portable data hash
""",
)
- mounts.add_argument(
- '--mount-shared',
- metavar='PATH',
- action='append',
- default=[],
- help="Make projects shared with you available under the mount at PATH",
- )
mounts.add_argument(
'--mount-by-tag',
metavar='PATH',
@@ -172,16 +182,6 @@ Make a subdirectory for each tag attached to a collection or project available
under the mount at PATH
""" ,
)
- mounts.add_argument(
- '--mount-tmp',
- metavar='PATH',
- action='append',
- default=[],
- help="""
-Make a new temporary writable collection available under the mount at PATH.
-This collection is deleted when the mount is unmounted.
-""",
- )
perms = self.add_argument_group("Mount access and permissions")
perms.add_argument(
commit fd6b6e7a917d414d76f0bd5b2550c85fd8d2eec9
Author: Brett Smith <brett.smith at curii.com>
Date: Mon Feb 26 10:25:05 2024 -0500
21504: Break out FUSE section of SDK reference
This includes modernizing the index:
* Fix the distro package name
* Update the list of C dependencies
* Copyediting throughout
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>
diff --git a/doc/_config.yml b/doc/_config.yml
index 2bc346759a..2b78eadd42 100644
--- a/doc/_config.yml
+++ b/doc/_config.yml
@@ -81,7 +81,6 @@ navbar:
- sdk/python/api-client.html.textile.liquid
- sdk/python/cookbook.html.textile.liquid
- sdk/python/python.html.textile.liquid
- - sdk/python/arvados-fuse.html.textile.liquid
- sdk/python/arvados-cwl-runner.html.textile.liquid
- sdk/python/events.html.textile.liquid
- Command line tools (CLI SDK):
@@ -89,6 +88,8 @@ navbar:
- sdk/cli/index.html.textile.liquid
- sdk/cli/reference.html.textile.liquid
- sdk/cli/subcommands.html.textile.liquid
+ - FUSE Driver:
+ - sdk/fuse/install.html.textile.liquid
- Go:
- sdk/go/index.html.textile.liquid
- sdk/go/example.html.textile.liquid
diff --git a/doc/install/install-shell-server.html.textile.liquid b/doc/install/install-shell-server.html.textile.liquid
index 57b79d2042..f864f37563 100644
--- a/doc/install/install-shell-server.html.textile.liquid
+++ b/doc/install/install-shell-server.html.textile.liquid
@@ -35,7 +35,7 @@ h2(#dependencies). Install Dependencies and SDKs
# "Install Ruby and Bundler":ruby.html
# "Install the Python SDK":{{site.baseurl}}/sdk/python/sdk-python.html
-# "Install the FUSE driver":{{site.baseurl}}/sdk/python/arvados-fuse.html
+# "Install the FUSE driver":{{site.baseurl}}/sdk/fuse/install.html
# "Install the CLI":{{site.baseurl}}/sdk/cli/install.html
# "Install the R SDK":{{site.baseurl}}/sdk/R/index.html (optional)
# "Install Docker":install-docker.html (optional)
diff --git a/doc/sdk/fuse/install.html.textile.liquid b/doc/sdk/fuse/install.html.textile.liquid
new file mode 100644
index 0000000000..52ffb2bbd1
--- /dev/null
+++ b/doc/sdk/fuse/install.html.textile.liquid
@@ -0,0 +1,42 @@
+---
+layout: default
+navsection: sdk
+navmenu: FUSE Driver
+title: Installing the FUSE Driver
+...
+{% comment %}
+Copyright (C) The Arvados Authors. All rights reserved.
+
+SPDX-License-Identifier: CC-BY-SA-3.0
+{% endcomment %}
+
+The Arvados FUSE driver is a Python utility that allows you to browse Arvados projects and collections in a filesystem, so you can access that data using existing Unix tools.
+
+h2. Installation
+
+If you are logged in to a managed Arvados VM, the @arv-mount@ utility should already be installed.
+
+To use the FUSE driver elsewhere, you can install from a distribution package or pip.
+
+h2. Option 1: Install from distribution packages
+
+First, "add the appropriate package repository for your distribution":{{ site.baseurl }}/install/packages.html.
+
+{% assign arvados_component = 'python3-arvados-fuse' %}
+
+{% include 'install_packages' %}
+
+h2. Option 2: Install with pip
+
+Run @pip install arvados_fuse@ in an appropriate installation environment, such as a virtualenv.
+
+Note: The FUSE driver depends on the @libcurl@ and @libfuse@ C libraries. To install the module you may need to install development headers from your distribution. On Debian-based distributions you can install them by running:
+
+<notextile>
+<pre><code># <span class="userinput">apt install build-essential python3-dev libcurl4-openssl-dev libfuse-dev libssl-dev</span>
+</code></pre>
+</notextile>
+
+h2. Usage
+
+For an introduction of how to mount and navigate data, refer to the "Access Keep as a GNU/Linux filesystem":{{site.baseurl}}/user/tutorials/tutorial-keep-mount-gnu-linux.html tutorial.
diff --git a/doc/sdk/python/arvados-fuse.html.textile.liquid b/doc/sdk/python/arvados-fuse.html.textile.liquid
deleted file mode 100644
index 254294a4d1..0000000000
--- a/doc/sdk/python/arvados-fuse.html.textile.liquid
+++ /dev/null
@@ -1,44 +0,0 @@
----
-layout: default
-navsection: sdk
-navmenu: Python
-title: Arvados FUSE driver
-...
-{% comment %}
-Copyright (C) The Arvados Authors. All rights reserved.
-
-SPDX-License-Identifier: CC-BY-SA-3.0
-{% endcomment %}
-
-The Arvados FUSE driver is a Python utility that allows you to see the Keep service as a normal filesystem, so that data can be accessed using standard tools. This driver requires the Python SDK installed in order to access Arvados services.
-
-h2. Installation
-
-If you are logged in to a managed Arvados VM, the @arv-mount@ utility should already be installed.
-
-To use the FUSE driver elsewhere, you can install from a distribution package, or PyPI.
-
-h2. Option 1: Install from distribution packages
-
-First, "add the appropriate package repository for your distribution":{{ site.baseurl }}/install/packages.html
-
-{% assign arvados_component = 'python-arvados-fuse' %}
-
-{% include 'install_packages' %}
-
-h2. Option 2: Install with pip
-
-Run @pip install arvados_fuse@ in an appropriate installation environment, such as a virtualenv.
-
-Note:
-
-The FUSE driver uses @pycurl@ which depends on the @libcurl@ C library. To build the module you may have to first install additional packages. On Debian-based distributions you can install them by running:
-
-<notextile>
-<pre><code># <span class="userinput">apt install git build-essential python3-dev libcurl4-openssl-dev libssl-dev</span>
-</code></pre>
-</notextile>
-
-h3. Usage
-
-Please refer to the "Accessing Keep from GNU/Linux":{{site.baseurl}}/user/tutorials/tutorial-keep-mount-gnu-linux.html tutorial for more information.
diff --git a/doc/user/getting_started/setup-cli.html.textile.liquid b/doc/user/getting_started/setup-cli.html.textile.liquid
index 999f848c13..18f675d04e 100644
--- a/doc/user/getting_started/setup-cli.html.textile.liquid
+++ b/doc/user/getting_started/setup-cli.html.textile.liquid
@@ -35,7 +35,7 @@ Here are the client packages you can install on your system. You can skip any yo
* "Python SDK":{{site.baseurl}}/sdk/python/sdk-python.html: This provides an Arvados API client in Python, as well as low-level command line tools.
* "Command-line SDK":{{site.baseurl}}/sdk/cli/install.html: This provides the high-level @arv@ command and user interface to the Arvados API.
-* "FUSE Driver":{{site.baseurl}}/sdk/python/arvados-fuse.html: This provides the @arv-mount@ command and FUSE driver that lets you access Keep using standard Linux filesystem tools.
+* "FUSE Driver":{{site.baseurl}}/sdk/fuse/install.html: This provides the @arv-mount@ command and FUSE driver that lets you access Keep using standard Linux filesystem tools.
* "CWL Runner":{{site.baseurl}}/sdk/python/arvados-cwl-runner.html: This provides the @arvados-cwl-runner@ command to register and run workflows in Crunch.
* "crunchstat-summary":{{site.baseurl}}/user/cwl/crunchstat-summary.html: This tool provides performance reports for Crunch containers.
* "arvados-client":{{site.baseurl}}/user/debugging/container-shell-access.html: This tool provides subcommands for inspecting Crunch containers, both interactively while they're running and after they've finished.
commit 08297c0769ab936d55ae37a216ec65d66cbd06e1
Author: Brett Smith <brett.smith at curii.com>
Date: Mon Feb 26 10:04:26 2024 -0500
21504: Modernize SDK documentation tree
* Spell out "CLI"
* Remove long-gone page listing from CLI section
* Ditch the "v2" moniker from the Java SDK
* Alphabetize the less-mature SDKs
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>
diff --git a/doc/_config.yml b/doc/_config.yml
index ad2864ca3a..2bc346759a 100644
--- a/doc/_config.yml
+++ b/doc/_config.yml
@@ -84,25 +84,24 @@ navbar:
- sdk/python/arvados-fuse.html.textile.liquid
- sdk/python/arvados-cwl-runner.html.textile.liquid
- sdk/python/events.html.textile.liquid
- - CLI:
+ - Command line tools (CLI SDK):
- sdk/cli/install.html.textile.liquid
- sdk/cli/index.html.textile.liquid
- sdk/cli/reference.html.textile.liquid
- sdk/cli/subcommands.html.textile.liquid
- - sdk/cli/project-management.html.textile.liquid
- Go:
- sdk/go/index.html.textile.liquid
- sdk/go/example.html.textile.liquid
+ - Java:
+ - sdk/java-v2/index.html.textile.liquid
+ - sdk/java-v2/example.html.textile.liquid
+ - sdk/java-v2/javadoc.html.textile.liquid
- R:
- sdk/R/index.html.md
- sdk/R/arvados/index.html.textile.liquid
- Ruby:
- sdk/ruby/index.html.textile.liquid
- sdk/ruby/example.html.textile.liquid
- - Java v2:
- - sdk/java-v2/index.html.textile.liquid
- - sdk/java-v2/example.html.textile.liquid
- - sdk/java-v2/javadoc.html.textile.liquid
api:
- Concepts:
- api/index.html.textile.liquid
diff --git a/doc/sdk/cli/index.html.textile.liquid b/doc/sdk/cli/index.html.textile.liquid
index 511a41e0b8..ea10c830bc 100644
--- a/doc/sdk/cli/index.html.textile.liquid
+++ b/doc/sdk/cli/index.html.textile.liquid
@@ -1,7 +1,7 @@
---
layout: default
navsection: sdk
-navmenu: CLI
+navmenu: Command line tools (CLI SDK)
title: "Overview"
...
diff --git a/doc/sdk/cli/install.html.textile.liquid b/doc/sdk/cli/install.html.textile.liquid
index 9657d236ad..e0d50b874b 100644
--- a/doc/sdk/cli/install.html.textile.liquid
+++ b/doc/sdk/cli/install.html.textile.liquid
@@ -1,7 +1,7 @@
---
layout: default
navsection: sdk
-navmenu: CLI
+navmenu: Command line tools (CLI SDK)
title: "Installation"
...
{% comment %}
diff --git a/doc/sdk/cli/reference.html.textile.liquid b/doc/sdk/cli/reference.html.textile.liquid
index 735ba5ca87..307fecd9a0 100644
--- a/doc/sdk/cli/reference.html.textile.liquid
+++ b/doc/sdk/cli/reference.html.textile.liquid
@@ -1,7 +1,7 @@
---
layout: default
navsection: sdk
-navmenu: CLI
+navmenu: Command line tools (CLI SDK)
title: "arv reference"
...
{% comment %}
diff --git a/doc/sdk/cli/subcommands.html.textile.liquid b/doc/sdk/cli/subcommands.html.textile.liquid
index 5dda77ab5e..dadb1d56c7 100644
--- a/doc/sdk/cli/subcommands.html.textile.liquid
+++ b/doc/sdk/cli/subcommands.html.textile.liquid
@@ -1,7 +1,7 @@
---
layout: default
navsection: sdk
-navmenu: CLI
+navmenu: Command line tools (CLI SDK)
title: "arv subcommands"
...
diff --git a/doc/sdk/java-v2/example.html.textile.liquid b/doc/sdk/java-v2/example.html.textile.liquid
index 8d2fc2f4af..a0841ec432 100644
--- a/doc/sdk/java-v2/example.html.textile.liquid
+++ b/doc/sdk/java-v2/example.html.textile.liquid
@@ -1,7 +1,7 @@
---
layout: default
navsection: sdk
-navmenu: Java SDK v2
+navmenu: Java
title: Examples
...
{% comment %}
diff --git a/doc/sdk/java-v2/index.html.textile.liquid b/doc/sdk/java-v2/index.html.textile.liquid
index ad9f0e1a9d..aca9c48078 100644
--- a/doc/sdk/java-v2/index.html.textile.liquid
+++ b/doc/sdk/java-v2/index.html.textile.liquid
@@ -1,7 +1,7 @@
---
layout: default
navsection: sdk
-navmenu: Java SDK v2
+navmenu: Java
title: "Installation"
...
{% comment %}
diff --git a/doc/sdk/java-v2/javadoc.html.textile.liquid b/doc/sdk/java-v2/javadoc.html.textile.liquid
index 872150f625..686cd2440f 100644
--- a/doc/sdk/java-v2/javadoc.html.textile.liquid
+++ b/doc/sdk/java-v2/javadoc.html.textile.liquid
@@ -1,7 +1,7 @@
---
layout: default
navsection: sdk
-navmenu: Java v2
+navmenu: Java
title: "Javadoc Reference"
no_nav_left: true
commit 164fe5d1d26297dd767dd2fd633b5bc448eab8f6
Author: Brett Smith <brett.smith at curii.com>
Date: Mon Feb 26 09:42:23 2024 -0500
21504: Modernize the SDK index
* Note these packages also include client tools.
* Highlight more mature SDKs. Attach caveats to the less mature ones.
* Update Workbench text for v2.
This is preparation to list most client tools here, and generally turn
the "SDK reference" into a "User reference."
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>
diff --git a/doc/sdk/index.html.textile.liquid b/doc/sdk/index.html.textile.liquid
index b733d03bfc..9abfa9789f 100644
--- a/doc/sdk/index.html.textile.liquid
+++ b/doc/sdk/index.html.textile.liquid
@@ -9,13 +9,18 @@ Copyright (C) The Arvados Authors. All rights reserved.
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-This section documents language bindings for the "Arvados API":{{site.baseurl}}/api/index.html and Keep that are available for various programming languages. Not all features are available in every SDK. The most complete SDK is the Python SDK. Note that this section only gives a high level overview of each SDK. Consult the "Arvados API":{{site.baseurl}}/api/index.html section for detailed documentation about Arvados API calls available on each resource.
+This section documents client tools and language bindings for the "Arvados API":{{site.baseurl}}/api/index.html and Keep that are available for various programming languages. The most mature, popular packages are:
+
+* "Python SDK":{{site.baseurl}}/sdk/python/sdk-python.html (also includes essential command line tools such as @arv-put@ and @arv-get@)
+* "Command line SDK":{{site.baseurl}}/sdk/cli/install.html (includes the @arv@ tool)
+
+Many Arvados Workbench pages provide examples of using the Python SDK and command line tools to access a given resource. Open "API details" from the action menu and open the tab with the example you're interested in.
+
+We provide API bindings for several other languages, but these SDKs may be missing some features or documentation:
-* "Python SDK":{{site.baseurl}}/sdk/python/sdk-python.html (also includes essential command line tools such as "arv-put" and "arv-get")
-* "Command line SDK":{{site.baseurl}}/sdk/cli/install.html ("arv")
* "Go SDK":{{site.baseurl}}/sdk/go/index.html
+* "Java SDK":{{site.baseurl}}/sdk/java-v2/index.html
* "R SDK":{{site.baseurl}}/sdk/R/index.html
* "Ruby SDK":{{site.baseurl}}/sdk/ruby/index.html
-* "Java SDK v2":{{site.baseurl}}/sdk/java-v2/index.html
-Many Arvados Workbench pages, under the *Advanced* tab, provide examples of API and SDK use for accessing the current resource .
+Consult the "Arvados API":{{site.baseurl}}/api/index.html section for detailed documentation about Arvados API calls available on each resource.
commit 404d326a66c0f83e2278a2cc012b4b6b87de6459
Author: Brett Smith <brett.smith at curii.com>
Date: Sun Feb 25 17:00:48 2024 -0500
21504: Rewrite arv-mount help for more consistent style
Some key points:
* Prefer describing the effect with a phrase over a sentence
* Only use periods when there are multiple sentences
* Defaults in parentheses
* Consistent text where needed (particularly across --by-* and
--mount-by-* options and different --unmount options)
We don't really have a style guide for documentation like this. I'm not
trying to establish one by fiat. I'd be open to discussing basically all
these points. But until that discussion happens, consistency is
valuable, so I'm using the rules that I tend to follow naturally.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>
diff --git a/services/fuse/arvados_fuse/command.py b/services/fuse/arvados_fuse/command.py
index cc00321cd0..c12e45255b 100644
--- a/services/fuse/arvados_fuse/command.py
+++ b/services/fuse/arvados_fuse/command.py
@@ -34,12 +34,12 @@ class ArgumentParser(argparse.ArgumentParser):
'--version',
action='version',
version=u"%s %s" % (sys.argv[0], __version__),
- help='Print version and exit.',
+ help="Print version and exit",
)
self.add_argument(
'mountpoint',
metavar='MOUNT_DIR',
- help="""Mount point.""",
+ help="Directory path to mount data",
)
mode_group = self.add_argument_group("Mount contents")
@@ -50,8 +50,8 @@ class ArgumentParser(argparse.ArgumentParser):
const='all',
dest='mode',
help="""
-Mount a subdirectory for each mode: home, shared, by_tag, by_id
-(default if no --mount-* arguments are given).
+Mount a subdirectory for each mode: home, shared, by_id, and by_tag
+(default if no --mount-* options are given)
""",
)
mode.add_argument(
@@ -60,54 +60,61 @@ Mount a subdirectory for each mode: home, shared, by_tag, by_id
const=None,
dest='mode',
help="""
-Mount a top level meta-directory with subdirectories as specified by additional --mount-* arguments
-(default if any --mount-* arguments are given).
+Mount a subdirectory for each mode specified by a --mount-* option
+(default if any --mount-* options are given;
+see "Mount custom layout and filtering" section)
""",
)
mode.add_argument(
'--collection',
metavar='UUID_OR_PDH',
- help="""Mount only the specified collection.""",
+ help="Mount the specified collection",
)
mode.add_argument(
'--home',
action='store_const',
const='home',
dest='mode',
- help="""Mount only the user's home project.""",
+ help="Mount your home project",
)
mode.add_argument(
'--project',
metavar='UUID',
- help="""Mount the specified project.""",
+ help="Mount the specified project",
)
mode.add_argument(
'--shared',
action='store_const',
const='shared',
dest='mode',
- help="""Mount only list of projects shared with the user.""",
+ help="Mount a subdirectory for each project shared with you",
)
mode.add_argument(
'--by-id',
action='store_const',
const='by_id',
dest='mode',
- help="""Mount subdirectories listed by portable data hash or uuid.""",
+ help="""
+Mount a magic directory where collections and projects are accessible through
+subdirectories named after their UUID or portable data hash
+""",
)
mode.add_argument(
'--by-pdh',
action='store_const',
const='by_pdh',
dest='mode',
- help="""Mount subdirectories listed by portable data hash.""",
+ help="""
+Mount a magic directory where collections are accessible through
+subdirectories named after their portable data hash
+""",
)
mode.add_argument(
'--by-tag',
action='store_const',
const='by_tag',
dest='mode',
- help="""Mount subdirectories listed by tag.""",
+ help="Mount a subdirectory for each tag attached to a collection or project",
)
mounts = self.add_argument_group("Mount custom layout and filtering")
@@ -125,7 +132,7 @@ The JSON object should be a list of filters in Arvados API list filter syntax.
metavar='PATH',
action='append',
default=[],
- help="Mount the current user's home project at mountpoint/PATH.",
+ help="Make your home project available under the mount at PATH",
)
mounts.add_argument(
'--mount-by-id',
@@ -133,8 +140,9 @@ The JSON object should be a list of filters in Arvados API list filter syntax.
action='append',
default=[],
help="""
-Mount each readable collection at mountpoint/PATH/UUID and mountpoint/PATH/PDH
-where PDH is the collection's portable data hash and UUID is its UUID.
+Make a magic directory available under the mount at PATH where collections and
+projects are accessible through subdirectories named after their UUID or
+portable data hash
""",
)
mounts.add_argument(
@@ -143,8 +151,8 @@ where PDH is the collection's portable data hash and UUID is its UUID.
action='append',
default=[],
help="""
-Mount each readable collection at mountpoint/PATH/P
-where P is the collection's portable data hash.
+Make a magic directory available under the mount at PATH where collections
+are accessible through subdirectories named after portable data hash
""",
)
mounts.add_argument(
@@ -152,14 +160,17 @@ where P is the collection's portable data hash.
metavar='PATH',
action='append',
default=[],
- help="Mount projects shared with the current user at mountpoint/PATH.",
+ help="Make projects shared with you available under the mount at PATH",
)
mounts.add_argument(
'--mount-by-tag',
metavar='PATH',
action='append',
default=[],
- help="Mount all collections with tag TAG at mountpoint/PATH/TAG/UUID.",
+ help="""
+Make a subdirectory for each tag attached to a collection or project available
+under the mount at PATH
+""" ,
)
mounts.add_argument(
'--mount-tmp',
@@ -167,8 +178,8 @@ where P is the collection's portable data hash.
action='append',
default=[],
help="""
-Create a new collection, mount it in read/write mode at mountpoint/PATH,
-and delete it when unmounting.
+Make a new temporary writable collection available under the mount at PATH.
+This collection is deleted when the mount is unmounted.
""",
)
@@ -176,21 +187,21 @@ and delete it when unmounting.
perms.add_argument(
'--allow-other',
action='store_true',
- help="""Let other users read the mount""",
+ help="Let other users on this system read mounted data (default false)",
)
perms.add_argument(
'--read-only',
action='store_false',
default=False,
dest='enable_write',
- help="Mount will be read only (default)",
+ help="Mounted data cannot be modified from the mount (default)",
)
perms.add_argument(
'--read-write',
action='store_true',
default=False,
dest='enable_write',
- help="Mount will be read-write",
+ help="Mounted data can be modified from the mount",
)
lifecycle = self.add_argument_group("Mount lifecycle management")
@@ -210,11 +221,11 @@ or specify --exec after your mount point.
'--foreground',
action='store_true',
default=False,
- help="""Run in foreground (default is to daemonize unless --exec specified)""",
+ help="Run mount process in the foreground instead of daemonizing (default false)",
)
lifecycle.add_argument(
'--subtype',
- help="""Report mounted filesystem type as "fuse.SUBTYPE", instead of just "fuse".""",
+ help="Set mounted filesystem type to `fuse.SUBTYPE` (default is just `fuse`)",
)
unmount = lifecycle.add_mutually_exclusive_group()
unmount.add_argument(
@@ -222,7 +233,10 @@ or specify --exec after your mount point.
action='store_true',
default=False,
help="""
-If a fuse mount is already present at mountpoint, forcefully unmount it before mounting
+If a FUSE mount is already mounted at the given directory,
+unmount it before mounting the requested data.
+If --subtype is specified, unmount only if the mount has that subtype.
+WARNING: This command can affect any kind of FUSE mount, not just arv-mount.
""",
)
unmount.add_argument(
@@ -230,9 +244,9 @@ If a fuse mount is already present at mountpoint, forcefully unmount it before m
action='store_true',
default=False,
help="""
-Forcefully unmount the specified mountpoint (if it's a fuse mount) and exit.
-If --subtype is given, unmount only if the mount has the specified subtype.
-WARNING: This command can affect any kind of fuse mount, not just arv-mount.
+If a FUSE mount is already mounted at the given directory, unmount it and exit.
+If --subtype is specified, unmount only if the mount has that subtype.
+WARNING: This command can affect any kind of FUSE mount, not just arv-mount.
""",
)
unmount.add_argument(
@@ -240,10 +254,9 @@ WARNING: This command can affect any kind of fuse mount, not just arv-mount.
action='store_true',
default=False,
help="""
-Forcefully unmount every fuse mount at or below the specified path and exit.
-If --subtype is given, unmount only mounts that have the specified subtype.
-Exit non-zero if any other types of mounts are found at or below the given path.
-WARNING: This command can affect any kind of fuse mount, not just arv-mount.
+Unmount all FUSE mounts at or below the given directory, then exit.
+If --subtype is specified, unmount only if the mount has that subtype.
+WARNING: This command can affect any kind of FUSE mount, not just arv-mount.
""",
)
lifecycle.add_argument(
@@ -252,7 +265,9 @@ WARNING: This command can affect any kind of fuse mount, not just arv-mount.
default=2.0,
metavar='SECONDS',
help="""
-Time to wait for graceful shutdown after --exec program exits and filesystem is unmounted
+The number of seconds to wait for a clean unmount after an --exec command has
+exited (default %(default).01f).
+After this time, the mount will be forcefully unmounted.
""",
)
@@ -267,11 +282,11 @@ Time to wait for graceful shutdown after --exec program exits and filesystem is
reporting.add_argument(
'--debug',
action='store_true',
- help="""Debug mode""",
+ help="Log debug information",
)
reporting.add_argument(
'--logfile',
- help="""Write debug logs and errors to the specified file (default stderr).""",
+ help="Write debug logs and errors to the specified file (default stderr)",
)
cache = self.add_argument_group("Mount local cache setup")
@@ -281,26 +296,26 @@ Time to wait for graceful shutdown after --exec program exits and filesystem is
action='store_true',
default=True,
dest='disk_cache',
- help="Use disk based caching (default)",
+ help="Cache data on the local filesystem (default)",
)
cachetype.add_argument(
'--ram-cache',
action='store_false',
default=True,
dest='disk_cache',
- help="Use in-memory caching only",
+ help="Cache data in memory",
)
cache.add_argument(
'--disk-cache-dir',
metavar="DIRECTORY",
- help="Disk cache location (default ~/.cache/arvados/keep)",
+ help="Filesystem cache location (default ~/.cache/arvados/keep)",
)
cache.add_argument(
'--directory-cache',
type=int,
default=128*1024*1024,
metavar='BYTES',
- help="Directory data cache size, in bytes (default 128 MiB)",
+ help="Size of directory data cache in bytes (default 128 MiB)",
)
cache.add_argument(
'--file-cache',
@@ -308,8 +323,8 @@ Time to wait for graceful shutdown after --exec program exits and filesystem is
default=0,
metavar='BYTES',
help="""
-File data cache size, in bytes
-(default 8 GiB for disk-based cache or 256 MiB with RAM-only cache)
+Size of file data cache in bytes
+(default 8 GiB for filesystem cache, 256 MiB for memory cache)
""",
)
@@ -319,20 +334,20 @@ File data cache size, in bytes
action='store_true',
dest='disable_event_listening',
default=False,
- help="Don't subscribe to events on the API server",
+ help="Don't subscribe to events on the API server to update mount contents",
)
plumbing.add_argument(
'--encoding',
default="utf-8",
help="""
-Character encoding to use for filesystem, default is utf-8
-(see Python codec registry for list of available encodings)
+Filesystem character encoding
+(default %(default)r; specify a name from the Python codec registry)
""",
)
plumbing.add_argument(
'--storage-classes',
metavar='CLASSES',
- help="Specify comma separated list of storage classes to be used when saving data of new collections",
+ help="Comma-separated list of storage classes to request for new collections",
)
commit 7675c71cff927a677c9f3366962800469fa502b5
Author: Brett Smith <brett.smith at curii.com>
Date: Sat Feb 24 23:33:20 2024 -0500
21504: Refine arv-mount description and epilog
* Modernize the description.
* Remove outdated mode information.
* Move epilog to --exec help.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>
diff --git a/services/fuse/arvados_fuse/command.py b/services/fuse/arvados_fuse/command.py
index 3c9d5f5822..cc00321cd0 100644
--- a/services/fuse/arvados_fuse/command.py
+++ b/services/fuse/arvados_fuse/command.py
@@ -28,14 +28,8 @@ class ArgumentParser(argparse.ArgumentParser):
def __init__(self):
super(ArgumentParser, self).__init__(
parents=[arv_cmd.retry_opt],
- description="""
-Mount Keep data under the local filesystem. Default mode is --home
-""",
- epilog="""
- Note: When using the --exec feature, you must either specify the
- mountpoint before --exec, or mark the end of your --exec arguments
- with "--".
- """)
+ description="Interact with Arvados data through a local filesystem",
+ )
self.add_argument(
'--version',
action='version',
@@ -204,7 +198,13 @@ and delete it when unmounting.
'--exec',
nargs=argparse.REMAINDER,
dest="exec_args",
- help="""Mount, run a command, then unmount and exit""",
+ help="""
+Mount data, run the specified command, then unmount and exit.
+--exec reads all remaining options as the command to run,
+so it must be the last option you specify.
+Either end your command arguments (and other options) with a `--` argument,
+or specify --exec after your mount point.
+""",
)
lifecycle.add_argument(
'--foreground',
commit 5a21c904426121a082c5daa2da2f65eb45154511
Author: Brett Smith <brett.smith at curii.com>
Date: Sat Feb 24 23:26:21 2024 -0500
21504: Refine metavars for arv-mount options
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>
diff --git a/services/fuse/arvados_fuse/command.py b/services/fuse/arvados_fuse/command.py
index 6d8c047a9f..3c9d5f5822 100644
--- a/services/fuse/arvados_fuse/command.py
+++ b/services/fuse/arvados_fuse/command.py
@@ -44,6 +44,7 @@ Mount Keep data under the local filesystem. Default mode is --home
)
self.add_argument(
'mountpoint',
+ metavar='MOUNT_DIR',
help="""Mount point.""",
)
@@ -71,7 +72,7 @@ Mount a top level meta-directory with subdirectories as specified by additional
)
mode.add_argument(
'--collection',
- metavar='UUID_or_PDH',
+ metavar='UUID_OR_PDH',
help="""Mount only the specified collection.""",
)
mode.add_argument(
@@ -203,7 +204,6 @@ and delete it when unmounting.
'--exec',
nargs=argparse.REMAINDER,
dest="exec_args",
- metavar=('command', 'args', '...', '--'),
help="""Mount, run a command, then unmount and exit""",
)
lifecycle.add_argument(
@@ -214,8 +214,7 @@ and delete it when unmounting.
)
lifecycle.add_argument(
'--subtype',
- metavar='STRING',
- help="""Report mounted filesystem type as "fuse.STRING", instead of just "fuse".""",
+ help="""Report mounted filesystem type as "fuse.SUBTYPE", instead of just "fuse".""",
)
unmount = lifecycle.add_mutually_exclusive_group()
unmount.add_argument(
@@ -251,6 +250,7 @@ WARNING: This command can affect any kind of fuse mount, not just arv-mount.
'--unmount-timeout',
type=float,
default=2.0,
+ metavar='SECONDS',
help="""
Time to wait for graceful shutdown after --exec program exits and filesystem is unmounted
""",
@@ -261,6 +261,7 @@ Time to wait for graceful shutdown after --exec program exits and filesystem is
'--crunchstat-interval',
type=float,
default=0.0,
+ metavar='SECONDS',
help="Write stats to stderr every N seconds (default disabled)",
)
reporting.add_argument(
@@ -291,18 +292,21 @@ Time to wait for graceful shutdown after --exec program exits and filesystem is
)
cache.add_argument(
'--disk-cache-dir',
+ metavar="DIRECTORY",
help="Disk cache location (default ~/.cache/arvados/keep)",
)
cache.add_argument(
'--directory-cache',
type=int,
default=128*1024*1024,
+ metavar='BYTES',
help="Directory data cache size, in bytes (default 128 MiB)",
)
cache.add_argument(
'--file-cache',
type=int,
default=0,
+ metavar='BYTES',
help="""
File data cache size, in bytes
(default 8 GiB for disk-based cache or 256 MiB with RAM-only cache)
commit fcc1154442af6919767c367a05ce86cf95622e57
Author: Brett Smith <brett.smith at curii.com>
Date: Sat Feb 24 23:17:53 2024 -0500
21504: Organize arv-mount options into groups
Groups are ordered roughly from more to less important, although
different users will have different needs. Options within a group are
mostly alphabetical except when organization requires otherwise.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>
diff --git a/services/fuse/arvados_fuse/command.py b/services/fuse/arvados_fuse/command.py
index cec5053c64..6d8c047a9f 100644
--- a/services/fuse/arvados_fuse/command.py
+++ b/services/fuse/arvados_fuse/command.py
@@ -46,18 +46,9 @@ Mount Keep data under the local filesystem. Default mode is --home
'mountpoint',
help="""Mount point.""",
)
- self.add_argument(
- '--allow-other',
- action='store_true',
- help="""Let other users read the mount""",
- )
- self.add_argument(
- '--subtype',
- metavar='STRING',
- help="""Report mounted filesystem type as "fuse.STRING", instead of just "fuse".""",
- )
- mode = self.add_mutually_exclusive_group()
+ mode_group = self.add_argument_group("Mount contents")
+ mode = mode_group.add_mutually_exclusive_group()
mode.add_argument(
'--all',
action='store_const',
@@ -78,6 +69,11 @@ Mount a top level meta-directory with subdirectories as specified by additional
(default if any --mount-* arguments are given).
""",
)
+ mode.add_argument(
+ '--collection',
+ metavar='UUID_or_PDH',
+ help="""Mount only the specified collection.""",
+ )
mode.add_argument(
'--home',
action='store_const',
@@ -85,6 +81,11 @@ Mount a top level meta-directory with subdirectories as specified by additional
dest='mode',
help="""Mount only the user's home project.""",
)
+ mode.add_argument(
+ '--project',
+ metavar='UUID',
+ help="""Mount the specified project.""",
+ )
mode.add_argument(
'--shared',
action='store_const',
@@ -92,13 +93,6 @@ Mount a top level meta-directory with subdirectories as specified by additional
dest='mode',
help="""Mount only list of projects shared with the user.""",
)
- mode.add_argument(
- '--by-tag',
- action='store_const',
- const='by_tag',
- dest='mode',
- help="""Mount subdirectories listed by tag.""",
- )
mode.add_argument(
'--by-id',
action='store_const',
@@ -114,26 +108,29 @@ Mount a top level meta-directory with subdirectories as specified by additional
help="""Mount subdirectories listed by portable data hash.""",
)
mode.add_argument(
- '--project',
- metavar='UUID',
- help="""Mount the specified project.""",
- )
- mode.add_argument(
- '--collection',
- metavar='UUID_or_PDH',
- help="""Mount only the specified collection.""",
+ '--by-tag',
+ action='store_const',
+ const='by_tag',
+ dest='mode',
+ help="""Mount subdirectories listed by tag.""",
)
- mounts = self.add_argument_group('Custom mount options')
+ mounts = self.add_argument_group("Mount custom layout and filtering")
mounts.add_argument(
- '--mount-by-pdh',
+ '--filters',
+ type=arv_cmd.JSONArgument(arv_cmd.validate_filters),
+ help="""
+Filters to apply to all project, shared, and tag directory contents.
+Pass filters as either a JSON string or a path to a JSON file.
+The JSON object should be a list of filters in Arvados API list filter syntax.
+""",
+ )
+ mounts.add_argument(
+ '--mount-home',
metavar='PATH',
action='append',
default=[],
- help="""
-Mount each readable collection at mountpoint/PATH/P
-where P is the collection's portable data hash.
-""",
+ help="Mount the current user's home project at mountpoint/PATH.",
)
mounts.add_argument(
'--mount-by-id',
@@ -146,25 +143,28 @@ where PDH is the collection's portable data hash and UUID is its UUID.
""",
)
mounts.add_argument(
- '--mount-by-tag',
+ '--mount-by-pdh',
metavar='PATH',
action='append',
default=[],
- help="Mount all collections with tag TAG at mountpoint/PATH/TAG/UUID.",
+ help="""
+Mount each readable collection at mountpoint/PATH/P
+where P is the collection's portable data hash.
+""",
)
mounts.add_argument(
- '--mount-home',
+ '--mount-shared',
metavar='PATH',
action='append',
default=[],
- help="Mount the current user's home project at mountpoint/PATH.",
+ help="Mount projects shared with the current user at mountpoint/PATH.",
)
mounts.add_argument(
- '--mount-shared',
+ '--mount-by-tag',
metavar='PATH',
action='append',
default=[],
- help="Mount projects shared with the current user at mountpoint/PATH.",
+ help="Mount all collections with tag TAG at mountpoint/PATH/TAG/UUID.",
)
mounts.add_argument(
'--mount-tmp',
@@ -177,103 +177,55 @@ and delete it when unmounting.
""",
)
- self.add_argument(
- '--debug',
- action='store_true',
- help="""Debug mode""",
- )
- self.add_argument(
- '--logfile',
- help="""Write debug logs and errors to the specified file (default stderr).""",
- )
- self.add_argument(
- '--foreground',
- action='store_true',
- default=False,
- help="""Run in foreground (default is to daemonize unless --exec specified)""",
- )
- self.add_argument(
- '--encoding',
- default="utf-8",
- help="""
-Character encoding to use for filesystem, default is utf-8
-(see Python codec registry for list of available encodings)"
-""",
- )
-
- self.add_argument(
- '--file-cache',
- type=int,
- default=0,
- help="""
-File data cache size, in bytes
-(default 8 GiB for disk-based cache or 256 MiB with RAM-only cache)
-""",
- )
- self.add_argument(
- '--directory-cache',
- type=int,
- default=128*1024*1024,
- help="Directory data cache size, in bytes (default 128 MiB)",
- )
-
- cachetype = self.add_mutually_exclusive_group()
- cachetype.add_argument(
- '--ram-cache',
- action='store_false',
- default=True,
- dest='disk_cache',
- help="Use in-memory caching only",
- )
- cachetype.add_argument(
- '--disk-cache',
- action='store_true',
- default=True,
- dest='disk_cache',
- help="Use disk based caching (default)",
- )
-
- self.add_argument(
- '--disk-cache-dir',
- help="Disk cache location (default ~/.cache/arvados/keep)",
- )
-
- self.add_argument(
- '--disable-event-listening',
+ perms = self.add_argument_group("Mount access and permissions")
+ perms.add_argument(
+ '--allow-other',
action='store_true',
- dest='disable_event_listening',
- default=False,
- help="Don't subscribe to events on the API server",
+ help="""Let other users read the mount""",
)
-
- self.add_argument(
+ perms.add_argument(
'--read-only',
action='store_false',
default=False,
dest='enable_write',
help="Mount will be read only (default)",
)
- self.add_argument(
+ perms.add_argument(
'--read-write',
action='store_true',
default=False,
dest='enable_write',
help="Mount will be read-write",
)
- self.add_argument(
- '--storage-classes',
- metavar='CLASSES',
- help="Specify comma separated list of storage classes to be used when saving data of new collections",
- )
- self.add_argument(
- '--crunchstat-interval',
- type=float,
- default=0.0,
- help="Write stats to stderr every N seconds (default disabled)",
+ lifecycle = self.add_argument_group("Mount lifecycle management")
+ lifecycle.add_argument(
+ '--exec',
+ nargs=argparse.REMAINDER,
+ dest="exec_args",
+ metavar=('command', 'args', '...', '--'),
+ help="""Mount, run a command, then unmount and exit""",
+ )
+ lifecycle.add_argument(
+ '--foreground',
+ action='store_true',
+ default=False,
+ help="""Run in foreground (default is to daemonize unless --exec specified)""",
+ )
+ lifecycle.add_argument(
+ '--subtype',
+ metavar='STRING',
+ help="""Report mounted filesystem type as "fuse.STRING", instead of just "fuse".""",
+ )
+ unmount = lifecycle.add_mutually_exclusive_group()
+ unmount.add_argument(
+ '--replace',
+ action='store_true',
+ default=False,
+ help="""
+If a fuse mount is already present at mountpoint, forcefully unmount it before mounting
+""",
)
-
- unmount = self.add_mutually_exclusive_group()
unmount.add_argument(
'--unmount',
action='store_true',
@@ -295,15 +247,7 @@ Exit non-zero if any other types of mounts are found at or below the given path.
WARNING: This command can affect any kind of fuse mount, not just arv-mount.
""",
)
- unmount.add_argument(
- '--replace',
- action='store_true',
- default=False,
- help="""
-If a fuse mount is already present at mountpoint, forcefully unmount it before mounting
-""",
- )
- self.add_argument(
+ lifecycle.add_argument(
'--unmount-timeout',
type=float,
default=2.0,
@@ -311,21 +255,80 @@ If a fuse mount is already present at mountpoint, forcefully unmount it before m
Time to wait for graceful shutdown after --exec program exits and filesystem is unmounted
""",
)
- self.add_argument(
- '--filters',
- type=arv_cmd.JSONArgument(arv_cmd.validate_filters),
+
+ reporting = self.add_argument_group("Mount logging and statistics")
+ reporting.add_argument(
+ '--crunchstat-interval',
+ type=float,
+ default=0.0,
+ help="Write stats to stderr every N seconds (default disabled)",
+ )
+ reporting.add_argument(
+ '--debug',
+ action='store_true',
+ help="""Debug mode""",
+ )
+ reporting.add_argument(
+ '--logfile',
+ help="""Write debug logs and errors to the specified file (default stderr).""",
+ )
+
+ cache = self.add_argument_group("Mount local cache setup")
+ cachetype = cache.add_mutually_exclusive_group()
+ cachetype.add_argument(
+ '--disk-cache',
+ action='store_true',
+ default=True,
+ dest='disk_cache',
+ help="Use disk based caching (default)",
+ )
+ cachetype.add_argument(
+ '--ram-cache',
+ action='store_false',
+ default=True,
+ dest='disk_cache',
+ help="Use in-memory caching only",
+ )
+ cache.add_argument(
+ '--disk-cache-dir',
+ help="Disk cache location (default ~/.cache/arvados/keep)",
+ )
+ cache.add_argument(
+ '--directory-cache',
+ type=int,
+ default=128*1024*1024,
+ help="Directory data cache size, in bytes (default 128 MiB)",
+ )
+ cache.add_argument(
+ '--file-cache',
+ type=int,
+ default=0,
help="""
-Filters to apply to all project, shared, and tag directory contents.
-Pass filters as either a JSON string or a path to a JSON file.
-The JSON object should be a list of filters in Arvados API list filter syntax.
+File data cache size, in bytes
+(default 8 GiB for disk-based cache or 256 MiB with RAM-only cache)
""",
)
- self.add_argument(
- '--exec',
- nargs=argparse.REMAINDER,
- dest="exec_args",
- metavar=('command', 'args', '...', '--'),
- help="""Mount, run a command, then unmount and exit""",
+
+ plumbing = self.add_argument_group("Mount interactions with Arvados and Linux")
+ plumbing.add_argument(
+ '--disable-event-listening',
+ action='store_true',
+ dest='disable_event_listening',
+ default=False,
+ help="Don't subscribe to events on the API server",
+ )
+ plumbing.add_argument(
+ '--encoding',
+ default="utf-8",
+ help="""
+Character encoding to use for filesystem, default is utf-8
+(see Python codec registry for list of available encodings)
+""",
+ )
+ plumbing.add_argument(
+ '--storage-classes',
+ metavar='CLASSES',
+ help="Specify comma separated list of storage classes to be used when saving data of new collections",
)
commit 84225502fc158312c02825da20f235232f66909d
Author: Brett Smith <brett.smith at curii.com>
Date: Sat Feb 24 22:44:47 2024 -0500
21504: Style arv-mount argument creation consistently
This is purely a code style change: every argparse argument is created
with one argument per line. This is preparation for readability
reorganization I'm about to do.
Remove arguments that match the argparse defaults as part of this.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>
diff --git a/services/fuse/arvados_fuse/command.py b/services/fuse/arvados_fuse/command.py
index 610da477ca..cec5053c64 100644
--- a/services/fuse/arvados_fuse/command.py
+++ b/services/fuse/arvados_fuse/command.py
@@ -28,105 +28,305 @@ class ArgumentParser(argparse.ArgumentParser):
def __init__(self):
super(ArgumentParser, self).__init__(
parents=[arv_cmd.retry_opt],
- description='''Mount Keep data under the local filesystem. Default mode is --home''',
+ description="""
+Mount Keep data under the local filesystem. Default mode is --home
+""",
epilog="""
Note: When using the --exec feature, you must either specify the
mountpoint before --exec, or mark the end of your --exec arguments
with "--".
""")
- self.add_argument('--version', action='version',
- version=u"%s %s" % (sys.argv[0], __version__),
- help='Print version and exit.')
- self.add_argument('mountpoint', type=str, help="""Mount point.""")
- self.add_argument('--allow-other', action='store_true',
- help="""Let other users read the mount""")
- self.add_argument('--subtype', type=str, metavar='STRING',
- help="""Report mounted filesystem type as "fuse.STRING", instead of just "fuse".""")
+ self.add_argument(
+ '--version',
+ action='version',
+ version=u"%s %s" % (sys.argv[0], __version__),
+ help='Print version and exit.',
+ )
+ self.add_argument(
+ 'mountpoint',
+ help="""Mount point.""",
+ )
+ self.add_argument(
+ '--allow-other',
+ action='store_true',
+ help="""Let other users read the mount""",
+ )
+ self.add_argument(
+ '--subtype',
+ metavar='STRING',
+ help="""Report mounted filesystem type as "fuse.STRING", instead of just "fuse".""",
+ )
mode = self.add_mutually_exclusive_group()
-
- mode.add_argument('--all', action='store_const', const='all', dest='mode',
- help="""Mount a subdirectory for each mode: home, shared, by_tag, by_id (default if no --mount-* arguments are given).""")
- mode.add_argument('--custom', action='store_const', const=None, dest='mode',
- help="""Mount a top level meta-directory with subdirectories as specified by additional --mount-* arguments (default if any --mount-* arguments are given).""")
- mode.add_argument('--home', action='store_const', const='home', dest='mode',
- help="""Mount only the user's home project.""")
- mode.add_argument('--shared', action='store_const', const='shared', dest='mode',
- help="""Mount only list of projects shared with the user.""")
- mode.add_argument('--by-tag', action='store_const', const='by_tag', dest='mode',
- help="""Mount subdirectories listed by tag.""")
- mode.add_argument('--by-id', action='store_const', const='by_id', dest='mode',
- help="""Mount subdirectories listed by portable data hash or uuid.""")
- mode.add_argument('--by-pdh', action='store_const', const='by_pdh', dest='mode',
- help="""Mount subdirectories listed by portable data hash.""")
- mode.add_argument('--project', type=str, metavar='UUID',
- help="""Mount the specified project.""")
- mode.add_argument('--collection', type=str, metavar='UUID_or_PDH',
- help="""Mount only the specified collection.""")
+ mode.add_argument(
+ '--all',
+ action='store_const',
+ const='all',
+ dest='mode',
+ help="""
+Mount a subdirectory for each mode: home, shared, by_tag, by_id
+(default if no --mount-* arguments are given).
+""",
+ )
+ mode.add_argument(
+ '--custom',
+ action='store_const',
+ const=None,
+ dest='mode',
+ help="""
+Mount a top level meta-directory with subdirectories as specified by additional --mount-* arguments
+(default if any --mount-* arguments are given).
+""",
+ )
+ mode.add_argument(
+ '--home',
+ action='store_const',
+ const='home',
+ dest='mode',
+ help="""Mount only the user's home project.""",
+ )
+ mode.add_argument(
+ '--shared',
+ action='store_const',
+ const='shared',
+ dest='mode',
+ help="""Mount only list of projects shared with the user.""",
+ )
+ mode.add_argument(
+ '--by-tag',
+ action='store_const',
+ const='by_tag',
+ dest='mode',
+ help="""Mount subdirectories listed by tag.""",
+ )
+ mode.add_argument(
+ '--by-id',
+ action='store_const',
+ const='by_id',
+ dest='mode',
+ help="""Mount subdirectories listed by portable data hash or uuid.""",
+ )
+ mode.add_argument(
+ '--by-pdh',
+ action='store_const',
+ const='by_pdh',
+ dest='mode',
+ help="""Mount subdirectories listed by portable data hash.""",
+ )
+ mode.add_argument(
+ '--project',
+ metavar='UUID',
+ help="""Mount the specified project.""",
+ )
+ mode.add_argument(
+ '--collection',
+ metavar='UUID_or_PDH',
+ help="""Mount only the specified collection.""",
+ )
mounts = self.add_argument_group('Custom mount options')
- mounts.add_argument('--mount-by-pdh',
- type=str, metavar='PATH', action='append', default=[],
- help="Mount each readable collection at mountpoint/PATH/P where P is the collection's portable data hash.")
- mounts.add_argument('--mount-by-id',
- type=str, metavar='PATH', action='append', default=[],
- help="Mount each readable collection at mountpoint/PATH/UUID and mountpoint/PATH/PDH where PDH is the collection's portable data hash and UUID is its UUID.")
- mounts.add_argument('--mount-by-tag',
- type=str, metavar='PATH', action='append', default=[],
- help="Mount all collections with tag TAG at mountpoint/PATH/TAG/UUID.")
- mounts.add_argument('--mount-home',
- type=str, metavar='PATH', action='append', default=[],
- help="Mount the current user's home project at mountpoint/PATH.")
- mounts.add_argument('--mount-shared',
- type=str, metavar='PATH', action='append', default=[],
- help="Mount projects shared with the current user at mountpoint/PATH.")
- mounts.add_argument('--mount-tmp',
- type=str, metavar='PATH', action='append', default=[],
- help="Create a new collection, mount it in read/write mode at mountpoint/PATH, and delete it when unmounting.")
-
-
- self.add_argument('--debug', action='store_true', help="""Debug mode""")
- self.add_argument('--logfile', help="""Write debug logs and errors to the specified file (default stderr).""")
- self.add_argument('--foreground', action='store_true', help="""Run in foreground (default is to daemonize unless --exec specified)""", default=False)
- self.add_argument('--encoding', type=str, help="Character encoding to use for filesystem, default is utf-8 (see Python codec registry for list of available encodings)", default="utf-8")
-
- self.add_argument('--file-cache', type=int, help="File data cache size, in bytes (default 8 GiB for disk-based cache or 256 MiB with RAM-only cache)", default=0)
- self.add_argument('--directory-cache', type=int, help="Directory data cache size, in bytes (default 128 MiB)", default=128*1024*1024)
+ mounts.add_argument(
+ '--mount-by-pdh',
+ metavar='PATH',
+ action='append',
+ default=[],
+ help="""
+Mount each readable collection at mountpoint/PATH/P
+where P is the collection's portable data hash.
+""",
+ )
+ mounts.add_argument(
+ '--mount-by-id',
+ metavar='PATH',
+ action='append',
+ default=[],
+ help="""
+Mount each readable collection at mountpoint/PATH/UUID and mountpoint/PATH/PDH
+where PDH is the collection's portable data hash and UUID is its UUID.
+""",
+ )
+ mounts.add_argument(
+ '--mount-by-tag',
+ metavar='PATH',
+ action='append',
+ default=[],
+ help="Mount all collections with tag TAG at mountpoint/PATH/TAG/UUID.",
+ )
+ mounts.add_argument(
+ '--mount-home',
+ metavar='PATH',
+ action='append',
+ default=[],
+ help="Mount the current user's home project at mountpoint/PATH.",
+ )
+ mounts.add_argument(
+ '--mount-shared',
+ metavar='PATH',
+ action='append',
+ default=[],
+ help="Mount projects shared with the current user at mountpoint/PATH.",
+ )
+ mounts.add_argument(
+ '--mount-tmp',
+ metavar='PATH',
+ action='append',
+ default=[],
+ help="""
+Create a new collection, mount it in read/write mode at mountpoint/PATH,
+and delete it when unmounting.
+""",
+ )
+
+ self.add_argument(
+ '--debug',
+ action='store_true',
+ help="""Debug mode""",
+ )
+ self.add_argument(
+ '--logfile',
+ help="""Write debug logs and errors to the specified file (default stderr).""",
+ )
+ self.add_argument(
+ '--foreground',
+ action='store_true',
+ default=False,
+ help="""Run in foreground (default is to daemonize unless --exec specified)""",
+ )
+ self.add_argument(
+ '--encoding',
+ default="utf-8",
+ help="""
+Character encoding to use for filesystem, default is utf-8
+(see Python codec registry for list of available encodings)"
+""",
+ )
+
+ self.add_argument(
+ '--file-cache',
+ type=int,
+ default=0,
+ help="""
+File data cache size, in bytes
+(default 8 GiB for disk-based cache or 256 MiB with RAM-only cache)
+""",
+ )
+ self.add_argument(
+ '--directory-cache',
+ type=int,
+ default=128*1024*1024,
+ help="Directory data cache size, in bytes (default 128 MiB)",
+ )
cachetype = self.add_mutually_exclusive_group()
- cachetype.add_argument('--ram-cache', action='store_false', dest='disk_cache', help="Use in-memory caching only", default=True)
- cachetype.add_argument('--disk-cache', action='store_true', dest='disk_cache', help="Use disk based caching (default)", default=True)
+ cachetype.add_argument(
+ '--ram-cache',
+ action='store_false',
+ default=True,
+ dest='disk_cache',
+ help="Use in-memory caching only",
+ )
+ cachetype.add_argument(
+ '--disk-cache',
+ action='store_true',
+ default=True,
+ dest='disk_cache',
+ help="Use disk based caching (default)",
+ )
- self.add_argument('--disk-cache-dir', type=str, help="Disk cache location (default ~/.cache/arvados/keep)", default=None)
+ self.add_argument(
+ '--disk-cache-dir',
+ help="Disk cache location (default ~/.cache/arvados/keep)",
+ )
- self.add_argument('--disable-event-listening', action='store_true', help="Don't subscribe to events on the API server", dest="disable_event_listening", default=False)
+ self.add_argument(
+ '--disable-event-listening',
+ action='store_true',
+ dest='disable_event_listening',
+ default=False,
+ help="Don't subscribe to events on the API server",
+ )
- self.add_argument('--read-only', action='store_false', help="Mount will be read only (default)", dest="enable_write", default=False)
- self.add_argument('--read-write', action='store_true', help="Mount will be read-write", dest="enable_write", default=False)
- self.add_argument('--storage-classes', type=str, metavar='CLASSES', help="Specify comma separated list of storage classes to be used when saving data of new collections", default=None)
+ self.add_argument(
+ '--read-only',
+ action='store_false',
+ default=False,
+ dest='enable_write',
+ help="Mount will be read only (default)",
+ )
+ self.add_argument(
+ '--read-write',
+ action='store_true',
+ default=False,
+ dest='enable_write',
+ help="Mount will be read-write",
+ )
+ self.add_argument(
+ '--storage-classes',
+ metavar='CLASSES',
+ help="Specify comma separated list of storage classes to be used when saving data of new collections",
+ )
- self.add_argument('--crunchstat-interval', type=float, help="Write stats to stderr every N seconds (default disabled)", default=0)
+ self.add_argument(
+ '--crunchstat-interval',
+ type=float,
+ default=0.0,
+ help="Write stats to stderr every N seconds (default disabled)",
+ )
unmount = self.add_mutually_exclusive_group()
- unmount.add_argument('--unmount', action='store_true', default=False,
- help="Forcefully unmount the specified mountpoint (if it's a fuse mount) and exit. If --subtype is given, unmount only if the mount has the specified subtype. WARNING: This command can affect any kind of fuse mount, not just arv-mount.")
- unmount.add_argument('--unmount-all', action='store_true', default=False,
- help="Forcefully unmount every fuse mount at or below the specified path and exit. If --subtype is given, unmount only mounts that have the specified subtype. Exit non-zero if any other types of mounts are found at or below the given path. WARNING: This command can affect any kind of fuse mount, not just arv-mount.")
- unmount.add_argument('--replace', action='store_true', default=False,
- help="If a fuse mount is already present at mountpoint, forcefully unmount it before mounting")
- self.add_argument('--unmount-timeout',
- type=float, default=2.0,
- help="Time to wait for graceful shutdown after --exec program exits and filesystem is unmounted")
+ unmount.add_argument(
+ '--unmount',
+ action='store_true',
+ default=False,
+ help="""
+Forcefully unmount the specified mountpoint (if it's a fuse mount) and exit.
+If --subtype is given, unmount only if the mount has the specified subtype.
+WARNING: This command can affect any kind of fuse mount, not just arv-mount.
+""",
+ )
+ unmount.add_argument(
+ '--unmount-all',
+ action='store_true',
+ default=False,
+ help="""
+Forcefully unmount every fuse mount at or below the specified path and exit.
+If --subtype is given, unmount only mounts that have the specified subtype.
+Exit non-zero if any other types of mounts are found at or below the given path.
+WARNING: This command can affect any kind of fuse mount, not just arv-mount.
+""",
+ )
+ unmount.add_argument(
+ '--replace',
+ action='store_true',
+ default=False,
+ help="""
+If a fuse mount is already present at mountpoint, forcefully unmount it before mounting
+""",
+ )
+ self.add_argument(
+ '--unmount-timeout',
+ type=float,
+ default=2.0,
+ help="""
+Time to wait for graceful shutdown after --exec program exits and filesystem is unmounted
+""",
+ )
self.add_argument(
'--filters',
type=arv_cmd.JSONArgument(arv_cmd.validate_filters),
- help="""Filters to apply to all project, shared, and tag directory
-contents. Pass filters as either a JSON string or a path to a JSON file.
+ help="""
+Filters to apply to all project, shared, and tag directory contents.
+Pass filters as either a JSON string or a path to a JSON file.
The JSON object should be a list of filters in Arvados API list filter syntax.
-""")
- self.add_argument('--exec', type=str, nargs=argparse.REMAINDER,
- dest="exec_args", metavar=('command', 'args', '...', '--'),
- help="""Mount, run a command, then unmount and exit""")
+""",
+ )
+ self.add_argument(
+ '--exec',
+ nargs=argparse.REMAINDER,
+ dest="exec_args",
+ metavar=('command', 'args', '...', '--'),
+ help="""Mount, run a command, then unmount and exit""",
+ )
class Mount(object):
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list