[arvados] updated: 2.7.0-6263-g78375ebf27
git repository hosting
git at public.arvados.org
Fri Apr 26 14:42:43 UTC 2024
Summary of changes:
doc/_includes/_container_glob_patterns.liquid | 27 ++++++++++++++++++++++
.../methods/container_requests.html.textile.liquid | 7 +++---
doc/api/methods/containers.html.textile.liquid | 4 +++-
lib/crunchrun/copier_test.go | 11 +++++++++
4 files changed, 45 insertions(+), 4 deletions(-)
create mode 100644 doc/_includes/_container_glob_patterns.liquid
via 78375ebf2707912016b3b94fec6b0f2998cb98fe (commit)
via 3c69a3d346adac763b6426538676810921d25646 (commit)
from c2ab17464100b4914b65473175ff1c85b102f1e3 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
commit 78375ebf2707912016b3b94fec6b0f2998cb98fe
Author: Tom Clegg <tom at curii.com>
Date: Fri Apr 26 10:39:36 2024 -0400
12430: Expand output_glob documentation.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom at curii.com>
diff --git a/doc/_includes/_container_glob_patterns.liquid b/doc/_includes/_container_glob_patterns.liquid
new file mode 100644
index 0000000000..4015e16220
--- /dev/null
+++ b/doc/_includes/_container_glob_patterns.liquid
@@ -0,0 +1,27 @@
+{% comment %}
+Copyright (C) The Arvados Authors. All rights reserved.
+
+SPDX-License-Identifier: CC-BY-SA-3.0
+{% endcomment %}
+
+h2. Glob patterns
+
+Each pattern in the @output_glob@ array can include the following special terms:
+
+table(table table-bordered table-condensed).
+|@*@|matches any sequence of non-@/@ characters|
+|@?@|matches any single non-@/@ character|
+|@[abcde]@ or @[a-e]@|matches any non-@/@ character in @abcde@|
+|@[^abcde]@ or @[^a-e]@ or
+@[!abcde]@ or @[!a-e]@|matches any non-@/@ character other than @abcde@|
+|@/**/@|matches zero or more levels of subdirectories|
+|@**/@|at the beginning of a pattern, matches zero or more directories|
+|@/**@|at the end of a pattern, matches any file in any subdirectory|
+
+Example patterns:
+
+table(table table-bordered table-condensed).
+|@*.txt@|matches files with extension @.txt@ at the top level|
+|@foo/**@|matches the entire tree rooted at @foo@ in the top level|
+|@**/fo[og]@|matches all files named @foo@ or @fog@ anywhere in the tree|
+|@foo/**/*.txt@|matches all files with extension @.txt@ anywhere in the tree rooted at @foo@ in the top level|
diff --git a/doc/api/methods/container_requests.html.textile.liquid b/doc/api/methods/container_requests.html.textile.liquid
index 66368848c6..38eb4909be 100644
--- a/doc/api/methods/container_requests.html.textile.liquid
+++ b/doc/api/methods/container_requests.html.textile.liquid
@@ -49,9 +49,8 @@ table(table table-bordered table-condensed).
|cwd|string|Initial working directory, given as an absolute path (in the container) or a path relative to the WORKDIR given in the image's Dockerfile.|Required.|
|command|array of strings|Command to execute in the container.|Required. e.g., @["echo","hello"]@|
|output_path|string|Path to a directory or file inside the container that should be preserved as container's output when it finishes. This path must be one of the mount targets. For best performance, point output_path to a writable collection mount. See "Pre-populate output using Mount points":#pre-populate-output for details regarding optional output pre-population using mount points and "Symlinks in output":#symlinks-in-output for additional details.|Required.|
-|output_glob|array of strings|Glob patterns determining which files (of those present in the output directory when the container finishes) will be included in the output collection. If multiple patterns are given, files that match any pattern are included. If null or empty, all files will be included.|"Doublestar" patterns are accepted:
- at foo/**@ matches the entire tree rooted at @foo@ in the top level of the output directory.
-@**/fo[og]@ matches all files named @foo@ or @fog@ anywhere in the tree.|
+|output_glob|array of strings|Glob patterns determining which files (of those present in the output directory when the container finishes) will be included in the output collection. If multiple patterns are given, files that match any pattern are included. If null or empty, all files will be included.|e.g., @["**/*.vcf", "**/*.vcf.gz"]@
+See "Glob patterns":#glob_patterns for more details.|
|output_name|string|Desired name for the output collection. If null or empty, a name will be assigned automatically.||
|output_ttl|integer|Desired lifetime for the output collection, in seconds. If zero, the output collection will not be deleted automatically.||
|priority|integer|Range 0-1000. Indicate scheduling order preference.|Clients are expected to submit container requests with zero priority in order to preview the container that will be used to satisfy it. Priority can be null if and only if state!="Committed". See "below for more details":#priority .|
@@ -141,6 +140,8 @@ h2(#runtime_constraints). {% include 'container_runtime_constraints' %}
h2(#scheduling_parameters). {% include 'container_scheduling_parameters' %}
+h2(#glob_patterns). {% include 'container_glob_patterns' %}
+
h2(#container_reuse). Container reuse
When a container request is "Committed", the system will try to find and reuse an existing Container with the same command, cwd, environment, output_path, container_image, mounts, secret_mounts, runtime_constraints, runtime_user_uuid, and runtime_auth_scopes being requested.
diff --git a/doc/api/methods/containers.html.textile.liquid b/doc/api/methods/containers.html.textile.liquid
index f6f59291da..776d4098c5 100644
--- a/doc/api/methods/containers.html.textile.liquid
+++ b/doc/api/methods/containers.html.textile.liquid
@@ -33,7 +33,7 @@ table(table table-bordered table-condensed).
|cwd|string|Initial working directory.|Must be equal to a ContainerRequest's cwd in order to satisfy the ContainerRequest.|
|command|array of strings|Command to execute.| Must be equal to a ContainerRequest's command in order to satisfy the ContainerRequest.|
|output_path|string|Path to a directory or file inside the container that should be preserved as this container's output when it finishes.|Must be equal to a ContainerRequest's output_path in order to satisfy the ContainerRequest.|
-|output_glob|array of strings|Glob patterns determining which files will be included in the output collection. See corresponding attribute in the "container_requests resource":container_requests.html.|Must be equal to a ContainerRequest's output_glob in order to satisfy the ContainerRequest.|
+|output_glob|array of strings|Glob patterns determining which files will be included in the output collection. See corresponding attribute in the "container_requests resource":container_requests.html.|Must be equal to a ContainerRequest's output_glob in order to satisfy the ContainerRequest. See "Glob patterns":#glob_patterns for more details.|
|mounts|hash|Must contain the same keys as the ContainerRequest being satisfied. Each value must be within the range of values described in the ContainerRequest at the time the Container is assigned to the ContainerRequest.|See "Mount types":#mount_types for more details.|
|secret_mounts|hash|Must contain the same keys as the ContainerRequest being satisfied. Each value must be within the range of values described in the ContainerRequest at the time the Container is assigned to the ContainerRequest.|Not returned in API responses. Reset to empty when state is "Complete" or "Cancelled".|
|runtime_constraints|hash|Compute resources, and access to the outside world, that are / were available to the container.
@@ -98,6 +98,8 @@ table(table table-bordered table-condensed).
h2(#scheduling_parameters). {% include 'container_scheduling_parameters' %}
+h2(#glob_patterns). {% include 'container_glob_patterns' %}
+
h2. Methods
See "Common resource methods":{{site.baseurl}}/api/methods.html for more information about @create@, @delete@, @get@, @list@, and @update at .
commit 3c69a3d346adac763b6426538676810921d25646
Author: Tom Clegg <tom at curii.com>
Date: Fri Apr 26 09:36:18 2024 -0400
14230: Comment about test coverage, add trailing-** case.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom at curii.com>
diff --git a/lib/crunchrun/copier_test.go b/lib/crunchrun/copier_test.go
index bf7e789e45..30e13f6503 100644
--- a/lib/crunchrun/copier_test.go
+++ b/lib/crunchrun/copier_test.go
@@ -225,6 +225,12 @@ func (s *copierSuite) writeFileInOutputDir(c *check.C, path, data string) {
c.Assert(f.Close(), check.IsNil)
}
+// applyGlobsToFilesAndDirs uses the same glob-matching code as
+// applyGlobsToCollectionFS, so we don't need to test all of the same
+// glob-matching behavior covered in TestApplyGlobsToCollectionFS. We
+// do need to check that (a) the glob is actually being used to filter
+// out files, and (b) non-matching dirs still included if and only if
+// they are ancestors of matching files.
func (s *copierSuite) TestApplyGlobsToFilesAndDirs(c *check.C) {
dirs := []string{"dir1", "dir1/dir11", "dir1/dir12", "dir2"}
files := []string{"dir1/file11", "dir1/dir11/file111", "dir2/file2"}
@@ -273,6 +279,11 @@ func (s *copierSuite) TestApplyGlobsToFilesAndDirs(c *check.C) {
dirs: []string{"dir1", "dir1/dir11"},
files: nil,
},
+ {
+ globs: []string{"dir1/**"},
+ dirs: []string{"dir1", "dir1/dir11", "dir1/dir12"},
+ files: []string{"dir1/file11", "dir1/dir11/file111"},
+ },
} {
c.Logf("=== globs: %q", trial.globs)
cp := copier{
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list