[ARVADOS] created: 1.3.0-1316-gb23721a01
Git user
git at public.curoverse.com
Tue Jul 16 19:57:50 UTC 2019
at b23721a01f61c6d9862ea6cb4d15bd620e06eeef (commit)
commit b23721a01f61c6d9862ea6cb4d15bd620e06eeef
Author: Peter Amstutz <pamstutz at veritasgenetics.com>
Date: Tue Jul 16 15:57:28 2019 -0400
15028: Add upgrade notes about going from CWL v1.0 to v1.1
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz at veritasgenetics.com>
diff --git a/doc/_config.yml b/doc/_config.yml
index 36e8c64b8..6c7d3f2f3 100644
--- a/doc/_config.yml
+++ b/doc/_config.yml
@@ -45,17 +45,17 @@ navbar:
- user/topics/arv-copy.html.textile.liquid
- user/topics/storage-classes.html.textile.liquid
- user/topics/collection-versioning.html.textile.liquid
- - Running workflows at the command line:
- - user/cwl/cwl-runner.html.textile.liquid
- - user/cwl/cwl-run-options.html.textile.liquid
- Working with git repositories:
- user/tutorials/add-new-repository.html.textile.liquid
- user/tutorials/git-arvados-guide.html.textile.liquid
+ - Running workflows at the command line:
+ - user/cwl/cwl-runner.html.textile.liquid
+ - user/cwl/cwl-run-options.html.textile.liquid
- Develop an Arvados workflow:
- user/tutorials/intro-crunch.html.textile.liquid
- user/tutorials/writing-cwl-workflow.html.textile.liquid
- - user/cwl/federated-workflows.html.textile.liquid
- user/cwl/cwl-style.html.textile.liquid
+ - user/cwl/federated-workflows.html.textile.liquid
- user/cwl/cwl-extensions.html.textile.liquid
- user/topics/arv-docker.html.textile.liquid
- Reference:
diff --git a/doc/_includes/_what_is_cwl.liquid b/doc/_includes/_what_is_cwl.liquid
index 2fda2a7fb..fea11b3a3 100644
--- a/doc/_includes/_what_is_cwl.liquid
+++ b/doc/_includes/_what_is_cwl.liquid
@@ -4,4 +4,4 @@ Copyright (C) The Arvados Authors. All rights reserved.
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-The "Common Workflow Language (CWL)":http://commonwl.org is a multi-vendor open standard for describing analysis tools and workflows that are portable across a variety of platforms. CWL is the recommended way to develop and run workflows for Arvados. Arvados supports the "CWL v1.0":http://commonwl.org/v1.0 specification.
+The "Common Workflow Language (CWL)":http://commonwl.org is a multi-vendor open standard for describing analysis tools and workflows that are portable across a variety of platforms. CWL is the primary way to develop and run workflows for Arvados. Arvados supports versions "v1.0":http://commonwl.org/v1.0 and "v1.1":http://commonwl.org/v1.1 of the CWL specification.
diff --git a/doc/user/cwl/cwl-extensions.html.textile.liquid b/doc/user/cwl/cwl-extensions.html.textile.liquid
index 052d4509a..847df78f2 100644
--- a/doc/user/cwl/cwl-extensions.html.textile.liquid
+++ b/doc/user/cwl/cwl-extensions.html.textile.liquid
@@ -13,15 +13,15 @@ Arvados provides several extensions to CWL for workflow optimization, site-speci
To use Arvados CWL extensions, add the following @$namespaces@ section at the top of your CWL file:
-<pre>
+{% codeblock as yaml %}
$namespaces:
arv: "http://arvados.org/cwl#"
cwltool: "http://commonwl.org/cwltool#"
-</pre>
+{% endcodeblock %}
For portability, Arvados extensions should go into the @hints@ section of your CWL file, for example:
-<pre>
+{% codeblock as yaml %}
hints:
arv:RunInSingleContainer: {}
arv:RuntimeConstraints:
@@ -47,13 +47,13 @@ hints:
arv:ClusterTarget:
cluster_id: clsr1
project_uuid: clsr1-j7d0g-qxc4jcji7n4lafx
-</pre>
+{% endcodeblock %}
-The one exception to this is @arv:APIRequirement@, see note below.
+The one exception to the @hints@ section this is @arv:APIRequirement@, see note below.
-h2. arv:RunInSingleContainer
+h2(#RunInSingleContainer). arv:RunInSingleContainer
-Indicates that a subworkflow should run in a single container and not be scheduled as separate steps.
+Apply this to a workflow step that runs a subworkflow. Indicates that all the steps of the subworkflow should run together in a single container and not be scheduled separately. If you have a sequence of short-running steps (less than 1-2 minutes each) this enables you to avoid scheduling and data transfer overhead by running all the steps together at once. To use this feature, @cwltool@ must be installed in the container image.
h2. arv:RuntimeConstraints
@@ -76,26 +76,14 @@ table(table table-bordered table-condensed).
|_. Field |_. Type |_. Description |
|partition|string or array of strings||
-h2. arv:APIRequirement
+h2(#APIRequirement). arv:APIRequirement
+
+For CWL v1.1 scripts, if a step requires network access but not specifically access to the Arvados API server, prefer the standard feature "NetworkAccess":https://www.commonwl.org/v1.1/CommandLineTool.html#NetworkAccess . In the future, these may be differentiated by whether ARVADOS_API_HOST and ARVADOS_API_TOKEN is injected into the container or not.
Indicates that process wants to access to the Arvados API. Will be granted network access and have @ARVADOS_API_HOST@ and @ARVADOS_API_TOKEN@ set in the environment. Tools which rely on the Arvados API being present should put @arv:APIRequirement@ in the @requirements@ section of the tool (rather than @hints@) to indicate that that it is not portable to non-Arvados CWL runners.
Use @arv:APIRequirement@ in @hints@ to enable general (non-Arvados-specific) network access for a tool.
-h2. cwltool:LoadListingRequirement
-
-In CWL v1.0 documents, the default behavior for Directory objects is to recursively expand the @listing@ for access by parameter references an expressions. For directory trees containing many files, this can be expensive in both time and memory usage. Use @cwltool:LoadListingRequirement@ to change the behavior for expansion of directory listings in the workflow runner.
-
-table(table table-bordered table-condensed).
-|_. Field |_. Type |_. Description |
-|loadListing|string|One of @no_listing@, @shallow_listing@, or @deep_listing@|
-
-*no_listing*: Do not expand directory listing at all. The @listing@ field on the Directory object will be undefined.
-
-*shallow_listing*: Only expand the first level of directory listing. The @listing@ field on the toplevel Directory object will contain the directory contents, however @listing@ will not be defined on subdirectories.
-
-*deep_listing*: Recursively expand all levels of directory listing. The @listing@ field will be provided on the toplevel object and all subdirectories.
-
h2. arv:IntermediateOutput
Specify desired handling of intermediate output collections.
@@ -105,14 +93,6 @@ table(table table-bordered table-condensed).
|outputTTL|int|If the value is greater than zero, consider intermediate output collections to be temporary and should be automatically trashed. Temporary collections will be trashed @outputTTL@ seconds after creation. A value of zero means intermediate output should be retained indefinitely (this is the default behavior).
Note: arvados-cwl-runner currently does not take workflow dependencies into account when setting the TTL on an intermediate output collection. If the TTL is too short, it is possible for a collection to be trashed before downstream steps that consume it are started. The recommended minimum value for TTL is the expected duration of the entire the workflow.|
-h2. arv:ReuseRequirement
-
-Enable/disable work reuse for current process. Default true (work reuse enabled).
-
-table(table table-bordered table-condensed).
-|_. Field |_. Type |_. Description |
-|enableReuse|boolean|Enable/disable work reuse for current process. Default true (work reuse enabled).|
-
h2. cwltool:Secrets
Indicate that one or more input parameters are "secret". Must be applied at the top level Workflow. Secret parameters are not stored in keep, are hidden from logs and API responses, and are wiped from the database after the workflow completes.
@@ -121,15 +101,6 @@ table(table table-bordered table-condensed).
|_. Field |_. Type |_. Description |
|secrets|array<string>|Input parameters which are considered "secret". Must be strings.|
-
-h2. cwltool:TimeLimit
-
-Set an upper limit on the execution time of a CommandLineTool or ExpressionTool. A tool execution which exceeds the time limit may be preemptively terminated and considered failed. May also be used by batch systems to make scheduling decisions.
-
-table(table table-bordered table-condensed).
-|_. Field |_. Type |_. Description |
-|timelimit|int|Execution time limit in seconds. If set to zero, no limit is enforced.|
-
h2. arv:WorkflowRunnerResources
Specify resource requirements for the workflow runner process (arvados-cwl-runner) that manages a workflow run. Must be applied to the top level workflow. Will also be set implicitly when using @--submit-runner-ram@ on the command line along with @--create-workflow@ or @--update-workflow at . Use this to adjust the runner's allocation if the workflow runner is getting "out of memory" exceptions or being killed by the out-of-memory (OOM) killer.
@@ -159,3 +130,43 @@ requirements:
dockerPull: "debian:9"
arv:dockerCollectionPDH: "feaf1fc916103d7cdab6489e1f8c3a2b+174"
</pre>
+
+h1. Deprecated extensions
+
+The following extensions are deprecated because equivalent features are part of the CWL v1.1 standard.
+
+h2. cwltool:LoadListingRequirement
+
+For CWL v1.1 scripts, this is deprecated in favor of "loadListing":https://www.commonwl.org/v1.1/CommandLineTool.html#CommandInputParameter or "LoadListingRequirement":https://www.commonwl.org/v1.1/CommandLineTool.html#LoadListingRequirement
+
+In CWL v1.0 documents, the default behavior for Directory objects is to recursively expand the @listing@ for access by parameter references an expressions. For directory trees containing many files, this can be expensive in both time and memory usage. Use @cwltool:LoadListingRequirement@ to change the behavior for expansion of directory listings in the workflow runner.
+
+table(table table-bordered table-condensed).
+|_. Field |_. Type |_. Description |
+|loadListing|string|One of @no_listing@, @shallow_listing@, or @deep_listing@|
+
+*no_listing*: Do not expand directory listing at all. The @listing@ field on the Directory object will be undefined.
+
+*shallow_listing*: Only expand the first level of directory listing. The @listing@ field on the toplevel Directory object will contain the directory contents, however @listing@ will not be defined on subdirectories.
+
+*deep_listing*: Recursively expand all levels of directory listing. The @listing@ field will be provided on the toplevel object and all subdirectories.
+
+h2. arv:ReuseRequirement
+
+For CWL v1.1 scripts, this is deprecated in favor of "WorkReuse":https://www.commonwl.org/v1.1/CommandLineTool.html#WorkReuse .
+
+Enable/disable work reuse for current process. Default true (work reuse enabled).
+
+table(table table-bordered table-condensed).
+|_. Field |_. Type |_. Description |
+|enableReuse|boolean|Enable/disable work reuse for current process. Default true (work reuse enabled).|
+
+h2. cwltool:TimeLimit
+
+For CWL v1.1 scripts, this is deprecated in favor of "ToolTimeLimit":https://www.commonwl.org/v1.1/CommandLineTool.html#ToolTimeLimit
+
+Set an upper limit on the execution time of a CommandLineTool or ExpressionTool. A tool execution which exceeds the time limit may be preemptively terminated and considered failed. May also be used by batch systems to make scheduling decisions.
+
+table(table table-bordered table-condensed).
+|_. Field |_. Type |_. Description |
+|timelimit|int|Execution time limit in seconds. If set to zero, no limit is enforced.|
diff --git a/doc/user/cwl/cwl-style.html.textile.liquid b/doc/user/cwl/cwl-style.html.textile.liquid
index fe53f4a45..4f5e372a6 100644
--- a/doc/user/cwl/cwl-style.html.textile.liquid
+++ b/doc/user/cwl/cwl-style.html.textile.liquid
@@ -1,7 +1,7 @@
---
layout: default
navsection: userguide
-title: Best Practices for writing CWL
+title: Writing Portable High-Performance Workflows
...
{% comment %}
Copyright (C) The Arvados Authors. All rights reserved.
@@ -9,19 +9,17 @@ Copyright (C) The Arvados Authors. All rights reserved.
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-* To run on Arvados, a workflow should provide a @DockerRequirement@ in the @hints@ section.
+h2. Portability
-* Build a reusable library of components. Share tool wrappers and subworkflows between projects. Make use of and contribute to "community maintained workflows and tools":https://github.com/common-workflow-language/workflows and tool registries such as "Dockstore":http://dockstore.org .
+To run on Arvados, a workflow must provide a @DockerRequirement@ in either the @hints@ or @requirements@ section.
-* When combining a parameter value with a string, such as adding a filename extension, write @$(inputs.file.basename).ext@ instead of @$(inputs.file.basename + 'ext')@. The first form is evaluated as a simple text substitution, the second form (using the @+@ operator) is evaluated as an arbitrary Javascript expression and requires that you declare @InlineJavascriptRequirement at .
+Build a reusable library of components. Share tool wrappers and subworkflows between projects. Make use of and contribute to "community maintained workflows and tools":https://github.com/common-workflow-language/workflows and tool registries such as "Dockstore":http://dockstore.org .
-* Avoid declaring @InlineJavascriptRequirement@ or @ShellCommandRequirement@ unless you specifically need them. Don't include them "just in case" because they change the default behavior and may imply extra overhead.
+Avoid declaring @InlineJavascriptRequirement@ or @ShellCommandRequirement@ unless you specifically need them. Don't include them "just in case" because they change the default behavior and may imply extra overhead.
-* Don't write CWL scripts that access the Arvados SDK. This is non-portable; a script that access Arvados directly won't work with @cwltool@ or crunch v2.
+CommandLineTools wrapping custom scripts should represent the script as an input parameter with the script file as a default value. Use @secondaryFiles@ for scripts that consist of multiple files. For example:
-* CommandLineTools wrapping custom scripts should represent the script as an input parameter with the script file as a default value. Use @secondaryFiles@ for scripts that consist of multiple files. For example:
-
-<pre>
+{% codeblock as yaml %}
cwlVersion: v1.0
class: CommandLineTool
baseCommand: python
@@ -45,15 +43,57 @@ outputs:
type: File
outputBinding:
glob: "*.fastq"
-</pre>
+{% endcodeblock %}
+
+You can get the designated temporary directory using @$(runtime.tmpdir)@ in your CWL file, or from the @$TMPDIR@ environment variable in your script.
+
+Similarly, you can get the designated output directory using $(runtime.outdir), or from the @HOME@ environment variable in your script.
+
+Avoid specifying resource requirements in CommandLineTool. Prefer to specify them in the workflow. You can provide a default resource requirement in the top level @hints@ section, and individual steps can override it with their own resource requirement.
+
+{% codeblock as yaml %}
+cwlVersion: v1.0
+class: Workflow
+inputs:
+ inp: File
+hints:
+ ResourceRequirement:
+ ramMin: 1000
+ coresMin: 1
+ tmpdirMin: 45000
+steps:
+ step1:
+ in: {inp: inp}
+ out: [out]
+ run: tool1.cwl
+ step2:
+ in: {inp: step1/inp}
+ out: [out]
+ run: tool2.cwl
+ hints:
+ ResourceRequirement:
+ ramMin: 2000
+ coresMin: 2
+ tmpdirMin: 90000
+{% endcodeblock %}
+
+h3. Upgrading to CWL v1.1
+
+CWL v1.1 introduces several features to the standard that were previously available as Arvados extensions. CWL v1.1 syntax is backwards compatible with v1.0, so you can just change @cwlVersion: v1.0@ to @cwlVersion: v1.1@ and update your script to using the standard features. On Arvados, there is only one behavior change between CWL v1.0 and v1.1 to be aware of: for performance reasons, Directory listings are no longer loaded by default. To control loading Directory listings, use "loadListing":https://www.commonwl.org/v1.1/CommandLineTool.html#CommandInputParameter or "LoadListingRequirement":https://www.commonwl.org/v1.1/CommandLineTool.html#LoadListingRequirement (the extension @cwltool:LoadListingRequirement@ is deprecated.)
-* You can get the designated temporary directory using @$(runtime.tmpdir)@ in your CWL file, or from the @$TMPDIR@ environment variable in your script.
+If a step requires network access, use "NetworkAccess":https://www.commonwl.org/v1.1/CommandLineTool.html#NetworkAccess instead of the Arvados-specific "arv:APIRequirement":cwl-extensions.html#APIRequirement .
-* Similarly, you can get the designated output directory using $(runtime.outdir), or from the @HOME@ environment variable in your script.
+To prevent misbehaving steps from running forever and wasting resources, you can fail the step if it exceeds a certain running time with "ToolTimeLimit":https://www.commonwl.org/v1.1/CommandLineTool.html#ToolTimeLimit instead of the deprecated @cwltool:TimeLimit@ .
-* Use @ExpressionTool@ to efficiently rearrange input files between steps of a Workflow. For example, the following expression accepts a directory containing files paired by @_R1_@ and @_R2_@ and produces an array of Directories containing each pair.
+To control if an individual step can be reused, use "WorkReuse":https://www.commonwl.org/v1.1/CommandLineTool.html#WorkReuse instead of the deprecated @arv:ReuseRequirement at .
-<pre>
+h2(#performance). Performance
+
+When combining a parameter value with a string, such as adding a filename extension, write @$(inputs.file.basename).ext@ instead of @$(inputs.file.basename + 'ext')@. The first form is evaluated as a simple text substitution, the second form (using the @+@ operator) is evaluated as an arbitrary Javascript expression and requires that you declare @InlineJavascriptRequirement at .
+
+Use @ExpressionTool@ to efficiently rearrange input files between steps of a Workflow. For example, the following expression accepts a directory containing files paired by @_R1_@ and @_R2_@ and produces an array of Directories containing each pair.
+
+{% codeblock as yaml %}
class: ExpressionTool
cwlVersion: v1.0
inputs:
@@ -83,43 +123,15 @@ expression: |
}
return {"out": dirs};
}
-</pre>
-
-* Avoid specifying resource requirements in CommandLineTool. Prefer to specify them in the workflow. You can provide a default resource requirement in the top level @hints@ section, and individual steps can override it with their own resource requirement.
-
-<pre>
-cwlVersion: v1.0
-class: Workflow
-inputs:
- inp: File
-hints:
- ResourceRequirement:
- ramMin: 1000
- coresMin: 1
- tmpdirMin: 45000
-steps:
- step1:
- in: {inp: inp}
- out: [out]
- run: tool1.cwl
- step2:
- in: {inp: step1/inp}
- out: [out]
- run: tool2.cwl
- hints:
- ResourceRequirement:
- ramMin: 2000
- coresMin: 2
- tmpdirMin: 90000
-</pre>
+{% endcodeblock %}
-* Available compute nodes types vary over time and across different cloud providers, so try to limit the RAM requirement to what the program actually needs. However, if you need to target a specific compute node type, see this discussion on "calculating RAM request and choosing instance type for containers.":{{site.baseurl}}/api/execution.html#RAM
+Available compute nodes types vary over time and across different cloud providers, so try to limit the RAM requirement to what the program actually needs. However, if you need to target a specific compute node type, see this discussion on "calculating RAM request and choosing instance type for containers.":{{site.baseurl}}/api/execution.html#RAM
-* Instead of scattering separate steps, prefer to scatter over a subworkflow.
+Instead of scattering separate steps, prefer to scatter over a subworkflow.
With the following pattern, @step1@ has to wait for all samples to complete before @step2@ can start computing on any samples. This means a single long-running sample can prevent the rest of the workflow from moving on:
-<pre>
+{% codeblock as yaml %}
cwlVersion: v1.0
class: Workflow
inputs:
@@ -140,12 +152,12 @@ steps:
scatter: inp
out: [out]
run: tool3.cwl
-</pre>
+{% endcodeblock %}
Instead, scatter over a subworkflow. In this pattern, a sample can proceed to @step2@ as soon as @step1@ is done, independently of any other samples.
Example: (note, the subworkflow can also be put in a separate file)
-<pre>
+{% codeblock as yaml %}
cwlVersion: v1.0
class: Workflow
steps:
@@ -174,12 +186,6 @@ steps:
in: {inp: step2/inp}
out: [out]
run: tool3.cwl
-</pre>
-
-h2(#migrate). Migrating running CWL on jobs API to containers API
-
-When migrating from jobs API (--api=jobs) (sometimes referred to as "crunch v1") to the containers API (--api=containers) ("crunch v2") there are a few differences in behavior:
+{% endcodeblock %}
-* A tool may fail to find an input file that could be found when run under the jobs API. This is because tools are limited to accessing collections explicitly listed in the input, and further limited to those individual files or subdirectories that are listed. For example, given an explicit file input @/dir/subdir/file1.txt@, a tool will not be allowed to implicitly access a file in the parent directory @/dir/file2.txt at . Use @secondaryFiles@ or a @Directory@ for files that need to be grouped together.
-* A tool may fail when attempting to rename or delete a file in the output directory. This may happen because files listed in @InitialWorkDirRequirement@ appear in the output directory as normal files (not symlinks) but cannot be moved, renamed or deleted unless marked as "writable" in CWL. These files will be added to the output collection but without any additional copies of the underlying data.
-* A tool may fail when attempting to access the network. This may happen because, unlike the jobs API, under the containers API network access is disabled by default. Tools which require network access should add @arv:APIRequirement: {}@ to the @requirements@ section.
+If you have a sequence of short-running steps (less than 1-2 minutes each), use the Arvados extension "arv:RunInSingleContainer":cwl-extensions.html#RunInSingleContainer to avoid scheduling and data transfer overhead by running all the steps together at once. To use this feature, @cwltool@ must be installed in the container image.
diff --git a/doc/user/tutorials/writing-cwl-workflow.html.textile.liquid b/doc/user/tutorials/writing-cwl-workflow.html.textile.liquid
index 2f1f80ca0..a69f0e6f3 100644
--- a/doc/user/tutorials/writing-cwl-workflow.html.textile.liquid
+++ b/doc/user/tutorials/writing-cwl-workflow.html.textile.liquid
@@ -15,9 +15,13 @@ SPDX-License-Identifier: CC-BY-SA-3.0
h2. Developing workflows
-For an introduction and and detailed documentation about writing CWL, see the "CWL User Guide":https://www.commonwl.org/user_guide and the "CWL Specification":http://commonwl.org/v1.0 .
+For an introduction and and detailed documentation about writing CWL, see the "CWL User Guide":https://www.commonwl.org/user_guide and the "CWL Specification":http://commonwl.org/v1.1 .
-See "Best Practices for writing CWL":{{site.baseurl}}/user/cwl/cwl-style.html and "Arvados CWL Extensions":{{site.baseurl}}/user/cwl/cwl-extensions.html for additional information about using CWL on Arvados.
+See "Writing Portable High-Performance Workflows":{{site.baseurl}}/user/cwl/cwl-style.html and "Arvados CWL Extensions":{{site.baseurl}}/user/cwl/cwl-extensions.html for additional information about using CWL on Arvados.
+
+See "Repositories of CWL Tools and Workflows":https://www.commonwl.org/#Repositories_of_CWL_Tools_and_Workflows for links to repositories of existing tools for reuse.
+
+See "Software for working with CWL":https://www.commonwl.org/#Software_for_working_with_CWL for links to software tools to help create CWL documents.
h2. Using Composer
@@ -203,3 +207,15 @@ arvados-cwl-runner 1.0.20160628195002, arvados-python-client 0.1.20160616015107,
}
</code></pre>
</notextile>
+
+h2(#migrate). Differences in running CWL on the legacy jobs API vs containers API
+
+Most users can ignore this section.
+
+When migrating your Arvados cluster from using the jobs API (--api=jobs) (sometimes referred to as "crunch v1") to the containers API (--api=containers) ("crunch v2") there are a few differences in behavior:
+
+A tool may fail to find an input file that could be found when run under the jobs API. This is because tools are limited to accessing collections explicitly listed in the input, and further limited to those individual files or subdirectories that are listed. For example, given an explicit file input @/dir/subdir/file1.txt@, a tool will not be allowed to implicitly access a file in the parent directory @/dir/file2.txt at . Use @secondaryFiles@ or a @Directory@ for files that need to be grouped together.
+
+A tool may fail when attempting to rename or delete a file in the output directory. This may happen because files listed in @InitialWorkDirRequirement@ appear in the output directory as normal files (not symlinks) but cannot be moved, renamed or deleted unless marked as "writable" in CWL. These files will be added to the output collection but without any additional copies of the underlying data.
+
+A tool may fail when attempting to access the network. This may happen because, unlike the jobs API, under the containers API network access is disabled by default. Tools which require network access should add @arv:APIRequirement: {}@ to the @requirements@ section.
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list