[ARVADOS] updated: 2.1.0-1429-ge91291c29
Git user
git at public.arvados.org
Mon Oct 4 19:26:15 UTC 2021
Summary of changes:
doc/_includes/_install_ruby_and_bundler.liquid | 37 ++++++++++----------------
sdk/cwl/tests/wf/runin-reqs-wf.cwl | 2 +-
sdk/cwl/tests/wf/runin-reqs-wf2.cwl | 2 +-
sdk/cwl/tests/wf/runin-reqs-wf4.cwl | 2 +-
sdk/cwl/tests/wf/runin-reqs-wf5.cwl | 2 +-
sdk/cwl/tests/wf/runin-wf.cwl | 2 +-
sdk/go/arvados/fs_project_test.go | 24 +++++++++++++++++
sdk/go/arvados/fs_site_test.go | 2 ++
services/api/app/models/group.rb | 4 +--
services/api/test/fixtures/groups.yml | 26 ++++++++++++++++++
tools/arvbox/bin/arvbox | 1 +
11 files changed, 74 insertions(+), 30 deletions(-)
discards 5898e36e5edc2850f3bf5ee4667676e37239c710 (commit)
discards 01d3af33556013452117fceeda31bc29afd940cd (commit)
discards a0352b2102ff74013893120d1267d5a38a0e4136 (commit)
discards 83492ceff77529aa93e833e5df6b9414eb21c3b1 (commit)
discards 1e56e730b6e819ec831229c1fceedcbef782f59f (commit)
via e91291c2959c07c9929617e3d080077477a19b33 (commit)
via 8d8ef7cca4001feb24eae52a9d458cafbf2de4fb (commit)
via 04ea44bd67f6b0c2a700e6e9403629ad842e148d (commit)
via 1b15db60a18e6bbb09a07e230b8f48630e05cdc1 (commit)
via a3b5825a4fd71f2b8fbfdd288655f62af0816ca3 (commit)
via 69b24ec1999603af6d8f36b083a9ce2fac9bd72b (commit)
via 4b06eb378afc84b31e0986283fb7f807adb09a9b (commit)
via ff05fbdf1fb4d68d519840d6f2007cf24064395f (commit)
via a4c2309a118912e43469235c0dedef3a9f5dbfa7 (commit)
via 9ce4b2c07e2b7809088e6333246501ad9418bbda (commit)
via cfe9b58cc374794df00910b7708ffa245aaf03bf (commit)
via 80889a19da3aacccc3ebcdfba73d037a54794b98 (commit)
This update added new revisions after undoing existing revisions. That is
to say, the old revision is not a strict subset of the new revision. This
situation occurs when you --force push a change and generate a repository
containing something like this:
* -- * -- B -- O -- O -- O (5898e36e5edc2850f3bf5ee4667676e37239c710)
\
N -- N -- N (e91291c2959c07c9929617e3d080077477a19b33)
When this happens we assume that you've already had alert emails for all
of the O revisions, and so we here report only the revisions in the N
branch from the common base, B.
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
commit e91291c2959c07c9929617e3d080077477a19b33
Author: Peter Amstutz <peter.amstutz at curii.com>
Date: Mon Oct 4 15:25:32 2021 -0400
18238: Update tests to use newer arvados/jobs image
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz at curii.com>
diff --git a/sdk/cwl/tests/wf/runin-reqs-wf.cwl b/sdk/cwl/tests/wf/runin-reqs-wf.cwl
index 3cc52936e..22cc82b7f 100644
--- a/sdk/cwl/tests/wf/runin-reqs-wf.cwl
+++ b/sdk/cwl/tests/wf/runin-reqs-wf.cwl
@@ -24,7 +24,7 @@ requirements:
StepInputExpressionRequirement: {}
hints:
DockerRequirement:
- dockerPull: arvados/jobs:1.4.0.20190604172024
+ dockerPull: arvados/jobs:2.2.2
steps:
substep:
in:
diff --git a/sdk/cwl/tests/wf/runin-reqs-wf2.cwl b/sdk/cwl/tests/wf/runin-reqs-wf2.cwl
index 7d06cb308..4bde6c562 100644
--- a/sdk/cwl/tests/wf/runin-reqs-wf2.cwl
+++ b/sdk/cwl/tests/wf/runin-reqs-wf2.cwl
@@ -24,7 +24,7 @@ requirements:
StepInputExpressionRequirement: {}
hints:
DockerRequirement:
- dockerPull: arvados/jobs:1.4.0.20190604172024
+ dockerPull: arvados/jobs:2.2.2
steps:
substep:
in:
diff --git a/sdk/cwl/tests/wf/runin-reqs-wf4.cwl b/sdk/cwl/tests/wf/runin-reqs-wf4.cwl
index 9a26d0113..d00ee8577 100644
--- a/sdk/cwl/tests/wf/runin-reqs-wf4.cwl
+++ b/sdk/cwl/tests/wf/runin-reqs-wf4.cwl
@@ -24,7 +24,7 @@ requirements:
StepInputExpressionRequirement: {}
hints:
DockerRequirement:
- dockerPull: arvados/jobs:1.4.0.20190604172024
+ dockerPull: arvados/jobs:2.2.2
steps:
substep:
in:
diff --git a/sdk/cwl/tests/wf/runin-reqs-wf5.cwl b/sdk/cwl/tests/wf/runin-reqs-wf5.cwl
index 34d7b2c39..647b07edf 100644
--- a/sdk/cwl/tests/wf/runin-reqs-wf5.cwl
+++ b/sdk/cwl/tests/wf/runin-reqs-wf5.cwl
@@ -24,7 +24,7 @@ requirements:
StepInputExpressionRequirement: {}
hints:
DockerRequirement:
- dockerPull: arvados/jobs:1.4.0.20190604172024
+ dockerPull: arvados/jobs:2.2.2
steps:
substep:
in:
diff --git a/sdk/cwl/tests/wf/runin-wf.cwl b/sdk/cwl/tests/wf/runin-wf.cwl
index 68a26a0d3..f819d0fe6 100644
--- a/sdk/cwl/tests/wf/runin-wf.cwl
+++ b/sdk/cwl/tests/wf/runin-wf.cwl
@@ -36,7 +36,7 @@ steps:
hints:
- class: arv:RunInSingleContainer
- class: DockerRequirement
- dockerPull: arvados/jobs:1.4.0.20190604172024
+ dockerPull: arvados/jobs:2.2.2
run:
class: Workflow
id: mysub
commit 8d8ef7cca4001feb24eae52a9d458cafbf2de4fb
Author: Peter Amstutz <peter.amstutz at curii.com>
Date: Fri Oct 1 16:31:06 2021 -0400
18238: Fix API revision check
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz at curii.com>
diff --git a/sdk/cwl/arvados_cwl/arvcontainer.py b/sdk/cwl/arvados_cwl/arvcontainer.py
index 165cbcf18..560e8979e 100644
--- a/sdk/cwl/arvados_cwl/arvcontainer.py
+++ b/sdk/cwl/arvados_cwl/arvcontainer.py
@@ -284,7 +284,7 @@ class ArvadosContainer(JobBase):
raise WorkflowException("Invalid value %d for output_ttl, cannot be less than zero" % container_request["output_ttl"])
- if api._rootDesc["revision"] >= "20210628":
+ if self.arvrunner.api._rootDesc["revision"] >= "20210628":
storage_class_req, _ = self.get_requirement("http://arvados.org/cwl#OutputStorageClass")
if storage_class_req and storage_class_req.get("intermediateStorageClass"):
container_request["output_storage_classes"] = aslist(storage_class_req["intermediateStorageClass"])
commit 04ea44bd67f6b0c2a700e6e9403629ad842e148d
Author: Peter Amstutz <peter.amstutz at curii.com>
Date: Fri Oct 1 10:26:42 2021 -0400
18238: Check API version before using output_storage_classes
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz at curii.com>
diff --git a/sdk/cwl/arvados_cwl/arvcontainer.py b/sdk/cwl/arvados_cwl/arvcontainer.py
index 1e79566f4..165cbcf18 100644
--- a/sdk/cwl/arvados_cwl/arvcontainer.py
+++ b/sdk/cwl/arvados_cwl/arvcontainer.py
@@ -283,11 +283,13 @@ class ArvadosContainer(JobBase):
if self.output_ttl < 0:
raise WorkflowException("Invalid value %d for output_ttl, cannot be less than zero" % container_request["output_ttl"])
- storage_class_req, _ = self.get_requirement("http://arvados.org/cwl#OutputStorageClass")
- if storage_class_req and storage_class_req.get("intermediateStorageClass"):
- container_request["output_storage_classes"] = aslist(storage_class_req["intermediateStorageClass"])
- else:
- container_request["output_storage_classes"] = runtimeContext.intermediate_storage_classes.strip().split(",")
+
+ if api._rootDesc["revision"] >= "20210628":
+ storage_class_req, _ = self.get_requirement("http://arvados.org/cwl#OutputStorageClass")
+ if storage_class_req and storage_class_req.get("intermediateStorageClass"):
+ container_request["output_storage_classes"] = aslist(storage_class_req["intermediateStorageClass"])
+ else:
+ container_request["output_storage_classes"] = runtimeContext.intermediate_storage_classes.strip().split(",")
if self.timelimit is not None and self.timelimit > 0:
scheduling_parameters["max_run_time"] = self.timelimit
diff --git a/services/api/app/controllers/arvados/v1/schema_controller.rb b/services/api/app/controllers/arvados/v1/schema_controller.rb
index 6c5472624..c1d4b74d6 100644
--- a/services/api/app/controllers/arvados/v1/schema_controller.rb
+++ b/services/api/app/controllers/arvados/v1/schema_controller.rb
@@ -37,7 +37,7 @@ class Arvados::V1::SchemaController < ApplicationController
# format is YYYYMMDD, must be fixed width (needs to be lexically
# sortable), updated manually, may be used by clients to
# determine availability of API server features.
- revision: "20210503",
+ revision: "20210628",
source_version: AppVersion.hash,
sourceVersion: AppVersion.hash, # source_version should be deprecated in the future
packageVersion: AppVersion.package_version,
commit 1b15db60a18e6bbb09a07e230b8f48630e05cdc1
Author: Peter Amstutz <peter.amstutz at curii.com>
Date: Fri Oct 1 10:02:40 2021 -0400
18238: Make backwards compatible when StorageClasses is not in config
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz at curii.com>
diff --git a/sdk/cwl/arvados_cwl/executor.py b/sdk/cwl/arvados_cwl/executor.py
index edb9d5b52..db1db2816 100644
--- a/sdk/cwl/arvados_cwl/executor.py
+++ b/sdk/cwl/arvados_cwl/executor.py
@@ -549,7 +549,7 @@ The 'jobs' API is no longer supported.
if runtimeContext.submit_request_uuid and self.work_api != "containers":
raise Exception("--submit-request-uuid requires containers API, but using '{}' api".format(self.work_api))
- default_storage_classes = ",".join([k for k,v in self.api.config()["StorageClasses"].items() if v.get("Default") is True])
+ default_storage_classes = ",".join([k for k,v in self.api.config().get("StorageClasses", {"default": {"Default": True}}).items() if v.get("Default") is True])
if runtimeContext.storage_classes == "default":
runtimeContext.storage_classes = default_storage_classes
if runtimeContext.intermediate_storage_classes == "default":
commit a3b5825a4fd71f2b8fbfdd288655f62af0816ca3
Author: Peter Amstutz <peter.amstutz at curii.com>
Date: Thu Sep 30 16:13:28 2021 -0400
18238: Add required container_engine parameter to builder
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz at curii.com>
diff --git a/sdk/cwl/arvados_cwl/runner.py b/sdk/cwl/arvados_cwl/runner.py
index 66dff809e..126a2b511 100644
--- a/sdk/cwl/arvados_cwl/runner.py
+++ b/sdk/cwl/arvados_cwl/runner.py
@@ -105,7 +105,8 @@ def make_builder(joborder, hints, requirements, runtimeContext, metadata):
outdir="", # type: Text
tmpdir="", # type: Text
stagedir="", # type: Text
- cwlVersion=metadata.get("http://commonwl.org/cwltool#original_cwlVersion") or metadata.get("cwlVersion")
+ cwlVersion=metadata.get("http://commonwl.org/cwltool#original_cwlVersion") or metadata.get("cwlVersion"),
+ container_engine="docker"
)
def search_schemadef(name, reqs):
commit 69b24ec1999603af6d8f36b083a9ce2fac9bd72b
Author: Peter Amstutz <peter.amstutz at curii.com>
Date: Thu Sep 30 10:18:12 2021 -0400
18238: Update cwltool and schema-salad deps
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz at curii.com>
diff --git a/sdk/cwl/setup.py b/sdk/cwl/setup.py
index 3f1f8a6be..0bde76e92 100644
--- a/sdk/cwl/setup.py
+++ b/sdk/cwl/setup.py
@@ -39,8 +39,8 @@ setup(name='arvados-cwl-runner',
# file to determine what version of cwltool and schema-salad to
# build.
install_requires=[
- 'cwltool==3.1.20210816212154',
- 'schema-salad==8.2.20210902094147',
+ 'cwltool==3.1.20210922203925',
+ 'schema-salad==8.2.20210918131710',
'arvados-python-client{}'.format(pysdk_dep),
'setuptools',
'ciso8601 >= 2.0.0',
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list