[ARVADOS] updated: 2.1.0-1544-g83c996d75
Git user
git at public.arvados.org
Wed Oct 27 18:34:56 UTC 2021
Summary of changes:
doc/_config.yml | 13 +-
.../_container_scheduling_parameters.liquid | 2 +-
doc/_includes/_install_custom_certificates.liquid | 26 ++++
.../_singularity_mksquashfs_configuration.liquid | 15 ++
doc/admin/upgrading.html.textile.liquid | 12 +-
doc/api/execution.html.textile.liquid | 2 +-
.../methods/container_requests.html.textile.liquid | 2 +-
doc/architecture/singularity.html.textile.liquid | 35 +++++
.../arvados-on-kubernetes-GKE.html.textile.liquid | 2 +-
.../install-compute-node.html.textile.liquid | 17 ++-
.../install-dispatch-cloud.html.textile.liquid | 2 +-
.../install-dispatch.html.textile.liquid | 13 +-
.../configure-slurm.html.textile.liquid | 8 +-
.../install-dispatch.html.textile.liquid | 8 +-
.../crunch2-slurm/install-test.html.textile.liquid | 4 +-
...nstall-compute-node-docker.html.textile.liquid} | 13 +-
...ll-compute-node-singularity.html.textile.liquid | 56 ++++++++
doc/install/index.html.textile.liquid | 2 +-
doc/install/salt-multi-host.html.textile.liquid | 4 +-
doc/install/salt-single-host.html.textile.liquid | 6 +
doc/install/singularity.html.textile.liquid | 49 -------
doc/sdk/index.html.textile.liquid | 2 +-
doc/user/topics/arv-docker.html.textile.liquid | 2 +-
lib/config/config.default.yml | 29 ++--
lib/config/generated_config.go | 29 ++--
lib/crunchrun/crunchrun.go | 5 +-
lib/crunchrun/crunchrun_test.go | 14 +-
lib/lsf/dispatch.go | 41 ++++--
lib/lsf/dispatch_test.go | 9 +-
sdk/cwl/arvados_cwl/executor.py | 2 +-
sdk/cwl/setup.py | 4 +-
.../api/lib/tasks/manage_long_lived_tokens.rake | 4 +-
services/api/script/get_anonymous_user_token.rb | 3 +
services/login-sync/bin/arvados-login-sync | 37 ++++-
tools/compute-images/arvados-images-aws.json | 3 +-
tools/compute-images/arvados-images-azure.json | 3 +-
tools/compute-images/build.sh | 12 +-
tools/compute-images/scripts/base.sh | 5 +
tools/salt-install/Vagrantfile | 8 +-
.../letsencrypt_controller_configuration.sls | 10 +-
.../letsencrypt_keepproxy_configuration.sls | 10 +-
.../pillars/letsencrypt_keepweb_configuration.sls | 11 --
.../pillars/letsencrypt_webshell_configuration.sls | 8 --
.../letsencrypt_websocket_configuration.sls | 10 +-
.../letsencrypt_workbench2_configuration.sls | 8 --
.../letsencrypt_workbench_configuration.sls | 8 --
.../aws/pillars/nginx_api_configuration.sls | 2 +-
.../pillars/nginx_collections_configuration.sls} | 27 ++--
.../aws/pillars/nginx_controller_configuration.sls | 11 +-
.../aws/pillars/nginx_download_configuration.sls} | 31 ++---
.../aws/pillars/nginx_keepproxy_configuration.sls | 11 +-
.../aws/pillars/nginx_keepweb_configuration.sls | 75 +---------
.../aws/pillars/nginx_webshell_configuration.sls | 11 +-
.../aws/pillars/nginx_websocket_configuration.sls | 9 +-
.../aws/pillars/nginx_workbench2_configuration.sls | 9 +-
.../aws/pillars/nginx_workbench_configuration.sls | 9 +-
.../multiple_hostnames/pillars/arvados.sls | 4 +-
.../pillars/nginx_controller_configuration.sls | 9 +-
.../pillars/nginx_keepproxy_configuration.sls | 5 +-
.../pillars/nginx_keepweb_configuration.sls | 19 ++-
.../multiple_hostnames/pillars/nginx_passenger.sls | 4 -
.../pillars/nginx_webshell_configuration.sls | 5 +-
.../pillars/nginx_websocket_configuration.sls | 5 +-
.../pillars/nginx_workbench2_configuration.sls | 5 +-
.../pillars/nginx_workbench_configuration.sls | 5 +-
.../multiple_hostnames/states/custom_certs.sls | 31 +++++
.../multiple_hostnames/states/snakeoil_certs.sls | 116 +++++++++-------
.../single_hostname/pillars/arvados.sls | 2 +-
.../local.params.example.multiple_hosts | 22 ++-
...l.params.example.single_host_multiple_hostnames | 21 ++-
tools/salt-install/provision.sh | 153 ++++++++++++++++++---
71 files changed, 726 insertions(+), 443 deletions(-)
create mode 100644 doc/_includes/_install_custom_certificates.liquid
create mode 100644 doc/_includes/_singularity_mksquashfs_configuration.liquid
create mode 100644 doc/architecture/singularity.html.textile.liquid
rename doc/install/{crunch2-slurm/install-compute-node.html.textile.liquid => crunch2/install-compute-node-docker.html.textile.liquid} (56%)
create mode 100644 doc/install/crunch2/install-compute-node-singularity.html.textile.liquid
delete mode 100644 doc/install/singularity.html.textile.liquid
copy tools/salt-install/config_examples/{single_host/multiple_hostnames/pillars/nginx_keepweb_configuration.sls => multi_host/aws/pillars/nginx_collections_configuration.sls} (65%)
copy tools/salt-install/config_examples/{single_host/multiple_hostnames/pillars/nginx_keepweb_configuration.sls => multi_host/aws/pillars/nginx_download_configuration.sls} (57%)
create mode 100644 tools/salt-install/config_examples/single_host/multiple_hostnames/states/custom_certs.sls
via 83c996d75698093446fbfff89ea4abeb36cbc8c4 (commit)
via 623ffe7320d0a159ba9a5912fc1948cac12ad505 (commit)
via aaabfc20e05a635a36be0049b38b2034f041398d (commit)
via 1f23032c8a10c03e9f4ff00b45576ad5c5e5afd9 (commit)
via e31979e7ef752c9adef3d3fa6096a47d720d524e (commit)
via 22516d3663a3c11384824dad0e052dc0630f08f0 (commit)
via 39464fc833e3ee2fb771f83dce9f94e3856c1075 (commit)
via 125dd3b1e5f34156b002951f504f5346cc5d7e8d (commit)
via 2582dc22a24ee7cdaf1a68c6b4b1c639f88c2efe (commit)
via c5e4fb5838d2f447ae126159a71340b90cfea33c (commit)
via 3dcd4659f10710621f429066f11ab7b67cd6e878 (commit)
via 01698bea4703ce073425e2080c7cad83e2f873cc (commit)
via 9f6f07fe6790e7c3a8f1b57990c16447c9d2713f (commit)
via 58df9c45cc6288b4424c6571b35373de8837a85f (commit)
via b03d26d7607c11e45f49a960ce147bbb2ebfb53c (commit)
via 169a7755916e1f3227ab9c3959447149ecb99503 (commit)
via 3c3cae35fe5b3cf0371e1cbb1ff295baf555f77a (commit)
via 9539317a22d8ea16f94b0e086507ab595d758216 (commit)
via 67d842388bc0542a7c3d8843635c3ccf5add20c7 (commit)
via dd5a2a0c2da55c95cb94cb67538c6693cf989bf9 (commit)
via 242e740db430759c3a09de3bf9ece89987c6b9b0 (commit)
via 332c20ac4de0d2706a7e6c9e3a96f1d4287eb4c5 (commit)
via 0bfda57681195c431e3b6063577fdab23ff40cd5 (commit)
via e1c4967befc7b4dd273b3d9d047a4e4262f5ba2f (commit)
via 0d9bc75b61d5227917ef43e4c716ae284d548644 (commit)
via ccb603fe5a8ca989d6db97cc723ccfcaba2781f5 (commit)
via 64194cf50545e1b6894b45cc18edbefbc3ac8707 (commit)
via 93131b473366ae5442103511d455d1ed5b73bcab (commit)
via b98969108fbda1754a03665a5be58bb9e0d16c01 (commit)
via 8fde01ca2dae5169917de4e8b25eada543a8223f (commit)
via f9582f44d4f31e941902f84fce760a8b770bc969 (commit)
via d68c3776fee61329f665ede740cca51946395d1c (commit)
via 089aaca1ed6fe672f01c5049f1bf3956bdd35065 (commit)
via d8d9b01ba4a5f9cbc216c43f8af88af1413673d4 (commit)
via f54cc984969657be50c093b917feb49a19d78c22 (commit)
via 3b0f80205c1942cc954eb891691c7c382aa9c87c (commit)
via 2eeaf4eacf23e166b60cf95562ea2727a68e1e08 (commit)
via 6f84dbb74b46470e937a52cbbb5de4c5b825e122 (commit)
via 03338fd9d7416560331bb96fd356f3d35de6760f (commit)
via b4bb4815b9e23c54d6bbf7ec166acdf7ddfcb10b (commit)
from 68259bcde57277cb709296fc24e86826d9c131d5 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
commit 83c996d75698093446fbfff89ea4abeb36cbc8c4
Author: Tom Clegg <tom at curii.com>
Date: Wed Oct 27 14:34:47 2021 -0400
16347: Add upgrade note.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom at curii.com>
diff --git a/doc/admin/upgrading.html.textile.liquid b/doc/admin/upgrading.html.textile.liquid
index 399ec35d2..82e56617a 100644
--- a/doc/admin/upgrading.html.textile.liquid
+++ b/doc/admin/upgrading.html.textile.liquid
@@ -35,7 +35,17 @@ TODO: extract this information based on git commit messages and generate changel
<div class="releasenotes">
</notextile>
-h2(#main). development main (as of 2021-09-07)
+h2(#main). development main (as of 2021-10-27)
+
+"Upgrading from 2.3.0":#v2_3_0
+
+h3. Dedicated keepstore process for each container
+
+When Arvados runs a container via @arvados-dispatch-cloud@, the @crunch-run@ supervisor process now brings up its own keepstore server to handle I/O for mounted collections, outputs, and logs. With the default configuration, the keepstore process allocates one 64 MiB block buffer per VCPU requested by the container. For most workloads this will increase throughput, reduce total network traffic, and make it possible to run more containers at once without provisioning additional keepstore nodes to handle the I/O load.
+* If you have containers that can effectively handle multiple I/O threads per VCPU, consider either increasing the @Containers.LocalKeepBlobBuffersPerVCPU@ value, or setting it to 0 to disable this feature and preserve the previous behavior of sending container I/O traffic to your separately provisioned keepstore servers.
+* This feature is enabled only if no volumes use @AccessViaHosts@, and no volumes have underlying @Replication@ less than @Collections.DefaultReplication at . If the feature is configured but cannot be enabled due to an incompatible volume configuration, this will be noted in the @crunch-run.txt@ file in the container log.
+
+h2(#v2_3_0). v2.3.0 (2021-10-27)
"Upgrading from 2.2.0":#v2_2_0
commit 623ffe7320d0a159ba9a5912fc1948cac12ad505
Merge: 68259bcde aaabfc20e
Author: Tom Clegg <tom at curii.com>
Date: Wed Oct 27 14:12:56 2021 -0400
16347: Merge branch 'main'
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom at curii.com>
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list