From peter.amstutz at curii.com Wed Oct 14 14:06:47 2020 From: peter.amstutz at curii.com (Peter Amstutz) Date: Wed, 14 Oct 2020 10:06:47 -0400 Subject: [arvados] Arvados v2.1.0 released Message-ID: The Arvados team is pleased to announce Arvados v2.1.0. This is a major upgrade, with many new features. Some older components have also been removed, superseded by newer components. Please also review the upgrade notes . A complete list of issues resolved in this release can be found on the developer site . Major Features Support for CWL 1.2 This release adds support for v1.2 of the Common Workflow Language standard, which adds conditional execution of workflow steps to the language among other features. Details can be found in the CWL v1.2 Changelog. Faster permission updates Adding or removing permissions now only updates users and groups that are directly affected, instead recomputing the permission table from scratch. For large databases, this dramatically improves the performance of updates such as adding a new project. S3 Compatible API for accessing files in Keep Alongside the existing WebDAV support, Keep-Web now provides an S3-compatible API for accessing files in Keep over HTTP. New S3 driver for Keep Historically, Arvados Keep has used the *goamz* driver to talk to S3-compatible services. This release adds a new driver based on *aws-sdk-go-v2*. The new driver can improve read performance by 50-100% but it has not had as much production use. See the S3 backend documentation for information on how to enable it. Many new Prometheus metrics for the Arvados cloud dispatcher Arvados-dispatch-cloud, the component that handles dynamic scaling of compute in cloud environments, now sports a number of new Prometheus metrics. The new metrics allow insight in container wait times, slowdowns due to cloud account quota, node boot outcomes, how long it takes for a compute node to become available, and more. Workbench 2 support for large collections Using Workbench 2, users can now browse collections of up to 50,000 files without significant performance degradation. SaltStack formulas for installing/managing Arvados The Arvados team is now publishing the SaltStack formulas that we use to install and manage our own Arvados clusters in order to make deployment of Arvados easier. SaltStack is a Python-based, open-source devops tool for event-driven IT automation, remote task execution, and configuration management, similar to Ansible or Puppet. Support for OpenID Connect Arvados now supports using the OpenID Connect protocol to authenticate users. Expanded PAM support Arvados now supports the use Portable Authentication Modules (PAM) in two ways: - For users of shell nodes, PAM makes it possible to log in with an Arvados username and use an Arvados token as a password. - For log in to Arvados Workbench, use PAM to validate the username and password (by default, local Unix accounts on the API server). Webshell Log in to Arvados shell nodes from Workbench via browser-based ShellInABox has been updated and is now documented . Arv-copy improvements The arv-copy tool can now copy projects and the collections, workflows, and subprojects they contain. arv-copy now also infers the source cluster from the uuid, and uses the current Arvados environment as the default destination cluster. Other changes The “devise-omniauth” SSO server is no longer supported Authentication methods have moved into the Arvados Controller component. Arvados installations using the “devise-omniauth” SSO server will continue to work, but cluster admins should plan to migrate to one of the built-in authentication methods as soon as possible. See setting up web based login for details. The “devise-omniauth” SSO server authentication flow is deprecated and will be removed in the next major release. The old PAM driver written in Python has been removed It has been replaced by a new driver written in Go. The distribution package name for the new driver is *libpam-arvados-go*. When upgrading, the PAM configuration will need to be tweaked, see the documentation for details. NodeManager is no longer supported or packaged NodeManager has been removed entirely from this release, having been superseded by arvados-dispatch-cloud since the Arvados 2.0 release. Python 2.7 is no longer supported Minimum supported Python version is now 3.5. Ruby 2.5 or higher required The minimum supported Ruby version is now 2.5. Debian 9 (“stretch”) support deprecated The Arvados 2.1 release series will be the last with support for Debian 9. Thanks, The Arvados Team -------------- next part -------------- An HTML attachment was scrubbed... URL: