[arvados] Arvados 2.5.0 released

Thu Dec 22 19:39:00 UTC 2022

Hello ,

The Arvados team is pleased to announce Arvados 2.5.0. This is a major
upgrade, with many new features, changes and bugfixes. We recommend that
new and existing installations of 2.4.4
<https://www.arvados.org/release-notes/2.4.4> or earlier upgrade to 2.5.0.
See Upgrading Arvados <https://doc.arvados.org/v2.5/admin/upgrading.html>
for upgrade instructions.  For a complete list of changes, visit the release
notes <https://www.arvados.org/release-notes/2.5.0/>.
New Features Workbench 2

Workbench 2 features a new color scheme, designed for improved contrast and
consistency. #19462 <https://dev.arvados.org/issues/19462>

The process pages features new Input and Output panels displaying the input
and output parameters of the process. #16073
<https://dev.arvados.org/issues/16073> #19700
<https://dev.arvados.org/issues/19700> #19848
<https://dev.arvados.org/issues/19848>

<https://www.arvados.org/images/release-notes/v2.5/input-output.png>

The process page also features a new Resources panel showing what resources
were requested by the container, and (on cloud installs) what instance type
it was actually run on. #19438 <https://dev.arvados.org/issues/19438>

<https://www.arvados.org/images/release-notes/v2.5/resources.png>

The picking dialog now includes the ability to search for projects and to
filter the list of collections within a project. #19783
<https://dev.arvados.org/issues/19783>

<https://www.arvados.org/images/release-notes/v2.5/picker-search.png>

The process panel includes new information about how much a workflow cost
to run. #19319 <https://dev.arvados.org/issues/19319>

<https://www.arvados.org/images/release-notes/v2.5/container-cost.png>

When you view a workflow step or output collection in Workbench 2, there’s
now a row of breadcrumbs at the top of the page that shows you the parent
workflow and project. This helps users understand how the object was
created, and navigate to those parent objects if desired. #19504
<https://dev.arvados.org/issues/19504>

<https://www.arvados.org/images/release-notes/v2.5/breadcrumbs.png>

The project view now offers the ability to configure a variety of
additional columns for display: UUID, Created at/Trash at/Delete at,
Properties, Portable Data Hash, Version, File Count, Requesting Container
UUID, Container UUID, Output UUID, Log UUID, and Modified By User UUID.
#19690 <https://dev.arvados.org/issues/19690>

<https://www.arvados.org/images/release-notes/v2.5/new-columns.png>

Workbench 2’s left navigation pane is now collapsible. #19434
<https://dev.arvados.org/issues/19434>

<https://www.arvados.org/images/release-notes/v2.5/collapse-LHS.png>

Workbench 2 fully supports frozen projects: projects can be frozen and
unfrozen, and frozen projects can’t be edited in any way. #18692
<https://dev.arvados.org/issues/18692>

When you maximize a Workbench 2 panel, there is now an un-maximize button
to easily restore the previous panels. #19300
<https://dev.arvados.org/issues/19300>
Installer

The Arvados installer includes a new script to manage the process of
setting up a fully-featured multi-node cluster. The multi host install guide
<https://doc.arvados.org/v2.5/install/salt-multi-host.html> guide has been
rewritten, updated, and expanded. In addition, we now provide a Terraform
script to assist in deploying on AWS. #19175
<https://dev.arvados.org/issues/19175> #19215
<https://dev.arvados.org/issues/19215>
Client tools

arv-mount now features a memory-mapped disk-backed cache instead of a pure
RAM cache. This enables the cache to be much bigger by default and leverage
the kernel’s filesystem cache without competing with user programs for RAM.
The result is same or better performance reading files without needing to
tune cache parameters. This is available both running arv-mount locally and
running containers on Arvados. The size of the cache is controlled with
--file-cache like before, and the location is controlled with
--disk-cache-dir. #18842 <https://dev.arvados.org/issues/18842> #19872
<https://dev.arvados.org/issues/19872>

arvados-client shell
<https://doc.arvados.org/v2.5/user/debugging/container-shell-access.html>
can now connect to containers running under LSF and SLURM, in addition to
the cloud dispatcher. #19166 <https://dev.arvados.org/issues/19166>
Running Workflows

Container records now have cost and subrequest_cost fields with information
about the container’s cost to run. See the containers reference
<https://doc.arvados.org/v2.5/api/methods/containers.html> for details.
Container request records now have a cumulative_cost field with information
about the request’s total cost to run. See the container requests reference
<https://doc.arvados.org/v2.5/api/methods/container_requests.html> for
details. This is calculated using the Price field of the instance type
definition in the configuration file. #18205
<https://dev.arvados.org/issues/18205>

Workflow steps submitted by arvados-cwl-runner now have properties cwl_input
and cwl_output on the container request. See the API properties
documentation <https://doc.arvados.org/v2.5/api/properties.html> for
details. #19466 <https://dev.arvados.org/issues/19466>
SDK

The Python SDK documentation now includes a full overview of how the API
client works <https://doc.arvados.org/v2.5/sdk/python/api-client.html>,
with lots of realistic examples for demonstration. #19791
<https://dev.arvados.org/issues/19791>

The Python SDK features a disk-backed cache for Keep. This is used by
arv-mount as discussed above, but also available to other users of the
Python SDK. #18842 <https://dev.arvados.org/issues/18842> #19872
<https://dev.arvados.org/issues/19872>
Administrator tools

Arvados includes a new tool sync-users-tool to synchronize Arvados user
records described by a CSV file. You can use this to keep Arvados user
records up-to-date with other identity sources like Active Directory or SSO
providers. The admin guide on synchronizing
<https://doc.arvados.org/v2.5/user/topics/arvados-sync-external-sources.html>
has more details. #18858 <https://dev.arvados.org/issues/18858>

arvados-client diagnostics
<https://doc.arvados.org/v2.5/admin/diagnostics.html> is now documented in
the admin guide. When possible, the diagnostics also performs a cluster
“health check” (arvados-server check described below) and includes any
problems in the diagnostics report. #19364
<https://dev.arvados.org/issues/19364> #19377
<https://dev.arvados.org/issues/19377>

arvados-server check <https://doc.arvados.org/v2.5/admin/health-checks.html>
is now documented in the admin guide. The arvados-server check tool now
detects and reports several likely error conditions on a cluster: when
different services in the cluster are not using the same configuration;
different services in the cluster are running from different versions of
Arvados; when clock times reported by different services in the cluster are
a minute apart or more. #18794 <https://dev.arvados.org/issues/18794>
Arvados API

The API server respects a new configuration option Users.CanCreateRoleGroups.
If you set this false, non-admin users will not be allowed to create role
groups. This is helpful to set when your cluster uses arvados-group-sync as
the single source of roles. #19513 <https://dev.arvados.org/issues/19513>

The API server now allows sharing objects to be “writable” by the “All
Users” group. This entailed some changes to the permissions model for
roles: in general, updating a role now requires can_manage permission.
Refer to the API permissions model documentation
<https://doc.arvados.org/v2.5/api/permission-model.html> for details. #19269
<https://dev.arvados.org/issues/19269>

Group objects returned by the API server now include boolean can_manage and
can_write fields indicating the current user’s level of access. The
writeable_by field is now deprecated in favor of these new fields. #19146
<https://dev.arvados.org/issues/19146>
Keep

If an unauthenticated user visits keep-web they will now be redirected to
Workbench 2 to log in first. Once the user is logged in, they will be
redirected back to keep-web. #17807 <https://dev.arvados.org/issues/17807>

Keepstore now uses a new driver for S3 driver by default. This driver
features improved S3 read performance. The old v1 driver will be removed in
Arvados 2.6.0. #19582 <https://dev.arvados.org/issues/19582>

The keep-web S3 API now publishes project and collection properties through
the X-Amz-Meta- headers. Control characters in values are MIME encoded.
Refer to the new section in our S3 API documentation
<https://doc.arvados.org/v2.5/api/keep-s3.html> for details. #19088
<https://dev.arvados.org/issues/19088> #19249
<https://dev.arvados.org/issues/19249>

Thanks,

The Arvados Team
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.arvados.org/pipermail/arvados/attachments/20221222/30de54db/attachment.html>