[arvados-dev] Arvados v1.4 announcement and release notes

Thu Jun 6 06:05:00 UTC 2019

The Arvados team is pleased to announce our latest release Arvados v1.4
which includes several new features, the most exciting of which is the
brand new implementation of the Arvados web UI component, Workbench. It is
currently still in beta, but we’d love for you to test it as part of your
daily work and provide us with feedback.

Other highlights include:

   -

   New Workbench - React-based single page web app with completely new UI
   design
   -

   New Java SDK with much more complete API coverage
   -

   API Server & traditional Workbench upgraded to Rails 5
   -

   Support for Prometheus metrics reporting in a number of components
   -

   Packaging changes for Arvados utilities to improve isolation of
   dependencies
   -

   Support for Ubuntu 18.04
   -

   More components ported to Python 3.x and Python 3.7 support added
   -

   More than 50 bug fixes

There are a couple of important release notes
<https://doc.arvados.org/admin/upgrading.html> that you should review
before upgrading. Installing the python-arvados-python-client distribution
package no longer installs the Arvados Python module (ie arvados) system-wide.
If you are depending on its availability, you will need to install
arvados-python-client
from PyPI. Using virtualenv or similar dependency manager is a good way to
isolate dependency requirements of different applications.

As we’ve announced previously v1.4 is the last release to support the Jobs
API (aka Crunch 1). If you haven’t already migrated to Crunch 2 and the
Container API, you should make plans to as soon as possible. Please contact
us if you have any questions or require assistance.

You can find a full list of the features and bug fixes on the release page
<https://dev.arvados.org/rb/release/15> and included below.

#

Subject

12958 <https://dev.arvados.org/issues/12958>

[Federation] Workbench login chooser

14291 <https://dev.arvados.org/issues/14291>

[crunch-dispatch-cloud] AWS driver

14322 <https://dev.arvados.org/issues/14322>

[CWL] Accept collection uuid in input

14324 <https://dev.arvados.org/issues/14324>

[crunch-dispatch-cloud] Azure driver

14325 <https://dev.arvados.org/issues/14325>

[crunch-dispatch-cloud] Dispatch containers to cloud VMs directly, without
slurm or nodemanager

14573 <https://dev.arvados.org/issues/14573>

[Spike] [API] Fully functional filename search

14718 <https://dev.arvados.org/issues/14718>

[API] Option to issue salted token in login procedure

14720 <https://dev.arvados.org/issues/14720>

[Federation] Workbench2 login chooser

14819 <https://dev.arvados.org/issues/14819>

Upgrade the arvados/jobs image to Debian Stretch

14853 <https://dev.arvados.org/issues/14853>

Merge chapmanb py3k-subprocess32 pull request

14947 <https://dev.arvados.org/issues/14947>

[Tests] run-tests.sh --interactive

15064 <https://dev.arvados.org/issues/15064>

[Workbench2] Use long-lived cookies to improve login chooser defaults

9945 <https://dev.arvados.org/issues/9945>

[SDK] Package Python apps as virtualenvs

13306 <https://dev.arvados.org/issues/13306>

arvados-cwl-runner supports Python 3

13823 <https://dev.arvados.org/issues/13823>

Package Arvados for Ubuntu Bionic Beaver 18.04 LTS

13937 <https://dev.arvados.org/issues/13937>

[keepstore] Move legacy metrics to prometheus /metrics endpoint

13996 <https://dev.arvados.org/issues/13996>

[API] Migrate RailsAPI to new cluster config file

14382 <https://dev.arvados.org/issues/14382>

Review & merge new contributed Java SDK

14484 <https://dev.arvados.org/issues/14484>

[API Server] Return collection size and number of files in collection record

14606 <https://dev.arvados.org/issues/14606>

[CWL] CWL v1.1 spec work and support

14645 <https://dev.arvados.org/issues/14645>

[arv-mount] Report FUSE operation latency

14669 <https://dev.arvados.org/issues/14669>

[Java] Merge into main Arvados repo

14670 <https://dev.arvados.org/issues/14670>

[Java] Write / merge Java SDK documentation

14725 <https://dev.arvados.org/issues/14725>

[GA4GH] WES 1.0 prototype

14794 <https://dev.arvados.org/issues/14794>

Link from Workbench 1 to Workbench 2

14796 <https://dev.arvados.org/issues/14796>

[crunch-dispatch-cloud] Document installation / migration from c-d-slurm +
node manager

14807 <https://dev.arvados.org/issues/14807>

[arvados-dispatch-cloud] Features/fixes needed before first production
deploy

14835 <https://dev.arvados.org/issues/14835>

Update cluster configuration wiki from scratch doc

14841 <https://dev.arvados.org/issues/14841>

Link from Workbench 2 to Workbench 1

14870 <https://dev.arvados.org/issues/14870>

[API] Access logs from previous attempts after auto-retrying a container
request

14873 <https://dev.arvados.org/issues/14873>

[API] Update to Rails 5

14939 <https://dev.arvados.org/issues/14939>

Update crunchstat-summary to Python 3

14988 <https://dev.arvados.org/issues/14988>

[Workbench] Upgrade to Rails 5

15002 <https://dev.arvados.org/issues/15002>

[API] Admin can prevent reuse by cancelling a completed container

15088 <https://dev.arvados.org/issues/15088>

[Workbench2] Replicate Workbench1 merge account feature

15180 <https://dev.arvados.org/issues/15180>

[Spike] Test migration on production database

#

Subject

8189 <https://dev.arvados.org/issues/8189>

[FUSE] Listing a project directory is slow when there are many subprojects

9865 <https://dev.arvados.org/issues/9865>

[CWL] Fix undefined behavior after ignoring an unhandled exception

10570 <https://dev.arvados.org/issues/10570>

[crunchstat-summary] should not report 0% CPU and 0 RAM in cases where
stats were not reported at all

11561 <https://dev.arvados.org/issues/11561>

[API] Limit number of lock/unlock cycles for a given container

12026 <https://dev.arvados.org/issues/12026>

[crunchstat-summary] gives invalid min_cores recommendation

13593 <https://dev.arvados.org/issues/13593>

[API] Sequence of "create group" requests runs slowly, and can crash API
server

14326 <https://dev.arvados.org/issues/14326>

Our custom-compiled `python-future` and `python3-future` packages can't be
installed together and have precedence

14383 <https://dev.arvados.org/issues/14383>

[API] Java SDK double slash bug with arvados-controller

14451 <https://dev.arvados.org/issues/14451>

crunchstat-summary recommends 563 TB Keep cache (or more!)

14482 <https://dev.arvados.org/issues/14482>

[SDKs] Handle "." empty directory placeholder in manifests in Ruby SDK

14539 <https://dev.arvados.org/issues/14539>

[SDKs] [arv-mount] Use "." placeholder to persist empty directories

14593 <https://dev.arvados.org/issues/14593>

1.3.0 a-c-r broken on crunch1

14627 <https://dev.arvados.org/issues/14627>

[crunchstat-summary] job elapsed time wrong for crunch2 jobs

14691 <https://dev.arvados.org/issues/14691>

Update suggested .ssh/config section in documentation and workbench

14703 <https://dev.arvados.org/issues/14703>

[SSO] Login fails when configured with only the local provider

14723 <https://dev.arvados.org/issues/14723>

[CWL] File that should appear multiple times in output collection only
appears once

14738 <https://dev.arvados.org/issues/14738>

[Workbench] Tag editor not loading

14745 <https://dev.arvados.org/issues/14745>

[crunch-dispatch-cloud] Azure cloud driver fixups

14770 <https://dev.arvados.org/issues/14770>

Registered workflows fail to run due to spurious --collection-keep-cache=256

14799 <https://dev.arvados.org/issues/14799>

[SSO] bump the omniauth-google-oauth2 gem to 0.6.0

14804 <https://dev.arvados.org/issues/14804>

[keepstore] Return 5xx (not 4xx) if block is not found due to transient
backend device failure

14806 <https://dev.arvados.org/issues/14806>

[crunch1] unescape filenames when parsing manifests in crunch-job

14826 <https://dev.arvados.org/issues/14826>

[API] Search CentOS certificates path when making callbacks to remote
clusters

14828 <https://dev.arvados.org/issues/14828>

[crunch-dispatch-cloud] Fix flaky test

14829 <https://dev.arvados.org/issues/14829>

Update gem dependencies from Arvados 1.2 to 1.3

14844 <https://dev.arvados.org/issues/14844>

[dispatch-cloud] Azure driver bugs discovered in trial run

14845 <https://dev.arvados.org/issues/14845>

[controller] Fix flaky test

14880 <https://dev.arvados.org/issues/14880>

arv-keepdocker is very picky about the order of its command line arguments
and crashes with an obscure error if the order is wrong

14884 <https://dev.arvados.org/issues/14884>

arv-put doesn't handle non-ASCII filenames correctly

14885 <https://dev.arvados.org/issues/14885>

Review and merge chapmanb's ciso and conda packaging pull request

14900 <https://dev.arvados.org/issues/14900>

[arv-put] when reusing a local cache, arv-put does not check if the blocks
exist and blindly creates the collection

14911 <https://dev.arvados.org/issues/14911>

[dispatch-cloud] Azure driver panic at startup

14920 <https://dev.arvados.org/issues/14920>

[crunch-dispatch-cloud] New Azure instances always have state=unknown
instead of state=booting

14936 <https://dev.arvados.org/issues/14936>

ImportError: No module named future_builtins

14938 <https://dev.arvados.org/issues/14938>

[crunch-dispatch-cloud] flaky test in lib/dispatchcloud/worker

14966 <https://dev.arvados.org/issues/14966>

[API] Fix hanging test - suspect permission changes

14977 <https://dev.arvados.org/issues/14977>

[arvados-dispatch-cloud] kill crunch-run procs for containers that are
deleted or have state=Cancelled when dispatcher starts up

15007 <https://dev.arvados.org/issues/15007>

[dispatch-cloud] Azure driver panic in Address() (missing nic?)

15040 <https://dev.arvados.org/issues/15040>

Fix federation tests

15044 <https://dev.arvados.org/issues/15044>

[Tests] Update tests to work on base debian:9 image

15050 <https://dev.arvados.org/issues/15050>

Limit number of containers crunch-dispatch-local runs at one time

15058 <https://dev.arvados.org/issues/15058>

[SSO] "Not found. Authentication passthru" when using local account
authentication

15080 <https://dev.arvados.org/issues/15080>

Installing arvados-cwl-runner==1.3.1 from PyPi is broken

15109 <https://dev.arvados.org/issues/15109>

[Documentation] Add details about logs table purging on the Admin section

15148 <https://dev.arvados.org/issues/15148>

keep-balance incorrectly accounts for blocks in collections with null
`modified_at` field

15161 <https://dev.arvados.org/issues/15161>

Dev clusters are failing: uninitialized constant ArInternalMetadatum
(NameError)

15164 <https://dev.arvados.org/issues/15164>

Container request not finalized

15167 <https://dev.arvados.org/issues/15167>

Complete logs table management docs with unlogged_attributes setting

15225 <https://dev.arvados.org/issues/15225>

[API] installation gives error: Creating git_internal_dir ''...mkdir:
cannot create directory ‘’: No such file or directory

15227 <https://dev.arvados.org/issues/15227>

[API][Workbench] Adding tags via workbench1's tag editor saves the JSON
string to properties

15230 <https://dev.arvados.org/issues/15230>

[wb2] Crash loading federated collection

15232 <https://dev.arvados.org/issues/15232>

[config] BlobSigningTTL should be expressable with a suffix

15241 <https://dev.arvados.org/issues/15241>

[a-c-r] crashing using collection in writable initialWorkDir
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.arvados.org/pipermail/arvados-dev/attachments/20190605/1575f4b7/attachment-0001.html>