From tfmorris at veritasgenetics.com Thu Jun 6 06:05:00 2019 From: tfmorris at veritasgenetics.com (Tom Morris) Date: Wed, 5 Jun 2019 23:05:00 -0700 Subject: [arvados-dev] Arvados v1.4 announcement and release notes Message-ID: The Arvados team is pleased to announce our latest release Arvados v1.4 which includes several new features, the most exciting of which is the brand new implementation of the Arvados web UI component, Workbench. It is currently still in beta, but we’d love for you to test it as part of your daily work and provide us with feedback. Other highlights include: - New Workbench - React-based single page web app with completely new UI design - New Java SDK with much more complete API coverage - API Server & traditional Workbench upgraded to Rails 5 - Support for Prometheus metrics reporting in a number of components - Packaging changes for Arvados utilities to improve isolation of dependencies - Support for Ubuntu 18.04 - More components ported to Python 3.x and Python 3.7 support added - More than 50 bug fixes There are a couple of important release notes that you should review before upgrading. Installing the python-arvados-python-client distribution package no longer installs the Arvados Python module (ie arvados) system-wide. If you are depending on its availability, you will need to install arvados-python-client from PyPI. Using virtualenv or similar dependency manager is a good way to isolate dependency requirements of different applications. As we’ve announced previously v1.4 is the last release to support the Jobs API (aka Crunch 1). If you haven’t already migrated to Crunch 2 and the Container API, you should make plans to as soon as possible. Please contact us if you have any questions or require assistance. You can find a full list of the features and bug fixes on the release page and included below. # Subject 12958 [Federation] Workbench login chooser 14291 [crunch-dispatch-cloud] AWS driver 14322 [CWL] Accept collection uuid in input 14324 [crunch-dispatch-cloud] Azure driver 14325 [crunch-dispatch-cloud] Dispatch containers to cloud VMs directly, without slurm or nodemanager 14573 [Spike] [API] Fully functional filename search 14718 [API] Option to issue salted token in login procedure 14720 [Federation] Workbench2 login chooser 14819 Upgrade the arvados/jobs image to Debian Stretch 14853 Merge chapmanb py3k-subprocess32 pull request 14947 [Tests] run-tests.sh --interactive 15064 [Workbench2] Use long-lived cookies to improve login chooser defaults 9945 [SDK] Package Python apps as virtualenvs 13306 arvados-cwl-runner supports Python 3 13823 Package Arvados for Ubuntu Bionic Beaver 18.04 LTS 13937 [keepstore] Move legacy metrics to prometheus /metrics endpoint 13996 [API] Migrate RailsAPI to new cluster config file 14382 Review & merge new contributed Java SDK 14484 [API Server] Return collection size and number of files in collection record 14606 [CWL] CWL v1.1 spec work and support 14645 [arv-mount] Report FUSE operation latency 14669 [Java] Merge into main Arvados repo 14670 [Java] Write / merge Java SDK documentation 14725 [GA4GH] WES 1.0 prototype 14794 Link from Workbench 1 to Workbench 2 14796 [crunch-dispatch-cloud] Document installation / migration from c-d-slurm + node manager 14807 [arvados-dispatch-cloud] Features/fixes needed before first production deploy 14835 Update cluster configuration wiki from scratch doc 14841 Link from Workbench 2 to Workbench 1 14870 [API] Access logs from previous attempts after auto-retrying a container request 14873 [API] Update to Rails 5 14939 Update crunchstat-summary to Python 3 14988 [Workbench] Upgrade to Rails 5 15002 [API] Admin can prevent reuse by cancelling a completed container 15088 [Workbench2] Replicate Workbench1 merge account feature 15180 [Spike] Test migration on production database # Subject 8189 [FUSE] Listing a project directory is slow when there are many subprojects 9865 [CWL] Fix undefined behavior after ignoring an unhandled exception 10570 [crunchstat-summary] should not report 0% CPU and 0 RAM in cases where stats were not reported at all 11561 [API] Limit number of lock/unlock cycles for a given container 12026 [crunchstat-summary] gives invalid min_cores recommendation 13593 [API] Sequence of "create group" requests runs slowly, and can crash API server 14326 Our custom-compiled `python-future` and `python3-future` packages can't be installed together and have precedence 14383 [API] Java SDK double slash bug with arvados-controller 14451 crunchstat-summary recommends 563 TB Keep cache (or more!) 14482 [SDKs] Handle "." empty directory placeholder in manifests in Ruby SDK 14539 [SDKs] [arv-mount] Use "." placeholder to persist empty directories 14593 1.3.0 a-c-r broken on crunch1 14627 [crunchstat-summary] job elapsed time wrong for crunch2 jobs 14691 Update suggested .ssh/config section in documentation and workbench 14703 [SSO] Login fails when configured with only the local provider 14723 [CWL] File that should appear multiple times in output collection only appears once 14738 [Workbench] Tag editor not loading 14745 [crunch-dispatch-cloud] Azure cloud driver fixups 14770 Registered workflows fail to run due to spurious --collection-keep-cache=256 14799 [SSO] bump the omniauth-google-oauth2 gem to 0.6.0 14804 [keepstore] Return 5xx (not 4xx) if block is not found due to transient backend device failure 14806 [crunch1] unescape filenames when parsing manifests in crunch-job 14826 [API] Search CentOS certificates path when making callbacks to remote clusters 14828 [crunch-dispatch-cloud] Fix flaky test 14829 Update gem dependencies from Arvados 1.2 to 1.3 14844 [dispatch-cloud] Azure driver bugs discovered in trial run 14845 [controller] Fix flaky test 14880 arv-keepdocker is very picky about the order of its command line arguments and crashes with an obscure error if the order is wrong 14884 arv-put doesn't handle non-ASCII filenames correctly 14885 Review and merge chapmanb's ciso and conda packaging pull request 14900 [arv-put] when reusing a local cache, arv-put does not check if the blocks exist and blindly creates the collection 14911 [dispatch-cloud] Azure driver panic at startup 14920 [crunch-dispatch-cloud] New Azure instances always have state=unknown instead of state=booting 14936 ImportError: No module named future_builtins 14938 [crunch-dispatch-cloud] flaky test in lib/dispatchcloud/worker 14966 [API] Fix hanging test - suspect permission changes 14977 [arvados-dispatch-cloud] kill crunch-run procs for containers that are deleted or have state=Cancelled when dispatcher starts up 15007 [dispatch-cloud] Azure driver panic in Address() (missing nic?) 15040 Fix federation tests 15044 [Tests] Update tests to work on base debian:9 image 15050 Limit number of containers crunch-dispatch-local runs at one time 15058 [SSO] "Not found. Authentication passthru" when using local account authentication 15080 Installing arvados-cwl-runner==1.3.1 from PyPi is broken 15109 [Documentation] Add details about logs table purging on the Admin section 15148 keep-balance incorrectly accounts for blocks in collections with null `modified_at` field 15161 Dev clusters are failing: uninitialized constant ArInternalMetadatum (NameError) 15164 Container request not finalized 15167 Complete logs table management docs with unlogged_attributes setting 15225 [API] installation gives error: Creating git_internal_dir ''...mkdir: cannot create directory ‘’: No such file or directory 15227 [API][Workbench] Adding tags via workbench1's tag editor saves the JSON string to properties 15230 [wb2] Crash loading federated collection 15232 [config] BlobSigningTTL should be expressable with a suffix 15241 [a-c-r] crashing using collection in writable initialWorkDir -------------- next part -------------- An HTML attachment was scrubbed... URL: