[ARVADOS] updated: 1.3.0
Git user
git at public.curoverse.com
Mon Dec 3 13:35:25 EST 2018
to 32b654cf6c80968bd3b048ffba8e4e64ce3a815d (tag)
from 91d9a602d609ffcf907740b07b51937af9273796 (which is now obsolete)
tagging 3131146af966744a8f19e56be2d4a870bc4ab615 (commit)
replaces 1.2.0
tagged by Ward Vandewege
on Mon Dec 3 13:34:06 2018 -0500
Release 1.3.0
refs #14559
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <wvandewege at veritasgenetics.com>
Colin Nolan (2):
Allow audit logging to be disabled.
Corrects logic to determine if audit logging is enabled.
Joshua C. Randall (4):
change http timeout keyword from `http-timeout` to `http_timeout`
Allow hardcoding of protocol and port components of root_url alongside host
fix ruby syntax of audit logging disabled check
limit concurrent slurm commands
Lucas Di Pentima (82):
13773: Adds runtime_status field to containers on API server
13773: Filter out running containers that will fail from reuse selection.
13773: Add documentation.
13773: Fix reuse query & add tests
13773: Expand test to prove that non-error runtime_status doesn't avoid reuse.
13773: Add runtime_status to be accesible via API
13773: Updates runner container's runtime_status on child failures.
13773: Updates tests checking that runtime_status_error() is called when needed
13773: Containers & CRs show "Failing" when a child job failed.
13773: Show 'Warning' label on jobs that have this runtime status and aren't failing.
13773: Update runtime_status_error() method to report warning & activity status
13773: Intercept logging calls to update runtime_status on runner containers.
13773: Updates documentation
13773: Don't intercept logger.info() calls for runtime status.
13773: Enhance error/warning runtime status display on workbench.
13773: Add well known runtime_status' keys data type validation.
13773: Removed superfluous documentation
13773: Update test that confirms runtime_status is updated when a child fails
13773: Fix test
13773: Update logging level checking for runtime status updates.
Merge branch 'master' into 13773-will-fail-container-status
Merge branch '13773-will-fail-container-status'
13561: Add initial collection version information to the database
13561: Add basic support for collection versioning at the model level.
13561: Add missing copyright notices on migrations
13561: Lock row when saving a new collection version, update tests.
13561: Sync selected fields changes with older versions, with tests.
13561: Avoid old versions to be updated, with test.
13561: Old collection version's modified_at reflects version's create time.
13561: Save new version when preserve_version==true or config idle time is up.
13561: Avoid collections.index to include old versions
13561: Expand index API to include past versions.
13561: Avoid permission links to be attached to past collection versions.
13561: Ignore version & current_version_uuid parama on API create endpoint.
13561: Allow request old collection version by UUID.
Merge branch 'master' into 13561-collection-versions-api
13561: Refactor version creation & sync logic.
13561: Fix validation & update related test.
13561: Move versioning code inside locking block.
Merge branch '13561-collection-versions-api'
14086: Add tests to prove that keep-web can serve past collection versions.
Merge branch '14086-keepweb-serve-past-versions'
13561: Fixes api to get a collection with its past versions.
13561: Raise exception instead of ignoring versioning attributes updates.
Merge branch '13561-collection-versions-fixes'
13561: Update documentation on collection versioning.
13561: Re-phrase user guide & add admin guide section.
13561: Exclude past versions regardless of user & include_trash setting.
13561: Code cleanup.
13561: Formatting fixes, examples enhancements & additions.
Merge branch '13561-trashed-collection-versions-fix'
13561: Rephrased some sentences to avoid confusion.
Merge branch '13561-collection-versions-doc'
14259: Add method to keep client to request remote block copy via HEAD request.
14299: Use include_old_versions in keep-balance to avoid losing blocks.
Merge branch '14299-keep-balance-past-versions'
14259: Move & improve test.
14259: Collection class copies remote blocks when saving.
Merge branch '14259-pysdk-remote-block-copy'
14259: Test updates & additions related to keep client HEAD requests
14259: Don't enforce bandwidth timeouts on keep client HEAD requests.
14259: Iterate just once when scanning for remote blocks.
14259: Don't scan the entire collection for remote blocks when not needed.
Merge branch 'master' into 14259-pysdk-remote-block-copy
14455: Update loofah & nokogiri on workbench & api for security reasons.
Merge branch '14455-loofah-upgrade'
14323: Prioritize PDH over UUID for mount resolving when both provided.
14323: Lookup collection mount by uuid when pdh is not given. Update tests.
14439: Adds federation configuration page to the admin docs.
14323: Fixes indentation.
Merge branch '14439-federation-admin-docs'
14259: Improvements on remote blocks copying logic.
14323: Only get collection mount by UUID when PDH is missing and UUID provided.
14323: Adds test for the empty collection mount resolution case.
Merge branch '14259-pysdk-remote-block-copy'
Merge branch '14323-container-collection-mounts'
14497: Upgrades rack to 1.6.11 on api & workbench.
Merge branch '14497-rack-upgrade'
14519: Test exposing the regression
14519: Fixes bug by adding table tags to the sanitizer white list
14519: Adds more table related tags to the whitelist
Merge branch '14519-wb-textile-issues'
Nico Cesar (3):
14535: add dependencies for ruamel.yaml
14535: bumping package revision up for python-ruamel.yaml
Merge branch '14535-ruamel-dependencies'
Peter Amstutz (213):
14082: Round up numeric values in container request that must be integers
13973: Use a straightforward maximum() query to compute max priority
Merge branch 'wtsi/14110-c-d-s-limit-slurm-concurrency' refs #14110
Merge branch 'wtsi/python-api-timeout' refs #13542
Merge branch 'wtsi/13809-root_url-protocol-port-configuration'
Fix gofmt in slurm.go refs #14110
Merge branch '13973-max-cr-priority' refs #13973
Merge branch '14082-round-off' refs #14082
Add http_timeout to runtimeContext refs #13542
13146: Projects shared with me WIP
13146: Returns real json response
13993: Support for federated collection requests by uuid
13993: Consolidate SignedLocatorPattern with keepclient.SignedLocatorRe
13993: Don't export internal types
13993: Need to explicitly handle paths that don't end in /
13993: Add TestGetLocalCollection
14087: Federated fetch by PDH WIP
14087: Fetch federated collection by PDH
14087: Check for remote clusters before searching
14087: Remove debug print. Comments.
14087: Rate limit the number of parallel requests
Merge branch '13993-federated-collection' refs #13993
14203: Mkdir failing with os.ErrExist is not a fatal error
14087: Refactor federation search for collections by PDH
14203: Tweak error messages.
13493: Federation docs wip
14087: Add Accept-Encoding to list of headers to delete when proxying
Merge branch '14203-crunch-run-mkdir' closes #14203
Merge branch '14087-federated-collection-by-pdh' refs #14087
13493: Federation documentation covers record retrieval, keep
13493: Notes about cluster id, work in progress
10865: Add enable_legacy_jobs_api configuration option
10865: Fix query for empty jobs table
14197: Verify manifest text matches portable data hash that was requested
14242: Add a test for manifest_text PDH checking
10865: Add test for check_enable_legacy_jobs_api
10865: Update documentation, replace disable_api_methods with enable_legacy_jobs_api
Merge branch '14242-verify-manifest-pdh' closes #14242
Merge branch '13493-document-federation' refs #13493
14197: Generalizing federated routing
Merge branch '10865-legacy-jobs-api' closes #10865
14197: Supports routing object create requests, with tests
14197: Add cluster_id to discovery document for create method.
14197: Federated container request test tweaks
14197: Another test checking uuid from federated container request.
Merge branch '14197-federated-containers' refs #14197
13619: Federated multi-object list wip
13619: MultiClusterQuery passes test
13619: Clean up, require count=none
13619: Support [uuid, =, ...], cleanups
13619: Polish error handling
13619: Test that "select" is passed through multi-object query
13619: More tests for paging, error conditions
13619: Test error reporting when one of the federates fails
13619: Document availability of federation features
a-c-r recognizes that http and https needs to be uploaded to keep
14201: add exclude_home_project to 'contents' endpoint
13619: Code cleanups
14201: Make exclude_home protected, deduplicate owner uuids
14196: Login process returns v2 tokens
Merge branch '14201-exclude-home-project' closes #14201
13619: Move configuration options to arvados.Cluster.RequestLimits
14196: Don't crash if element of params[:reader_tokens] is not a String
Merge branch '13619-fed-object-list' closes #13619
14274: Fix FactoryGirl deprication warning (renamed to FactoryBot)
12690 & 12748: Read stats from arv-mount.txt
14196: Multi-site search works with v2 tokens.
Merge branch '12690-12748-crunchstat-summary'
arvbox fix "can't write to /dev/stderr" message no issue #
Merge branch '14274-factory-bot' closes #14274
14196: Revert debug change to Access-Control-Max-Age
Merge branch '14196-api-v2-tokens' refs #14196
14260: Migration adding runtime_token, runtime_user_uuid, runtime_auth_scopes
14260: Container runtime token wip
14260: Test container runtime_token
14260: runtime_token goes in container auth
14260: Added runtime_token to container record
14260: Don't set auth_uuid when runtime_token is set.
14260: Container auth tokens must have the container uuid in the 4th position
14260: crunch-run constructs a v2 token with added container uuid
14260: Update API doc for runtime_* fields (token/user_uuid/auth_scopes)
14260: Additional validation when container uuid is provided in token
14260: Add invalid remote token test
14260: Fix tests.
14260: Fix workbench functionals test
14260: Bugfix for runtime_token/secret_mounts and retries
14260: add runtime_user_uuid and runtime_auth_scopes to API response
14260: secret_mounts works with runtime_token
14260: secret_mounts works with runtime_token (really this time)
14366: Improve wording of "upgrading to containers API" section
14366: Add "upgrade to containers api" to "upgrading" section
Merge branch 'master' into 14260-runtime-token
14260: Sort runtime_auth_scopes
14260: Remove redundant schema_migrations in structure.sql
Merge branch '14366-crunch2-upgrade' refs #14366
Merge branch '14260-runtime-token' refs #14260
14260: Use straight SQL DELETE instead of destroy_all
Merge branch '14260-expired-token-delete' refs #14260
14262: saltAuthToken returns copy of request object
14262: Use container token for access to load Docker image
14262: Fix bug moving api_token to header
14262: Refactoring proxy
14262: Refactoring, split up federation code into smaller files
14262: Add createAPIToken, with test
14262: Tests for setting and checking container tokens.
14262: Fix tests
14262: Missing file
14262: Fix crunch-run tests
14262: Handle container_request posted as a string parameter
14198: Fix error responses in container POST
14262: Fix logic for when to allow unknown PDH for containers
14262: Only allow unknown PDH for images when there are remote_hosts
14262: Make sure cancel() from proxy.Do() gets called
14262: Rewrite collectionFederatedRequestHandler PDH search to use channels
14262: Fix spillover of tests changing remote_hosts
14262: Only create runtime_token on home cluster for the authorization
14262: Fix permissions so runtime_token can set container progress/output
14262: Move the context deadline to the top of the handler stack
14262: Revert changes to client.go
14262: Fix remote_user_test.rb so it doesn't mess up remote_hosts
14262: Avoid out-of-bounds panics checking cluster prefixes
Merge branch '14262-remote-container' refs #14262
14198: Add cluster_id to index parameters
Merge branch '14198-index-accepts-cluster-id' refs #14198
14458: Avoid panic by removing defer close()
14458: Drain errors channel on failure case instead of accumulating separately
"arvbox status" includes cluster id refs #14198
14458: Drain errorChan without waiting for close
14458: Fix syntax error
Merge branch '14458-controller-panic' refs #14458
14198: Use container token for reading input/creating output
14198: Add comment about different client objects used by runner
Merge branch '14198-crunch-run-tokens' refs #14198
14198: Expand federation support for collections and links
Merge branch '14198-fed-collection-list' refs #14198
14198: WIP handling various cases of remote users submitting container requests
14198: Remote user uses current token in runtime_token
14198: Strip 4th position when setting runtime_token from provided auth
14198: Add test that runtime token is set from auth for remote user
Merge branch '14198-request-token' refs #14198
Add --no-tty to apt-key in arvbox dockerfile. no issue #
CWL arvados-tests.sh uses --api=containers no issue #
14198: Initial support ClusterTarget hint
14198: Use PDH for container_image instead of docker repo+tag
14198: Refactor and add support for --submit-runner-cluster
14198: Resolve to Docker images to PDH and set "http://arvados.org/cwl#dockerCollectionPDH"
14198: Support expressions in TargetCluster[clusterID, ownerUUID]
14198: Log current container
14198: Fix typo current -> current_container, add copyright header
14198: Use ArvRuntimeContext.submit_runner_cluster for target cluster
14198: Add paging on container list, check for valid cluster id
14198: Bringing unit tests up to date WIP
14198: More test updates
14198: test_submit passes
14198: Unit tests pass again
14198: Add --submit-runner-cluster test
14198: Add --always-submit-runner
14198: Update cwltool and schema-salad
14198: Update version deps for schema-salad, cwltool, ruamel.yaml, requests
14198: Construct ArvadosWorkflowStep
14198: Fix ruamel.yaml versioning
14198: Fix jobstep builder init
14198: Set fake 'class' so _init_job is happy
14198: Set arvrunner on ArvadosWorkflowStep
14198: Set arvrunner before calling super constructor
14198: Federation feature and testing
14198: Refactor arvbox federation setup
14198: Working on two step test cases
14198: Add more twostep tests
14198: Add remote-case and twostep-both-remote
14198: Add some basic doc comments.
14198: Fix collection PDHs for remote-case
14198: Add test twostep-remote-copy-to-home
14198: Add scatter-gather
14198: Add threestep-remote
14198: Connect test cases to outputs
14198: Rewrite README
14198: Set ClusterTarget hint on steps instead of tools
14198: Test cases for places that ClusterTarget can appear
14198: Tests prefer setting ClusterTarget hint on the workflow step.
14198: Tests for hints in different places
14198: Fix secondaryFile tests
14198: Update build.list for latest cwltool/schema-salad
14198: Add check for uuidPrefix if local cluster isn't in the remoteHosts.
14198: Call validate_cluster_target on --submit-runner-cluster
Merge branch '14198-fed-testing' refs #14198
Fix --submit-runner-cluster tests refs #14198
Fix ClusterTarget.project_uuid to correctly set the project uuid
Finish fixing setting project_uuid from runtimeContext, fix tests.
14198: Restore behavior of creating an empty Builder object
Merge branch '14198-make-builder' refs #14198
14440: Working on docs for federated workflows.
14440: Federated workflow docs WIP
14440: Add project_uuid to ClusterTarget example
14440: Add project
14440: Comments on how federated.cwl works
14440: Federated workflow diagram.
14440: Add discussion of federation features.
14440: Refresh cwl-run-options, add note about dockerCollectionPDH
14440: Fix formatting of cwl-run-options table.
Merge branch '14440-fed-user-doc' refs #14440
Restore default value of --on-error to 'continue' refs #14440
14510: Perfomance fixes
14510: Setting collection cache wip
14510: Update tests/docs for --collection-cache-size
14510: Estimate collection cache size wip
14510: Set collection metadata cache size as 150% of sum of inputs
14510: Don't initialize workflow redundantly.
14510: Bugfix order of precedence for setting collection cache size
Merge branch '14510-cwl-perf' refs #14510
14558: Version dependency fixes.
Merge branch '14558-cwl-pkg-deps' refs #14558
Merge branch '14560-tsvector-too-long' refs #14560
ThomasHickman (1):
Add timeout for the python api + A.C.R.
Tom Clegg (130):
14008: Add index to speed up dispatcher queries.
14009: Index containers table for reuse searches.
14008: Add container queue index.
14008: Merge branch 'master' into 14008-containers-index
Merge branch '14009-container-reuse-index'
14152: Use V4 signer for S3 in regions that use V4 for EC2.
14008: Merge branch 'master' into 14008-containers-index
Remove redundant test case.
Add test case for filtering on is_trashed in groups#contents.
Merge branch '14008-containers-index'
Merge branch '14152-s3-v4-signature'
11065: Merge branch 'master' into 11065-optional-audit-logging
Merge branch '11065-optional-audit-logging'
13994: Proxy to remote cluster if +R hint given.
13994: Fix deadlock in keepstore tests.
13994: Fix import cycle.
13994: Use entire token for blob signatures.
13994: Fix token handling in collection performance test.
13994: Fix service discovery race.
13994: Fix missing error message.
13752: De-duplicate file and stream names in collections.file_names.
13752: Migrate file_names column using multiple transactions.
13994: Merge branch 'master' into 13994-proxy-remote
Merge branch '13994-proxy-remote'
13752: Merge branch 'master' into 13752-migrate-index-data
Merge branch '13752-migrate-index-data'
14236: Test deleting the last file in a collection.
14236: Don't omit empty manifest_text in update requests.
10181: Add Size method to arvados.CollectionFileSystem.
10181: Save log collection snapshots periodically during run.
14236: Fix regexp in test.
10181: Permit dispatcher to update log while container is running.
10181: Update container log PDH after updating log collection.
10181: Cleanup identifiers.
10181: Load checkpoint configs from discovery doc if available.
10181: Fix timing sensitivity in test case.
10181: Fix race in MarshalManifest.
10181: Keep CR logs synchronized with container logs while running.
10181: Merge branch 'master' into 10181-incremental-log
10181: Fix overloaded Size method.
Add missing copyright headers.
10181: Rename configs, add to API server config, support SIGUSR1.
Merge branch '14236-delete-last-file'
10181: Fix up redundant error message.
10181: Don't set log collection is_trashed until final save.
10181: Restore container's ability to update its own runtime_status.
10181: Merge branch 'master' into 10181-incremental-log
Merge branch '10181-incremental-log'
14199: Refactor test case.
14199: Store data if X-Keep-Signature given in proxied GET/HEAD req.
14199: Drop Vary response header for non-proxyable requests.
14199: Fix deadlock in test suite.
14285: Export stats as prometheus metrics.
14199: Require "X-Keep-Signature: local[, ...]" to invoke signing.
14199: Rename (*remoteResponseCacher)Flush -> Close.
14199: Abandon processing sooner if client disconnects.
14285: Update metrics doc page.
14199: Merge branch 'master' into 14199-copy-from-remote
14199: Reduce logging noise from azure test stubs.
Merge branch '14199-copy-from-remote'
14285: Split Server to server.go.
14285: Update sample config.
14285: Add token middleware. Require management token for metrics.
14285: Add keep-balance to example cluster config.
14285: Add keep-balance to health aggregator test stub.
14285: Merge branch 'master' into 14285-keep-balance-metrics
14285: Add tests for LoadToken and RequireLiteralToken.
14285: Merge branch 'master' into 14285-keep-balance-metrics
Merge branch '14285-keep-balance-metrics'
Fix missing namespace in metric names.
14285: Add deduplication ratio to keep-balance metrics.
14285: Update admin/metrics docs.
Merge branch '14285-keep-balance-metrics'
14344: Move "set -e" back to top.
14328: Cancel if container ends but ContainerWait does not return.
14328: Check broken-node blacklist when watchdog hits errors.
14388: Fix log message.
14388: Fix typo.
14388: Don't prefer positions that are only temporarily untrashable.
14388: Merge branch 'master' into 14388-overreplication
14398: Fix deadlock on error in GetCurrentState.
14407: Upgrade selenium-webdriver, add geckodriver.
14407: Fix "too much recursion" error.
14407: Add geckodriver to arvbox image.
Merge branch '14388-overreplication'
Merge branch '14407-selenium'
14328: Check ContainerInspect instead of List, give up on error.
Merge branch '14398-error-deadlock'
Fix crash in test cleanup.
Make geckodriver sanity check non-fatal.
Merge branch '14328-watch-docker-ps'
14406: Pass output through collectionFS if it has remote block refs.
14406: Fix safety warning.
14406: Remove debug printfs.
14406: Tone down slow stress test.
14406: Copy remote blocks to local storage during MarshalManifest.
14406: Only call LocalLocator() once per block per stream.
14406: Merge branch 'master'
Merge branch '14406-remote-data-in-output'
14345: Use "." placeholder to persist empty directories.
14345: Unescape all \ooo in names in manifests, not just \040.
14345: Accept lock/unlock requests as no-ops.
14345: Accept "." placeholder: create parent dir, but no fake file.
14345: Handle "MOVE foo/ bar/" requests.
Merge branch '14345-webdav-lock-and-empty-dir'
14328: Remove containerd watchdog.
14328: Fix flaky test.
14328: Merge branch 'master'
Merge branch '14328-containerd'
14397: Update goamz to fix v4 signature.
14397: Improve error reporting.
14397: Fix S3 ListOps stats.
14397: Fix unchecked errors in S3 block listing.
14397: Use {sha1}-dev as version in dev builds of Go programs.
14397: Pass SHA256 to s3 library.
14397: Add comment about memory implications of providing SHA256.
Merge branch '14397-keepstore-v4-signature'
14475: Fix Content-Type header parsing.
Merge branch '14475-update-collection-error'
Fix git command.
Update tests to match new template.
14419: Fix concurrent map write.
Merge branch '14419-concurrent-map-write'
14345: Support PROPPATCH method.
Merge branch '14345-proppatch'
14345: Disable Nginx proxy request buffering for keep-web requests.
14345: Disable proxy request buffering in Nginx keepproxy example.
Merge branch '14345-nginx-proxy-request-buffering'
14560: Only index the first 1M bytes of ft-searchable data.
14560: Limit to_tsvector input to 8K.
Tom Morris (4):
12690: Report Keep stats for Crunch2 container requests.
12748: Computed elapsed seconds for Crunch2 container requests.
Handle missing arv-mount.txt for older jobs
Add OS X description & tweak Windows
Ward Vandewege (34):
A new version of our libcloud fork is available.
Fix package building - support modern versions of pip.
Sanitize how we get the rvm GPG key, and remove a dependency on the GPG
Documentation tweaks. Point at Biostars as the forum to discuss Arvados
Small documentation fixes.
The `arv group create` command doesn't set group_class by default, and we need
Webshell requires the short hostname for the shell server object.
Make sure that the arvados-docker-cleaner systemd service file is installed in
Add a flag to hide repositories entries in the Workbench menus. This is useful
Merge branch '14338-workbench-flag-to-disable-repositories'
Revert change to setup.py for arvados-docker-cleaner, pip doesn't agree with
Fix the detection of rpm vs deb systems.
14344: Merge branch '14344-fix-rpm-vs-deb-detection'
14344: bump package version for arvados-docker-cleaner so that we get a new
Remove spurious output from run-build-packages.sh
Add a systemd service file to arvados-node-manager.
For the package test docker images, sanitize how we get the rvm GPG key,
Add a new release note for v1.2.0.
Fix typos.
Make sure keep-balance gets started with the lowest possible system priority by
In the notification e-mail to the user that a shell account has been created,
The Dockerfiles for package building and testing on debian 9 seem to want
14516: when calculating the next available username, make sure to take merged
Add user documentation for accessing Keep from Windows.
14535: fix typos.
14345: Some more language tweaks.
Merge branch 'master' into 14345-add-webdav-user-doc
Merge branch '14345-add-webdav-user-doc'
Release 1.2.1 does not require special attention when upgrading.
Stop relying on the keyservers during arvbox docker image build, they are too
Work around ruamel.yaml packaging issue.
Make sure the dev version prefix is based on the latest tag in the repo,
Make sure the dev version prefix is based on the latest tag in the repo,
Make sure the dev version prefix is based on the latest tag in the repo, not
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list