[ARVADOS] updated: 1.2.0
Git user
git at public.curoverse.com
Mon Dec 3 13:35:24 EST 2018
to d0067751351975a86e2400ff592ed747d0f38de1 (tag)
from 55dae8886e92de7737c8d89557d19be08af30f64 (which is now obsolete)
tagging 3c85ce36c1188474b6b94ad534daf51cb1497274 (commit)
replaces 1.1.4
tagged by Fernando Monserrat
on Wed Sep 5 16:13:28 2018 -0300
closes #14032 . Git tag 1.2.0 Release
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat at veritasgenetics.com>
Fernando Monserrat (38):
13339: update gemspecs for new versioning
13339: fix typo assigning git_latest_tag
13339: fix git_latest_tag formatting
13339: fix git_latest_tag formatting
13339: fix git_hash variable
refs #13339 "Merge branch '13339-update-gemspecs-new-versioning'".
13369: update ubuntu1404 Dockerfile
refs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
13369: update ubuntu1404 test Dockerfile
refs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
13369: update ubuntu1404 test packages Dockerfile
refs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
Merge branch 'master' into 13369-update-ubuntu1404-dockerfile
13369: add ulimit to the docker run fixing slowness bug on ubuntu1404
refs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
refs #13369 remove added packages install. rollback
refs #13369 rollback packages install on ubuntu1404 Dockerfile for builds
13667: Add version file to ROR packages
13667: fix variable in adding version file to ROR packages
13667: fix variable in adding version file to ROR packages
13667: fix variable in adding version file to ROR packages
13667: fix variable in adding version file to ROR packages
13667: clean up version file generationx
13667: test add config-file for rpm
13667: test add config-file for rpm
13667: test add PKG_VERSION in run-build-packages
13667: remove PKG_VERSION in run-build-packages
13667: remove PKG_VERSION in run-build-packages
13667: test fail on rpm package
13667: test fail on rpm package
13667: test fail on rpm package
13667: test fail on rpm package
13667: test fail on rpm package
13667: test fail on rpm package
13667: test fail on rpm package
13667: test fail on rpm package
13667: clean up
refs #13667 "Merge branch '13667-add-version-file'".
Fuad Muhic (70):
Added generateAPI function which will autogenerate REST API
Funtions that use POST verb now work correctley.
Integrated autogenerated classes with existing content
Merge branch 'master' of git.curoverse.com:arvados into 13076-r-autogen-api
Added error handlind code and proper folding to autogenerated content
Merge branch 'master' of git.curoverse.com:arvados into 13076-r-autogen-api
Documentation for Arvados classes is generated automatically now
Merge branch 'master' of git.curoverse.com:arvados into 13076-r-autogen-api
Added feature to autogenerate documentation for Arvados methods.
Merge branch 'master' of git.curoverse.com:arvados into 13076-r-autogen-api
Added fix for discovery document inconsistency
Merge branch 'master' of git.curoverse.com:arvados into 13076-r-autogen-api
Refactored autoGenApi to be more readable
Removed all autogenerated Arvados classes and updated rest of the code to reflect that
Added listAll utility function which will always fetch all resources.
Added project manipulation methods to autogenerated Arvados class.
Merge branch 'master' of git.curoverse.com:arvados into 13076-r-autogen-api
Filtered out all methods ending with index, show or destroy in Arvados class.
Removed obsolated methods from HttpRequest class and updated tests to reflect that change.
When env variable ARVADOS_API_HOST_INSECURE is set HTTP client will not try to validate the server's TLS certificate.
Updated README and removed debug prints
Arvados projects can be accessed from by_id directory in keep mount
Users can only access groups with group_class set to project from by_id
Optimized API call to not return non project groups
Merge branch '12320-access-projects-from-by_id'
Added auto-generated documentation for Arvados class.
Added documentation for hardcoded Arvados class project methods.
Method documentation now shows headline properly.
Updated README and small fix for Subcollection docs.
Added script to generate html doc for .Rmd files.
Merge branch '13076-r-autogen-api'
Replaced call to non-existent method execute in RESTService class with
Added --storage-classes argument to arv-put.
Added tests to arv-put to cover storage classes feature.
Added tests and documentation for save and save_new methods in
Updated save method of Collection class to always update when
Added test that checks that saving a committed collection updates the storage_classes_desired
Merge branch '13453-r-sdk-incorrect-rest-call-fix'
Merge branch '13430-allow-storage-classes'
Added properties and trash_at parameters to save and save_new methods
Add storage classes supper to arvados cwl runner
Add getters for properties and trash_at attributes and small bugfix for
Add check for multiple storage classes in arvados-cwl-runner
Fix ciso8601 issue in collections tests.
Merge branch '13330-collection-save'
Change multiple storage classes error handling in cwl_runner
Merge branch 'master' of git.curoverse.com:arvados into 13429-cwl-runner-storage-classes-support
Update storage_classes support for arvados_cwl_runner to work correctly
Add storage_classes tests for arvados_cwl_runner
Merge branch 'master' of git.curoverse.com:arvados into 13429-cwl-runner-storage-classes-support
Merge branch '13330-intermediates-test' of git.curoverse.com:arvados into 13330-cwl-intermediate-collections-cleanup
Update intermediate collection name, properties and trash_at atrribute
Merge branch '13429-cwl-runner-storage-classes-support'
Add runtime check for min R version.
Update README to document min R version required for ArvadosR
getConnection now works correctly when ARVADOS_API_HOST_INSECURE is set to TRUE
Add tests for intermediate collections
Merge branch '13613-document-min-r-version'
Remove trailig white-space
Merge branch 'master' of git.curoverse.com:arvados into 13330-cwl-intermediate-collections-cleanup
Remove _get_intermediate_collection_info method and create function
Update get_intermediate_collection_info and add owner_uuid to save_new
Merge branch 'master' of git.curoverse.com:arvados into 13330-cwl-intermediate-collections-cleanup
Merge branch '13330-cwl-intermediate-collections-cleanup'
add tests for exec and getConnection methods in HttpRequest class
remove trailing white space from all files
Merge branch '13645-fix-arvados-api-host-insecure'
Implement copy method, update move method and remove trailing
Improve Collections create and move methods and update documentation
Merge branch 'master' of git.curoverse.com:arvados into 13609-r-sdk-copy
Joshua C. Randall (4):
13558: add req_id as a tag to Rails.logger
fix "submitted container [uuid]" log from a-c-r
Make dispatcher Run loop use count=none and BatchSize limit
fix slow queries in uploadfiles
Lucas Di Pentima (116):
13105: Set remoteHosts and remoteHostsViaDNS default values when missing.
13105: discoveryDoc() call simplification.
Merge branch '13105-wb-ignore-missing-remotes'
12737: Updated gem dependencies, adressing a couple of known vulnerabilities.
Merge branch '12737-api-security-deps-fixes'
12737: Upgrade workbench from rails 4.1 to 4.2 (WIP)
12737: Remove attribute from previous tests
12737: Send serialized attributes when save()
12737: Assign empty default values on serialized attributes
12737: Fix controller tests
12737: Fix integration tests
Merge branch '12737-wb-rails42-upgrade'
7478: Use our libcloud fork
7478: Preemptable instances scheduling parameter on API server
7478: Preemptable instances support on GoSDK & dispatchcloud instance chooser
7478: Allow different configured node sizes to refer to the same instance type
7478: On EC2 driver ask for a spot instance when needed.
Merge branch 'master' into 7478-anm-spot-instances
7478: Scheduling parameters validation fixes.
7478: Ignore redundant instance_type field on CloudSizeWrapper
7478: Override CloudSizeWrapper id with config Size name.
7478: Set arvados-node-size-id tag on node creation.
7478: Test fixes.
7478: Unify arvados node size access across different drivers
Merge branch 'master' into 7478-anm-spot-instances
7478: Fixes integration tests.
13559: Pin ciso8601 to version <2.0.0 due to API changes.
Merge branch '13559-ciso8601-pin'
Merge branch 'master' into 7478-anm-spot-instances
13431: Adds storage class support to Keepclient.
13431: Make keepproxy propagate X-Keep-Storage-Classes headers
7478: Fixes child CR default preemptable setting logic.
13431: Adds test confirming that keepproxy propagates X-Keep-Storage-Classes
13431: Fixes unrelated test
13431: Removes commented code
Merge branch '13431-keepproxy-storage-classes-headers'
13480: Adds test making the bug show itself.
13480: Ignore nodes with no tags instead of crashing.
Merge branch '13480-anm-tagless-azure-node'
7478: Assign invalid cloud sizes to nodes with stale arvados_node_size tags
Merge branch 'master' into 7478-anm-spot-instances
7478: Config load fix & test updates about instance type decoupling
7478: Adds tests on driver's create_node calls.
7478: Adds test to check that state is 'down' with 'invalid' size.
7478: Fixes GCE driver's arvados_node_size tag handling.
13581: Available types reported to the user when CR is not satisfiable.
7478: Moves invalid cloud size node's shutdown decision to proper method.
Merge branch '7478-anm-spot-instances'
7478: Fixes InvalidCloudSize creation. Adds wishlist related node info to logs.
Merge branch '7478-invalid-size-not-defined'
7478: Fix nodemanager's libcloud install dependency.
Merge branch '7478-anm-libcloud-deps-fix'
13581: Adds log message when no instance types are configured.
13617: Replaces all logging about size names with size ids.
Merge branch '13617-anm-instance-type-logging'
Merge branch '13581-cr-not-satisfiable-report'
7478: Replaces term 'preemptable' with 'preemptible'
Merge branch 'master' into 7478-s-preemptable-preemptible
Merge branch '7478-s-preemptable-preemptible'
13649: Fixes default preemptible parameter when submitting committed child CRs
13649: Merges 2 tests into one.
Merge branch '13649-preemptible-committed-child-cr'
7478: Replaces term 'preemptable' with 'preemptible'
13649: Adds admin documentation describing how to set up spot instances.
13219: Adds default max_run_time value with validation, documentation and tests.
13219: Checks for expired run time and cancel container if needed.
13649: Fixes in wording
Merge branch '13649-spot-instances-doc'
13219: Removes unnecessary default max_run_time scheduling parameter
13219: Moves time out code from dispatch to crunch-run
13219: Adds TimeLimit support on arvados-cwl-runner
13219: Simplifies test
Merge branch '13219-jobs-time-limit'
13219: Adds TimeLimit support on Arvados CWL schema.
13219: Adds documentation
Merge branch '13219-arv-cwl-schema-fix'
13501: Replaces subprocess module with subprocess32 on nodemanager.
13501: Replaces subprocess with subprocess32 on arvados-cwl-runner
13501: Replaces subprocess with subprocess32 on keepdocker.
13501: Adds subprocess32 install dependency on python-client for keepdocker.
13668: Adds support for package version on API server.
13668: Adds package version to workbench. Reports versions on dialog.
13668: Replaced package_version with packageVersion on discovery doc.
13668: Updates VersionHelper on workbench to read packageVersion key
Merge branch '13668-api-wb-package-version'
Merge branch '13501-subprocess32'
13803: Upgrade oj gem on API server by removing arvados-cli dependency
13803: Upgrade oj dependency on arvados-cli gem.
13803: Upgrade oj to latest on workbench
13803: Remove oj_mimic_json gem.
13803: Re-enable Oj's rails optimizations & fix time format test failures.
13803: Fix behavior change on Oj 3.x
Merge branch '13803-oj-gem-malloc-bug'
13851: Reverted use of discoveryDoc() on findAPI()
Merge branch '13851-wb-multisite-search'
13888: Adds arvados-cli dependency back on API server
Merge branch '13888-cli-dependency-on-api'
12444: Adds tmpdir stats to crunchstat reporting
12444: Allow crunch-run to provide tmpdir to crunchstat
13910: Fixes arvados gem dependency on arvados-cli & arvados-login-sync
Merge branch '12444-tmpdir-monitoring'
Merge branch '13910-arvados-gem-retrieval'
13920: Fixes google-api-client gem issue with ruby 2.3.7+
Merge branch '13920-arvados-gem-with-newer-rubies'
12762: Suppress SafeYAML warnings by setting its default mode.
Merge branch '12762-safeyaml-warning'
13976: Add CWL test exposing the bug
13976: Fix keep reference error
14028: Add test exposing the bug
14028: Sanitize RedCloth's output.
14028: Confirm that safe tags (like <b></b>) don't get cleaned by sanitize()
Merge branch '14028-wb-textile-html-sanitization'
13340: Add test to expose bug
13340: Correctly show all locator links on a text.
13340: Enhance regex to match non quoted locators
Merge branch '13340-wb-keep-links'
Nico Cesar (1):
13579: Admin documentation setting up Azure resources
Pavel Mikhailovskii (1):
Fix a typo in Crunch Dispatch documentation. Arvados-DCO-1.1-Signed-off-by: Pavel Mikhailovskii <pavel at thehyve.nl>
Peter Amstutz (275):
13108: Handle multiple instances of arv-keepdocker uploading same docker image
13108: Use ThreadSafeApiCache
13108: Simplify locking, add methods for recording process status
13108: Work queue submit wip
13108: Collect outputs in separate threads as well
13108: Rename runnable_queue -> task_queue
13108: Rename parallel_submit_count to thread_count
13108: Fix tracking tasks in flight
13108: Refactor task queue into its own class.
13108: Fix tests.
13108: Add copyright notice to task_queue.py
13108: Add test for taskqueue
Merge branch '13108-cwl-parallel-submit' closes #13108
13108: Don't try to install signal handler in background thread
13108: crunch_script uses safeapi
13108: Refactor signal handling
13108: Fix import
Merge branch '13108-acr-threading-fixes' refs #13108
fix cwltest junit-xml dependency refs #13108
arv-put: restore_signal_handlers conditional on install_sig_handlers
Bump arvados-python-client version dependency.
13076: Get rid of pkgdown, just call knitr
13076: R SDK online docs include both README and method docs.
Make the runin-reqs less stingy about RAM refs #13301
arvbox pin pip to 9.0.3. Add LINES/COLUMNS to "arvbox shell", no issue #
Fix R SDK online doc generation refs #13076
Merge branch 'thehyve/master' LocalJumpError in arvados-login-sync
Fix nodemanager tests. no issue #
13385: login-sync test works even if there is no fuse group
Merge branch '13385-login-sync-test' refs #13385
login-sync test fix also make docker group optional refs #13385
Add tag to secrets test so it is recognized by new cwltest as "optional".
13282: First pass at reorganizing
13282: Add architecture section
13282: Remove some obsolete crunchv1 documenation from python sdk.
13282: Add note about supported platforms to install guide.
13282: doc updates
Merge branch '13282-admin-docs' refs #13282
13377: Ensure that collection fetched by PDH has longest available lifetime.
Merge branch '13377-block-sig-time' refs #13377
13338: Running workflow from workbench puts collections in correct project.
Node manager integration test writes to stderr in chunks.
13365: Update release notes about a-c-r bug in v1.1.4.
13365: Tweak text.
Merge branch '13338-wb-run-wf-proj' refs #13338
13365: Discover secondaryFiles associated with default files inputs
13365: Reintroduce secondaryFile discovery.
Bump cwltool version for secondaryFile checker support.
13365: Bump schema-salad for better error formatting.
13365: Bump cwltool, 3rd party package versions.
13365: Add example error messages and highlight the changes.
Merge branch '13365-note-bug-in-docs' refs #13365
13365: Fix race condition between job submitted and event monitor threads
13365: Further fixes to ensure output callback only happens once
13365: Fix state logging
13365: clean up discover_secondary_files
13365: Add tests to confirm regressions are fixed.
13365: Remove check for callback that now happens in a different place.
Merge branch '13365-default-secondaryFiles' refs #13365
13435: Update ruamel.yaml and ciso8601 versions.
13435: Build scandir to satisfy pathlib2 dependency of a-c-r
Merge branch '13435-ruamel-yaml-ver' closes #13435
Tweak tests for slightly different ruamel.yaml behavior, refs #13435
Fix subprocess32 version constraint, refs #13435
12256: Fix anonymous directory default inputs with file literals
12256: Add a bunch of test cases.
12256: Handle default references in RunInSingleContainer
12256: Bump cwltool version for workflow packing bugfix.
12256: Pull up minimum ruamel.yaml version to get consistent output.
12256: Only scan for referenced files first time after packing
12256: Comments about workflow references / extra files.
Merge branch '12256-cwl-defaults' closes #12256
Bump arvados-python-client version for consistent dependencies
Bump package iteration for a-c-r in run-build-docker-jobs-image.sh
12995: Add link account page with confirmation.
12995: Linking alternate login works.
12995: Can merge accounts in both directions.
12995: Can now merge with inactive accounts.
12995: Add testing
12995: Don't let admins lose admin privs by linking a non-admin account.
12995: Bugfix to support following chained user uuid redirection.
12995: Do nested logout/login to workaround SSO session.
12995: Add user and admin documentation pages.
12995: Add username to link_accounts page.
12995: Fix tests
Merge branch '12995-wb-merge-acct' refs #12995
13330: Test workflow that generates intermediates.
13330: Fix test workflow.
13516: Bump cwltool, cwltest and subprocess32 versions
Merge branch '13516-fix-cwl-tests' refs #13516
Fix tests refs #13516
Tweak arvados-cwl-runner subprocess32 version dependency refs #13516
11162: Support public http and https file references
11162: Handle missing git
11162: Add --submit-request-uuid
11162: Add progress for http data download.
11162: Set a default cache time of 24 hours for files fetched over http.
11162: Add logging about checking cache
11162: Smarter http downloads.
11162: Bump cwltool for fix in generating basename from url
11162: Fix tests.
11162: Add test for --submit-request-uuid
11162: Add tests for http_to_keep.
Replace unicode copyright character with ASCII in README.Rmd
Fix crunch_script for jobs api refs #13516
11162: Remove debugging check
Merge branch '11162-wes-support' refs #11162
11907: Add option to uploadfiles to disable packing.
Merge branch '11907-uploadfile-no-pack' refs #11907
11907: Make collections containing dependencies have predictable PDHs
11907: Fix tests
12960: Migrate properties fields to jsonb to support subproperty filters.
11907: Fix timezone handling for HTTP timestamps.
Merge branch '11907-acr-stable-pdh' refs #11907
13562: Add "properties" field to groups, supports subproperty query.
13562: Update documentation
12960: Update documentation.
13562: Migration needs to update full text search index on groups
12960: Fix test due to slightly different serialization from jsonb
Merge branch '12960-migrate-properties' closes #12960
Merge branch 'master' into 13562-group-properties
Merge branch '13562-group-properties' closes #13562
13579: Keepstore install documentation now describes using config.yml
13579: Remove some example output because I don't know if it is up to date.
13579: Document ManagementToken. Add information about trash behavior.
13579: Update from comments
docs wip
13579: Admin documentation about storage classes
13579: Add a little more introductory material on allocating resources
13580: Add storage classes section to user guide
13580: Add arvados-cwl-runner to storage class usage docs.
API fixup
13627: a-c-r migration to cwltool loading/runtimeContext API
13627: Fix loadingContext used for reloading document.
13627: Improve error reporting.
13627: Add missing file.
13627: Fix crunch_script
13627: Fix ArvadosWorkflow use of Builder
13627: Fix tests after rebase
Merge branch '13579-keepstore-config-doc' refs #13579 refs #13580
13627: Tweak test to possibly avoid name clash
Merge branch '13627-update-cwltool' refs #13627
13638: Restore logic of arvados-cwl-runner version is max(sdk, acr)
Merge branch '13638-acr-version' refs #13638
Add mypy-extensions to build.list refs #13627
13627: Make sure work_api is set on runtimeContext.
Merge branch '13627-set-work-api' refs #13627
13255: Account activate docs wip
13255: Edits
13255: More discussion of federated users in account activation
13255: Implement auto_activate_users_from
13255: Add test for auto_activate_users_from
13255: Doc format tweaks
13255: Add documentation & test that remote users can be pre-activated.
Merge branch 'master' into 13255-account-activation
Merge branch '13255-account-activation' closes #13255
Add mypy-extensions to build.list refs #13627
13627: Make sure work_api is set on runtimeContext.
13692: Make Python SDK error reporting more user friendly.
Merge branch '13692-acr-error' closes #13692
13766: upload_workflow dumps to json
13681: Implement WorkflowRunnerResources hint
13681: Can use --submit-runner-ram with --create/update-workflow
13681: Update documentation for arv:WorkflowRunnerResources
13681: Add new test file
13681: Revert to yaml.round_trip_dump instead of json.dumps().
13681: Fix workbench for workflows that don't have "hints"
Merge branch '13681-wb-acr-submit-ram' closes #13681
13767: Update crunch-dispatch-slurm install docs
Merge branch '13766-libyaml-workaround' refs #13766
12983: Periodically check for containerd
12983: Command line option takes duration. Add tests.
12983: Add another error message to the "broken node" blacklist.
12983: Tweak error regex
13780: Update sprockets gem
13690: Add Composer install documentation
Merge branch '13780-sprockets' closes #13780
12983: Change break -> return to exit goroutine, add comment
Merge branch '12983-containerd' closes #12983
13767: Tweak text for ReserveExtraRAM to specify default value.
Merge branch '13767-reserve-ram-doc' closes #13767
Merge branch '13690-composer-install' refs #13690
13799: Install doc refactor WIP
13799: More install doc refactor
Merge branch '13799-install-doc-sections' refs #13799
13822: Don't call list_sizes() in cloud client constructor.
13804: Node must report as "idle" two consecutive times
13804: Try to cancel pending shutdowns if nodes are needed
13804: Smarter shutdown behavior WIP
Merge branch 'master' into 13822-nm-delayed-daemon
Merge branch '13822-nm-delayed-daemon' refs #13822
13804: Fix tests
13804: Update comments for comments for "consecutive_idle_count"
Merge branch 'master' into 13804-no-shutdown-wanted-nodes
Merge branch '13804-no-shutdown-wanted-nodes' refs #13804
13868: Fallback behavior when arvados_node_size tag is missing
Merge branch '13868-invalid-node-size' refs #13868
13879: Arvbox support for arvados-controller
13546: crunch-job has timeout on srun_sync
13879: Fix cut and paste typo.
Merge branch '13879-arvados-server' closes #13879
Merge branch '13546-crunch1-timeout' refs #13546
13791: Health check endpoint docs wip
13791: Admin documentation for monitoring wip
Arvbox fix correct permissions to stop one-shot services. no issue #
13570: Add section describing RAM allocation behavior.
13791: More detail about monitoring
13791: Tweak doc ordering.
13570: Add formulas. Add link from cwl-style to crunch discussion.
13570: Doc tweaks
Merge branch '13570-crunch-ram-doc' closes #13570
arvbox uses keep config file instead of cli switches. no issue #
13892: Add jobs API deprecation notice to arvados-cwl-runner
Merge branch '13791-monitoring-docs' closes #13791
13690: Walkthrough of Composer on Arvados
13690: Fill in some missing steps.
13690: Add/fix links
Merge branch '13690-composer-user-guide' closes #13690
13892: Update disable_api_methods with the full list
13892: Make warning more obvious.
Merge branch '13892-deprecate-jobs-api' closes #13892
13609: Update version to 0.0.5
13609: Improve README fomatting a bit.
13609: Spelling fix
13609: Note that move/copy between collections isn't supported
Merge branch '13609-r-sdk-copy' refs #13609
13961: Separate config for cloudlist_poll_time, nodelist_poll_time, wishlist_poll_time
13961: Use poll_time as fallback
Merge branch '13961-separate-polling' refs #13961
Merge branch '13399-renice-too-low-priority-jobs' refs #13399
13931: Add test that file sizes are available.
13931: Support for getting file sizes of files in Keep
13931: Update cwltool for add_sizes fix
13931: Add keep size test to test list and fix Dockerfile for dev jobs image
13931: Update tests for sizes being added
Merge branch '13931-cwl-size' refs #13931
13931: Add dependencies required by latest cwltool
13931: Fix "don't rebuild packages that already exist" logic
13931: package deps for centos7
13931: Reorganize for consistency.
Merge branch '13931-cwltool-deps' refs #13931
13146: Shared endpoint wip
Fix version of 'typing' dep refs #13931
13976: Filter out keep references with secondaryFiles from list of files to upload
13976: Add code comment about only_real in upload_dependencies
Merge branch '13976-acr-keep-references' refs #13976
Fix version dependency on 'typing' refs #13931
13931: un-uriquote keep: URIs when converting to (collection, path) tuple
Merge branch '13931-uri-unquote' refs #13931
Merge branch 'jrandall/patch-4' refs #14017
13146: Support for query filters, "includes"
13146: Update comment
13146: Add tests for shared endpoint.
13146: FUSE uses the "groups.shared" API for shared/ directory.
13146: FUSE tests pass
13146: Add "include" parameter and rename return field to "included"
13146: Apply select clause to included objects
13146: "included" is returned as_api_response.
13146: Don't return "included" field unless non-nil
13146: Clarify documentation for shared endpoint
13973: Set priority of child container requests based on parent
Merge branch '13146-shared-rails' refs #13146
14082: Datatype validation for command, environment, mounts
14082: Fix test
Merge branch '14082-numbertype' refs #14082
13973: Tweak test for inheriting CR priority
14075: Fix cwl unit tests
14075: Tweak ruamel.yaml verision range in setup.py
Merge branch 'master' into 14075-uploadfiles
Merge branch '14075-uploadfiles' refs #14075
Merge branch '13973-child-priority' refs #13973
Build ruamel.yaml 0.15.26 for a-c-r
'trollop' gem renamed to 'optimist' fix build no issue #
Fix test for changed expectation refs #13973
Revert ruamel.yaml build to 0.14.12 refs #14075
Tom Clegg (315):
13111: Mount collections on demand in mnt/by_id/.
13111: Fix inode not updated during rename().
13111: .arvados#collection special file.
13111: Rearrange source files.
13111: Move code from fs_collection to fs_base and fs_filehandle.
13111: Save on sync().
13111: Clear up some type assertions.
13111: Combine newDirnode and newFIlenode to just newNode.
13111: Tidy up, add comments.
13111: Fix collectionfs crash.
13111: Sync name when changing parents.
13111: Update parent dir modtime when adding/removing/renaming.
13111: Port "rename a to a/b/c" fix from 11377b3b.
13111: Move SetParent() responsibility from Child() to caller.
13111: Merge branch 'master' into 12308-go-fuse
13111: Merge branch 'master' into 12308-go-fuse
13111: Improve deadlock prevention. Prevent unsupported renames.
13111: Fix nil pointer dereference at sitefs root.
13111: Merge branch 'master' into 12308-go-fuse
13111: Add projects hierarchy under "home".
13111: Add /home test case.
13111: Add spying client for testing deferred loading.
13111: Merge branch 'master' into 12308-go-fuse
13111: Propagate errors in Readdir().
13111: Propagate errors in Child().
13111: Reload project dir if fsync(2) was called since last load.
13111: Merge branch 'master' into 12308-go-fuse
13111: Test webdav with http basic authentication.
13111: Add tests for /users/ paths.
13111: Add /users virtual dir to siteFS.
13111: Expose read-only /users/ tree.
13111: Merge branch 'master' into 13111-webdav-projects
13111: Update tests to accept username in API response.
13100: Use collection FS for output and logs.
13111: Merge branch 'master' into 13111-webdav-projects
13111: Test /users/name and /users paths with cadaver.
13111: Clarify Child() comment.
13111: Skip collections named "." or "..".
13111: Serve "site filesystem" at keep-web root URL.
13111: Redirect /dir to /dir/ at siteFS paths.
13111: Update comment.
13111: Note safe use of append().
13100: Update comment.
13100: Handle writable collections mounted below output dir.
13100: Handle output symlinks to writable collections.
13100: 12606: Test chain of relative symlinks.
13100: Merge branch 'master' into 13100-crunch-run-output
Merge branch '13100-crunch-run-output'
13142: Raise open files limit.
Merge branch '13142-dispatch-nfiles'
13348: Don't submit child jobs/containers with submit=True.
Merge branch '13348-recursive-submit'
13111: Ignore projects and collections with "/" in name.
13111: Avoid double WriteHeader after implicit 200 on first Write.
13111: Fix skipped error check when Write called before WriteHeader.
13111: Serve /by_id/ dir.
13111: Follow attachment-only policy at /by_id/ and /users/ paths.
13111: Update docs.
13111: Avoid multi-page API reqs when looking up entries by name.
13111: Accept chunked responses to GET requests.
13111: Merge branch 'master' into 13111-webdav-projects
12708: Pull blocks to desired storage classes.
13111: Fix bogus form-encoding in example URL.
13111: Merge branch 'master' into 13111-webdav-projects
Merge branch '13111-webdav-projects'
13111: Access projects via /by_id/$uuid/.
13111: Update arvbox and package-build images to Go 1.9.5.
Update mock to match new arvados.File interface.
12167: Propagate X-Request-Id in API calls.
12167: Log keep-web requests and responses as JSON.
12167: Fix impossible fixture.
12167: Propagate X-Request-Id in keep requests.
12167: Propagate X-Request-Id in API calls.
13111: Merge branch 'master'
13111: Update to Go 1.10.1.
Merge branch '13111-projects-by-id'
12708: Precompute len(slots) to reduce mallocs. Rename setupCaches.
12708: TODO notes about memory use and class combinations.
12708: Merge branch 'master' into 12708-balance-storage-classes
13399: Log when a slurm job's priority is unrealistically low.
Merge branch '13399-log-low-priority'
Merge branch '12708-balance-storage-classes'
Avoid dangling child procs in test suite.
13111: Disable siteFS routes for anonymous requests.
Fix wrong event_type when getting buffered live logs for crunch2.
Fix #ZgotmplZ in dir listing page when filename contains a colon.
12167: Pass request ID via keepclient instead of custom code.
Merge branch '13111-no-anonymous-sitefs'
12626: Add arvados/v1/users/merge API.
12626: Check for scoped tokens.
12626: Log UUID of auth record (if any) instead of supplied token.
12167: Merge branch 'master'
12167: Generate a new request ID if caller explicitly uses "".
12167: Generate only one request ID per get/put.
12167: Test X-Request-ID headers.
Use released version of llfuse for tests.
Fix wrong temp path in govendor recipe.
Skip explicit libcloud preinstall when not using a fork.
Fix gopath/govendor recipe.
Skip some unnecessary pip invocations.
Avoid broken llfuse 1.3.4 ("pip install" fails).
12626: Update pg_terminate_backend recipe for PostgreSQL >= 9.2.
12626: Be more selective/explicit about changing various UUIDs.
12626: Change variable dst_user -> new_user for consistency.
12626: Improve error message.
12626: Advertise required params for users#merge in discovery doc.
12626: Add tests for missing new_owner_uuid.
Merge branch 'tests-cleanup'
Merge branch '12167-go-request-id'
13383: Add PullWorkers, TrashWorkers, EmptyTrashWorkers configs.
13383: Fix warning about expected 404 error.
13383: Implement EmptyTrashWorkers in azure and filesystem backends.
Merge branch '13383-trash-workers'
13278: Release jobs that are pending because "launch failed".
Merge branch '13278-launch-failed'
12626: Update permission graph after user merge.
12626: Copy omniauth info into auth account, not target account.
12626: Merge branch 'master' into 12626-merge-accounts
Fix run script syntax.
Merge branch '12626-merge-accounts'
12708: Log desired/short/surplus figures for each storage class.
Avoid http client race in test case.
Avoid copying mutex after first use.
13446: Enable TLS if given TLSCertificateFile and TLSKeyFile.
9918: Don't mask http/network error with response body format error.
Merge branch '9918-index-error'
Merge branch '13446-keepstore-tls'
13473: Drop slurm resource constraints when using instancetype=X.
13473: Tidy up slurmConstraintArgs func.
Merge branch '13473-no-cpu-constraints'
13164: Lock tables when updating multiple containers + CRs.
12737: Fix serialized column setup.
12737: Rename config serve_static_assets to serve_static_files.
Merge branch '13164-container-locking'
Merge branch '12708-report-storage-classes'
13164: Lock tables in order for container updates.
13164: Lock tables in order when locking a container.
Merge branch '13164-container-deadlock'
9918: Add RequestTimeout config for keep-balance.
13513: Fix deadlocked workers.
Merge branch '13513-balance-deadlock'
12061: Change busywait approach to preserve assertion messages.
12061: Don't round down to 1s in timestamp parser.
13517: Fix leaked goroutine and buffer on failed S3 write.
12061: Explicitly join actor thread after stopping.
Merge branch '12061-flaky-tests'
Merge branch '13517-buffer-leak'
12061: Allow sub-second poll times and sub-minute shutdown windows.
12061: Poll aggressively in integration tests.
12061: Don't wait around for test daemon to shutdown gracefully.
12061: Use shorter retry-after sleeps in integration test.
12061: Fix nodes marked busy while expecting shutdown decision.
12061: Reduce per-test timeout.
12061: Remove unreliable check.
12061: Disable nodemanager integration tests.
Merge branch '12061-nm-integration-tests'
13164: Remove locking. Clean up after races in background instead.
Merge branch '13164-cr-locking'
13456: Restart driver after each test.
More gopath symlink hacks.
Merge branch '13456-phantomjs-died'
13427: Ignore readonly devices mounted read-write elsewhere.
13427: Handle same backend device mounted RW in multiple places.
13427: Fix replication stats reporting for multiple-mounted devices.
13427: More symlink hack
13427: Use same index result N times if a device is mounted N times.
13497: Add controller, proxy to Rails API.
13497: Use basename($0) as subcommand, if it is one.
13497: Add arvados-server command.
13427: More symlink hack
13497: Build arvados-server and arvados-controller packages.
Merge branch '13427-multiple-mounts'
13594: Remove table lock.
13407: Take keepstore-reported volume replication into account.
13407: Fix stats reporting for mounts with replication>1.
13407: Test stats reporting for volumes with replication>1.
Merge branch '13407-volume-replication'
13497: Move common system service code to lib/service.
13497: Proxy requests to Rails API.
13497: Add timeout for proxy requests.
13497: Send request body when proxying.
13497: Don't propagate connection-oriented headers when proxying.
13497: Route API traffic through controller in test suites.
13497: Remove alternate integration test glue in Workbench.
13497: Set usable path for nginx request body buffering.
13497: Add controller to health check.
13497: Accept host key on command line.
13497: Add systemd unit to arvados-controller package.
13497: Send test server logs to {workspace}/tmp/*.log.
13497: Suppress nginx "info" logs from console.
13497: Quote file paths in test suite nginx.conf.
13497: Bump version numbers for Go packages when lib/ changes.
13497: Support "run-tests.sh --only go".
13497: Abort startup if Rails API cannot be found.
13497: Rename SystemNodes to NodeProfiles in config.
13497: Load env vars from /etc/arvados/environment in controller.
13497: Merge branch 'master' into 13497-controller
13164: Fix priority of containers that have priority=0 due to races.
13497: Merge branch 'master' into 13497-controller
13497: Ensure every test run produces something in tmp/*.log.
Merge branch '13497-controller'
Merge branch '13164-fix-zero-priority-after-race'
Ensure $WORKSPACE/tmp/ exists. Fix unchecked error.
13493: Proxy requests to remote clusters.
13493: Test PATCH request.
13164: Fix priority of containers that have priority=0 due to races.
13493: Salt tokens when forwarding requests to remote clusters.
13493: Move proxy and federation code to their own source files.
13493: Verify TLS unless config specifies insecure.
13493: Merge branch 'master' into 13493-federation-proxy
13674: Change InstanceTypes config from array to hash.
13674: Don't build heavy error unless needed.
13569: Accept metric/binary-prefixed units in RAM/scratch configs.
13569: Update dispatchcloud to use arvados.ByteSize.
13547: Fix up formatting.
13547: Avoid leaking insecure flag between different configs.
13569: Accept all JSON numbers (decimals, exponents), not just ints.
Merge branch '13569-size-prefixes'
13547: Merge branch 'master' into 13547-respect-insecure-flag-when-talking-ssl-to-keep
13547: Test insecure-tls flag passed through to pycurl.
Fix crash in error handler.
Fix startup warning by setting log_level explicitly.
13493: Remove old api_token query param when mangling tokens.
13493: Comment+cleanup tests.
13493: Check forwarded token.
Fix "order" param docs for groups#contents.
13493: Test DELETE and PUT methods.
13493: Fix excess newline.
13493: Return 502 Bad Gateway on proxy connection/proto failure.
13493: Add PostgreSQL connection to cluster config.
13493: Connect to database.
13493: Fix HMAC usage.
13493: Upgrade locally-issued v1 tokens to v2 when proxying.
13493: Test OPTIONS method.
Fix: subcommand not recognized in $0 when no CLI args are given.
13497: 13779: Forward Host header, and don't follow redirects.
13497: Merge branch 'master' into 13497-controller
13497: Fix bypassing controller in test suite after calling reset().
Fix slow tests skipped even when not in --short mode.
13497: Fix Rails redirect targets.
13497: Use X-Forwarded-Proto as scheme for keep-web redirect URLs.
13497: Avoid misdirecting POST to GET when request has double slash.
13497: Improve nginx logging in test suite.
13497: Add comments on odd Nginx config. Less confusing vhost name.
13497: Merge branch 'master' into 13497-controller
Merge branch '13497-controller'
13493: Merge branch 'master' into 13493-federation-proxy
13493: Fix proxy-redirect implementation.
13198: Move metrics to httpserver package.
13493: Update CORS OPTIONS test.
13198: Test case for keep-web metrics.
13198: Track & serve keep-web metrics.
13198: Document metrics endpoints.
13399: If slurm refuses to renice past 10K, stop trying.
13399: Recognize held jobs with priority between 1 and 20K.
13399: Link to wiki in warning message.
Merge branch '13399-nice-logs'
13788: Fix concurrent map write.
Merge branch '13198-keep-web-metrics'
Merge branch '13788-concurrent-map-write'
13493: Merge branch 'master' into 13493-federation-proxy
Merge branch '13493-federation-proxy'
13198: Add cache stats to metrics.
13497: Add controller to install docs.
13497: Add controller to upgrade notes.
13497: Link issues and commit hashes to redmine.
Merge branch '13198-keep-web-metrics'
Merge branch '13497-controller'
13934: Add comment to guard dispatch bugfix.
13939: Test orders with no table name qualifier.
13939: In groups#contents, apply unprefixed orders to all tables.
Merge branch '13934-guard-comment'
13399: Don't try to bury jobs below very-low-priority jobs.
13752: Dump structure with postgresql 9.6.
13399: Adjust locking strategy to avoid starvation.
13399: Don't spam logs about low priority jobs that are running.
13752: Remove column size limit.
13823: Build packages for ubuntu1804.
13823: Upgrade ws4py to version that ships in ubuntu 1804.
13823: Switch to stock fpm.
13823: Use pip instead of easy_install if easy_install3 is missing.
13823: Accept ruamel.yaml 0.15.x, build 0.15.34.
Merge branch '13399-lock-starvation'
13198: Remove old counters from status.json.
Merge branch '13198-keep-web-metrics'
Merge branch '13752-index-all-filenames'
Merge branch '13939-unqualified-contents-orders'
13558: Merge branch 'master' into wtsi-hgi-13558-debug-log-tag-req-id
Merge branch 'wtsi-hgi-13558-debug-log-tag-req-id'
13399: Don't block job submissions during release/renice batches.
Merge branch '13399-nb-squeue-check'
Note slow migration in upgrade notes.
13198: Note keep-web metrics in admin docs.
Merge branch '13198-metrics-docs'
13752: Don't rewrite entire file_names column during the migration.
Merge branch '13752-index-all-filenames'
13959: Default 5-minute timeout for API calls.
13959: Use logrus for crunch-dispatch-slurm logging.
13933: Fix formatting.
13933: Update error text check.
Merge branch '13933-dispatch-batch-size'
13959: Use logrus for crunch-dispatch-local logging.
13933: Update error message expectation.
13959: Merge branch 'master' into 13959-timeouts-and-logging
14020: Allow WebDAV headers in CORS requests.
13924: Add index for PDH lookups.
13924: Drop redundant index.
Merge branch '14020-webdav-cors'
Merge branch '13924-collections-index'
13959: Merge branch 'master' into 13959-timeouts-and-logging
13959: Fix test regexps.
13959: Merge branch 'master' into 13959-timeouts-and-logging
Merge branch '13959-timeouts-and-logging'
13456: Use HTTPRequestTimeout in test suite.
Merge branch '13456-controller-timeout'
Ward Vandewege (44):
Update instructions to get arvados node manager Azure credentials to v2
Remove remaining build script references to ubuntu1204, which we do not
libcloud 2.3.0 now requires python-requests 2.4.3 or higher, otherwise
Documentation: update PostgreSQL installation instructions
Merge branch 'thehyve/fix-crunch-documentation' Fix a typo in Crunch Dispatch installation manual
Fix another ineffassign warning in our golang code.
9918: Merge branch 'master' into 9918-index-timeout
9918: Merge branch '9918-index-timeout'
Add Veritas Genetics to the AUTHORS file.
2411: another pass at adding copyright headers where missing.
2411: Add support for R source files to build/check-copyright-notices
2411: add copyright headers to our R files.
2411: Add license header to sdk/R/README.Rmd
2411: Add a bunch of exceptions to .licenseignore for our R sdk
2411: Merge branch '2411-another-pass'
13547: make the Python SDK respect the ARVADOS_API_HOST_INSECURE environment
13547: if an instance of KeepClient has an api_client object with the insecure
Add basic documentation for the new Arvados on Kubernetes install option.
A series of documentation improvements based on review feedback.
13650: Merge branch 'master' into 13650-document-arvados-kubernetes
13650: Merge branch 'master' into 13650-document-arvados-kubernetes
13650: Merge branch '13650-document-arvados-kubernetes'
Further doc improvements for Arvados on Kubernetes.
13650: Merge branch '13650-document-arvados-kubernetes'
Minikube is not supported yet by our Arvados on Kubernetes Helm Chart.
Merge branch '13547-respect-insecure-flag-when-talking-ssl-to-keep'
Fix typo.
Reflect the new name and url for the public evaluation and trial instance,
Fix a few typos.
We no longer build for Ubuntu 12.04
Merge branch 'master' into 13823-bionic
Remove spurious error output in package testing script.
Remove spurious error output in package testing script.
Add support for package testing to our ubuntu 18.04 packages.
Merge branch '13823-bionic'
Fix for run-tests.sh: do not run sdk/R install if the --skip sdk/R option is
Bugfix for run-tests.sh: run sdk/R install if docs are being built,
Improvement for run-tests.sh: support multiple occurrences of the
Switch to use ha.pool.sks-keyservers.net everywhere, which is more reliable
We need to build ws 0.4.2 for ubuntu1404 and ubuntu1604
Documentation fix: make the use of $uuid_prefix consistent.
Make sure to bump the dependency on the arvados gems to the 1.2 series.
Change dependency from google-api-client to cure-google-api-client so
14153: Merge branch '14153-support-ruby-2.3.7'
strangepleasures (3):
Fix LocalJumpError in arvados-login-sync. Arvados-DCO-1.1-Signed-off-by: Pavel Mikhailovskii <pavel at thehyve.nl>
Merge remote-tracking branch 'upstream/master'
Simplify filtering in arvados-login-sync. Arvados-DCO-1.1-Signed-off-by: Pavel Mikhailovskii <pavel at thehyve.nl>
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list