[ARVADOS] created: 1.2.0

Git user git at public.curoverse.com
Tue Nov 13 16:57:06 EST 2018

        at  55dae8886e92de7737c8d89557d19be08af30f64 (tag)
   tagging  3c85ce36c1188474b6b94ad534daf51cb1497274 (commit)
  replaces  1.1.4
 tagged by  Fernando Monserrat
        on  Tue Nov 13 18:55:58 2018 -0300

no issue # . Remove incorrect tag and fix it for Release 1.2.0
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat at veritasgenetics.com>

Fernando Monserrat (38):
      13339: update gemspecs for new versioning
      13339: fix typo assigning git_latest_tag
      13339: fix git_latest_tag formatting
      13339: fix git_latest_tag formatting
      13339: fix git_hash variable
      refs #13339 "Merge branch '13339-update-gemspecs-new-versioning'".
      13369: update ubuntu1404 Dockerfile
      refs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
      13369: update ubuntu1404 test Dockerfile
      refs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
      13369: update ubuntu1404 test packages Dockerfile
      refs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
      Merge branch 'master' into 13369-update-ubuntu1404-dockerfile
      13369: add ulimit to the docker run fixing slowness bug on ubuntu1404
      refs #13369 "Merge branch '13369-update-ubuntu1404-dockerfile'".
      refs #13369 remove added packages install. rollback
      refs #13369 rollback packages install on ubuntu1404 Dockerfile for builds
      13667: Add version file to ROR packages
      13667: fix variable in adding version file to ROR packages
      13667: fix variable in adding version file to ROR packages
      13667: fix variable in adding version file to ROR packages
      13667: fix variable in adding version file to ROR packages
      13667: clean up version file generationx
      13667: test add config-file for rpm
      13667: test add config-file for rpm
      13667: test add PKG_VERSION in run-build-packages
      13667: remove PKG_VERSION in run-build-packages
      13667: remove PKG_VERSION in run-build-packages
      13667: test fail on rpm package
      13667: test fail on rpm package
      13667: test fail on rpm package
      13667: test fail on rpm package
      13667: test fail on rpm package
      13667: test fail on rpm package
      13667: test fail on rpm package
      13667: test fail on rpm package
      13667: clean up
      refs #13667 "Merge branch '13667-add-version-file'".

Fuad Muhic (70):
      Added generateAPI function which will autogenerate REST API
      Funtions that use POST verb now work correctley.
      Integrated autogenerated classes with existing content
      Merge branch 'master' of git.curoverse.com:arvados into 13076-r-autogen-api
      Added error handlind code and proper folding to autogenerated content
      Merge branch 'master' of git.curoverse.com:arvados into 13076-r-autogen-api
      Documentation for Arvados classes is generated automatically now
      Merge branch 'master' of git.curoverse.com:arvados into 13076-r-autogen-api
      Added feature to autogenerate documentation for Arvados methods.
      Merge branch 'master' of git.curoverse.com:arvados into 13076-r-autogen-api
      Added fix for discovery document inconsistency
      Merge branch 'master' of git.curoverse.com:arvados into 13076-r-autogen-api
      Refactored autoGenApi to be more readable
      Removed all autogenerated Arvados classes and updated rest of the code to reflect that
      Added listAll utility function which will always fetch all resources.
      Added project manipulation methods to autogenerated Arvados class.
      Merge branch 'master' of git.curoverse.com:arvados into 13076-r-autogen-api
      Filtered out all methods ending with index, show or destroy in Arvados class.
      Removed obsolated methods from HttpRequest class and updated tests to reflect that change.
      When env variable ARVADOS_API_HOST_INSECURE is set HTTP client will not try to validate the server's TLS certificate.
      Updated README and removed debug prints
      Arvados projects can be accessed from by_id directory in keep mount
      Users can only access groups with group_class set to project from by_id
      Optimized API call to not return non project groups
      Merge branch '12320-access-projects-from-by_id'
      Added auto-generated documentation for Arvados class.
      Added documentation for hardcoded Arvados class project methods.
      Method documentation now shows headline properly.
      Updated README and small fix for Subcollection docs.
      Added script to generate html doc for .Rmd files.
      Merge branch '13076-r-autogen-api'
      Replaced call to non-existent method execute in RESTService class with
      Added --storage-classes argument to arv-put.
      Added tests to arv-put to cover storage classes feature.
      Added tests and documentation for save and save_new methods in
      Updated save method of Collection class to always update when
      Added test that checks that saving a committed collection updates the storage_classes_desired
      Merge branch '13453-r-sdk-incorrect-rest-call-fix'
      Merge branch '13430-allow-storage-classes'
      Added properties and trash_at parameters to save and save_new methods
      Add storage classes supper to arvados cwl runner
      Add getters for properties and trash_at attributes and small bugfix for
      Add check for multiple storage classes in arvados-cwl-runner
      Fix ciso8601 issue in collections tests.
      Merge branch '13330-collection-save'
      Change multiple storage classes error handling in cwl_runner
      Merge branch 'master' of git.curoverse.com:arvados into 13429-cwl-runner-storage-classes-support
      Update storage_classes support for arvados_cwl_runner to work correctly
      Add storage_classes tests for arvados_cwl_runner
      Merge branch 'master' of git.curoverse.com:arvados into 13429-cwl-runner-storage-classes-support
      Merge branch '13330-intermediates-test' of git.curoverse.com:arvados into 13330-cwl-intermediate-collections-cleanup
      Update intermediate collection name, properties and trash_at atrribute
      Merge branch '13429-cwl-runner-storage-classes-support'
      Add runtime check for min R version.
      Update README to document min R version required for ArvadosR
      getConnection now works correctly when ARVADOS_API_HOST_INSECURE is set to TRUE
      Add tests for intermediate collections
      Merge branch '13613-document-min-r-version'
      Remove trailig white-space
      Merge branch 'master' of git.curoverse.com:arvados into 13330-cwl-intermediate-collections-cleanup
      Remove _get_intermediate_collection_info method and create function
      Update get_intermediate_collection_info and add owner_uuid to save_new
      Merge branch 'master' of git.curoverse.com:arvados into 13330-cwl-intermediate-collections-cleanup
      Merge branch '13330-cwl-intermediate-collections-cleanup'
      add tests for exec and getConnection methods in HttpRequest class
      remove trailing white space from all files
      Merge branch '13645-fix-arvados-api-host-insecure'
      Implement copy method, update move method and remove trailing
      Improve Collections create and move methods and update documentation
      Merge branch 'master' of git.curoverse.com:arvados into 13609-r-sdk-copy

Joshua C. Randall (4):
      13558: add req_id as a tag to Rails.logger
      fix "submitted container [uuid]" log from a-c-r
      Make dispatcher Run loop use count=none and BatchSize limit
      fix slow queries in uploadfiles

Lucas Di Pentima (116):
      13105: Set remoteHosts and remoteHostsViaDNS default values when missing.
      13105: discoveryDoc() call simplification.
      Merge branch '13105-wb-ignore-missing-remotes'
      12737: Updated gem dependencies, adressing a couple of known vulnerabilities.
      Merge branch '12737-api-security-deps-fixes'
      12737: Upgrade workbench from rails 4.1 to 4.2 (WIP)
      12737: Remove attribute from previous tests
      12737: Send serialized attributes when save()
      12737: Assign empty default values on serialized attributes
      12737: Fix controller tests
      12737: Fix integration tests
      Merge branch '12737-wb-rails42-upgrade'
      7478: Use our libcloud fork
      7478: Preemptable instances scheduling parameter on API server
      7478: Preemptable instances support on GoSDK & dispatchcloud instance chooser
      7478: Allow different configured node sizes to refer to the same instance type
      7478: On EC2 driver ask for a spot instance when needed.
      Merge branch 'master' into 7478-anm-spot-instances
      7478: Scheduling parameters validation fixes.
      7478: Ignore redundant instance_type field on CloudSizeWrapper
      7478: Override CloudSizeWrapper id with config Size name.
      7478: Set arvados-node-size-id tag on node creation.
      7478: Test fixes.
      7478: Unify arvados node size access across different drivers
      Merge branch 'master' into 7478-anm-spot-instances
      7478: Fixes integration tests.
      13559: Pin ciso8601 to version <2.0.0 due to API changes.
      Merge branch '13559-ciso8601-pin'
      Merge branch 'master' into 7478-anm-spot-instances
      13431: Adds storage class support to Keepclient.
      13431: Make keepproxy propagate X-Keep-Storage-Classes headers
      7478: Fixes child CR default preemptable setting logic.
      13431: Adds test confirming that keepproxy propagates X-Keep-Storage-Classes
      13431: Fixes unrelated test
      13431: Removes commented code
      Merge branch '13431-keepproxy-storage-classes-headers'
      13480: Adds test making the bug show itself.
      13480: Ignore nodes with no tags instead of crashing.
      Merge branch '13480-anm-tagless-azure-node'
      7478: Assign invalid cloud sizes to nodes with stale arvados_node_size tags
      Merge branch 'master' into 7478-anm-spot-instances
      7478: Config load fix & test updates about instance type decoupling
      7478: Adds tests on driver's create_node calls.
      7478: Adds test to check that state is 'down' with 'invalid' size.
      7478: Fixes GCE driver's arvados_node_size tag handling.
      13581: Available types reported to the user when CR is not satisfiable.
      7478: Moves invalid cloud size node's shutdown decision to proper method.
      Merge branch '7478-anm-spot-instances'
      7478: Fixes InvalidCloudSize creation. Adds wishlist related node info to logs.
      Merge branch '7478-invalid-size-not-defined'
      7478: Fix nodemanager's libcloud install dependency.
      Merge branch '7478-anm-libcloud-deps-fix'
      13581: Adds log message when no instance types are configured.
      13617: Replaces all logging about size names with size ids.
      Merge branch '13617-anm-instance-type-logging'
      Merge branch '13581-cr-not-satisfiable-report'
      7478: Replaces term 'preemptable' with 'preemptible'
      Merge branch 'master' into 7478-s-preemptable-preemptible
      Merge branch '7478-s-preemptable-preemptible'
      13649: Fixes default preemptible parameter when submitting committed child CRs
      13649: Merges 2 tests into one.
      Merge branch '13649-preemptible-committed-child-cr'
      7478: Replaces term 'preemptable' with 'preemptible'
      13649: Adds admin documentation describing how to set up spot instances.
      13219: Adds default max_run_time value with validation, documentation and tests.
      13219: Checks for expired run time and cancel container if needed.
      13649: Fixes in wording
      Merge branch '13649-spot-instances-doc'
      13219: Removes unnecessary default max_run_time scheduling parameter
      13219: Moves time out code from dispatch to crunch-run
      13219: Adds TimeLimit support on arvados-cwl-runner
      13219: Simplifies test
      Merge branch '13219-jobs-time-limit'
      13219: Adds TimeLimit support on Arvados CWL schema.
      13219: Adds documentation
      Merge branch '13219-arv-cwl-schema-fix'
      13501: Replaces subprocess module with subprocess32 on nodemanager.
      13501: Replaces subprocess with subprocess32 on arvados-cwl-runner
      13501: Replaces subprocess with subprocess32 on keepdocker.
      13501: Adds subprocess32 install dependency on python-client for keepdocker.
      13668: Adds support for package version on API server.
      13668: Adds package version to workbench. Reports versions on dialog.
      13668: Replaced package_version with packageVersion on discovery doc.
      13668: Updates VersionHelper on workbench to read packageVersion key
      Merge branch '13668-api-wb-package-version'
      Merge branch '13501-subprocess32'
      13803: Upgrade oj gem on API server by removing arvados-cli dependency
      13803: Upgrade oj dependency on arvados-cli gem.
      13803: Upgrade oj to latest on workbench
      13803: Remove oj_mimic_json gem.
      13803: Re-enable Oj's rails optimizations & fix time format test failures.
      13803: Fix behavior change on Oj 3.x
      Merge branch '13803-oj-gem-malloc-bug'
      13851: Reverted use of discoveryDoc() on findAPI()
      Merge branch '13851-wb-multisite-search'
      13888: Adds arvados-cli dependency back on API server
      Merge branch '13888-cli-dependency-on-api'
      12444: Adds tmpdir stats to crunchstat reporting
      12444: Allow crunch-run to provide tmpdir to crunchstat
      13910: Fixes arvados gem dependency on arvados-cli & arvados-login-sync
      Merge branch '12444-tmpdir-monitoring'
      Merge branch '13910-arvados-gem-retrieval'
      13920: Fixes google-api-client gem issue with ruby 2.3.7+
      Merge branch '13920-arvados-gem-with-newer-rubies'
      12762: Suppress SafeYAML warnings by setting its default mode.
      Merge branch '12762-safeyaml-warning'
      13976: Add CWL test exposing the bug
      13976: Fix keep reference error
      14028: Add test exposing the bug
      14028: Sanitize RedCloth's output.
      14028: Confirm that safe tags (like <b></b>) don't get cleaned by sanitize()
      Merge branch '14028-wb-textile-html-sanitization'
      13340: Add test to expose bug
      13340: Correctly show all locator links on a text.
      13340: Enhance regex to match non quoted locators
      Merge branch '13340-wb-keep-links'

Nico Cesar (1):
      13579: Admin documentation setting up Azure resources

Pavel Mikhailovskii (1):
      Fix a typo in Crunch Dispatch documentation. Arvados-DCO-1.1-Signed-off-by: Pavel Mikhailovskii <pavel at thehyve.nl>

Peter Amstutz (275):
      13108: Handle multiple instances of arv-keepdocker uploading same docker image
      13108: Use ThreadSafeApiCache
      13108: Simplify locking, add methods for recording process status
      13108: Work queue submit wip
      13108: Collect outputs in separate threads as well
      13108: Rename runnable_queue -> task_queue
      13108: Rename parallel_submit_count to thread_count
      13108: Fix tracking tasks in flight
      13108: Refactor task queue into its own class.
      13108: Fix tests.
      13108: Add copyright notice to task_queue.py
      13108: Add test for taskqueue
      Merge branch '13108-cwl-parallel-submit' closes #13108
      13108: Don't try to install signal handler in background thread
      13108: crunch_script uses safeapi
      13108: Refactor signal handling
      13108: Fix import
      Merge branch '13108-acr-threading-fixes' refs #13108
      fix cwltest junit-xml dependency refs #13108
      arv-put: restore_signal_handlers conditional on install_sig_handlers
      Bump arvados-python-client version dependency.
      13076: Get rid of pkgdown, just call knitr
      13076: R SDK online docs include both README and method docs.
      Make the runin-reqs less stingy about RAM refs #13301
      arvbox pin pip to 9.0.3.  Add LINES/COLUMNS to "arvbox shell", no issue #
      Fix R SDK online doc generation refs #13076
      Merge branch 'thehyve/master' LocalJumpError in arvados-login-sync
      Fix nodemanager tests.  no issue #
      13385: login-sync test works even if there is no fuse group
      Merge branch '13385-login-sync-test' refs #13385
      login-sync test fix also make docker group optional refs #13385
      Add tag to secrets test so it is recognized by new cwltest as "optional".
      13282: First pass at reorganizing
      13282: Add architecture section
      13282: Remove some obsolete crunchv1 documenation from python sdk.
      13282: Add note about supported platforms to install guide.
      13282: doc updates
      Merge branch '13282-admin-docs' refs #13282
      13377: Ensure that collection fetched by PDH has longest available lifetime.
      Merge branch '13377-block-sig-time' refs #13377
      13338: Running workflow from workbench puts collections in correct project.
      Node manager integration test writes to stderr in chunks.
      13365: Update release notes about a-c-r bug in v1.1.4.
      13365: Tweak text.
      Merge branch '13338-wb-run-wf-proj' refs #13338
      13365: Discover secondaryFiles associated with default files inputs
      13365: Reintroduce secondaryFile discovery.
      Bump cwltool version for secondaryFile checker support.
      13365: Bump schema-salad for better error formatting.
      13365: Bump cwltool, 3rd party package versions.
      13365: Add example error messages and highlight the changes.
      Merge branch '13365-note-bug-in-docs' refs #13365
      13365: Fix race condition between job submitted and event monitor threads
      13365: Further fixes to ensure output callback only happens once
      13365: Fix state logging
      13365: clean up discover_secondary_files
      13365: Add tests to confirm regressions are fixed.
      13365: Remove check for callback that now happens in a different place.
      Merge branch '13365-default-secondaryFiles' refs #13365
      13435: Update ruamel.yaml and ciso8601 versions.
      13435: Build scandir to satisfy pathlib2 dependency of a-c-r
      Merge branch '13435-ruamel-yaml-ver' closes #13435
      Tweak tests for slightly different ruamel.yaml behavior, refs #13435
      Fix subprocess32 version constraint, refs #13435
      12256: Fix anonymous directory default inputs with file literals
      12256: Add a bunch of test cases.
      12256: Handle default references in RunInSingleContainer
      12256: Bump cwltool version for workflow packing bugfix.
      12256: Pull up minimum ruamel.yaml version to get consistent output.
      12256: Only scan for referenced files first time after packing
      12256: Comments about workflow references / extra files.
      Merge branch '12256-cwl-defaults' closes #12256
      Bump arvados-python-client version for consistent dependencies
      Bump package iteration for a-c-r in run-build-docker-jobs-image.sh
      12995: Add link account page with confirmation.
      12995: Linking alternate login works.
      12995: Can merge accounts in both directions.
      12995: Can now merge with inactive accounts.
      12995: Add testing
      12995: Don't let admins lose admin privs by linking a non-admin account.
      12995: Bugfix to support following chained user uuid redirection.
      12995: Do nested logout/login to workaround SSO session.
      12995: Add user and admin documentation pages.
      12995: Add username to link_accounts page.
      12995: Fix tests
      Merge branch '12995-wb-merge-acct' refs #12995
      13330: Test workflow that generates intermediates.
      13330: Fix test workflow.
      13516: Bump cwltool, cwltest and subprocess32 versions
      Merge branch '13516-fix-cwl-tests' refs #13516
      Fix tests refs #13516
      Tweak arvados-cwl-runner subprocess32 version dependency refs #13516
      11162: Support public http and https file references
      11162: Handle missing git
      11162: Add --submit-request-uuid
      11162: Add progress for http data download.
      11162: Set a default cache time of 24 hours for files fetched over http.
      11162: Add logging about checking cache
      11162: Smarter http downloads.
      11162: Bump cwltool for fix in generating basename from url
      11162: Fix tests.
      11162: Add test for --submit-request-uuid
      11162: Add tests for http_to_keep.
      Replace unicode copyright character with ASCII in README.Rmd
      Fix crunch_script for jobs api refs #13516
      11162: Remove debugging check
      Merge branch '11162-wes-support' refs #11162
      11907: Add option to uploadfiles to disable packing.
      Merge branch '11907-uploadfile-no-pack' refs #11907
      11907: Make collections containing dependencies have predictable PDHs
      11907: Fix tests
      12960: Migrate properties fields to jsonb to support subproperty filters.
      11907: Fix timezone handling for HTTP timestamps.
      Merge branch '11907-acr-stable-pdh' refs #11907
      13562: Add "properties" field to groups, supports subproperty query.
      13562: Update documentation
      12960: Update documentation.
      13562: Migration needs to update full text search index on groups
      12960: Fix test due to slightly different serialization from jsonb
      Merge branch '12960-migrate-properties' closes #12960
      Merge branch 'master' into 13562-group-properties
      Merge branch '13562-group-properties' closes #13562
      13579: Keepstore install documentation now describes using config.yml
      13579: Remove some example output because I don't know if it is up to date.
      13579: Document ManagementToken.  Add information about trash behavior.
      13579: Update from comments
      docs wip
      13579: Admin documentation about storage classes
      13579: Add a little more introductory material on allocating resources
      13580: Add storage classes section to user guide
      13580: Add arvados-cwl-runner to storage class usage docs.
      API fixup
      13627: a-c-r migration to cwltool loading/runtimeContext API
      13627: Fix loadingContext used for reloading document.
      13627: Improve error reporting.
      13627: Add missing file.
      13627: Fix crunch_script
      13627: Fix ArvadosWorkflow use of Builder
      13627: Fix tests after rebase
      Merge branch '13579-keepstore-config-doc' refs #13579 refs #13580
      13627: Tweak test to possibly avoid name clash
      Merge branch '13627-update-cwltool' refs #13627
      13638: Restore logic of arvados-cwl-runner version is max(sdk, acr)
      Merge branch '13638-acr-version' refs #13638
      Add mypy-extensions to build.list refs #13627
      13627: Make sure work_api is set on runtimeContext.
      Merge branch '13627-set-work-api' refs #13627
      13255: Account activate docs wip
      13255: Edits
      13255: More discussion of federated users in account activation
      13255: Implement auto_activate_users_from
      13255: Add test for auto_activate_users_from
      13255: Doc format tweaks
      13255: Add documentation & test that remote users can be pre-activated.
      Merge branch 'master' into 13255-account-activation
      Merge branch '13255-account-activation' closes #13255
      Add mypy-extensions to build.list refs #13627
      13627: Make sure work_api is set on runtimeContext.
      13692: Make Python SDK error reporting more user friendly.
      Merge branch '13692-acr-error' closes #13692
      13766: upload_workflow dumps to json
      13681: Implement WorkflowRunnerResources hint
      13681: Can use --submit-runner-ram with --create/update-workflow
      13681:  Update documentation for arv:WorkflowRunnerResources
      13681: Add new test file
      13681: Revert to yaml.round_trip_dump instead of json.dumps().
      13681: Fix workbench for workflows that don't have "hints"
      Merge branch '13681-wb-acr-submit-ram' closes #13681
      13767: Update crunch-dispatch-slurm install docs
      Merge branch '13766-libyaml-workaround' refs #13766
      12983: Periodically check for containerd
      12983: Command line option takes duration.  Add tests.
      12983: Add another error message to the "broken node" blacklist.
      12983: Tweak error regex
      13780: Update sprockets gem
      13690: Add Composer install documentation
      Merge branch '13780-sprockets' closes #13780
      12983: Change break -> return to exit goroutine, add comment
      Merge branch '12983-containerd' closes #12983
      13767: Tweak text for ReserveExtraRAM to specify default value.
      Merge branch '13767-reserve-ram-doc' closes #13767
      Merge branch '13690-composer-install' refs #13690
      13799: Install doc refactor WIP
      13799: More install doc refactor
      Merge branch '13799-install-doc-sections' refs #13799
      13822: Don't call list_sizes() in cloud client constructor.
      13804: Node must report as "idle" two consecutive times
      13804: Try to cancel pending shutdowns if nodes are needed
      13804: Smarter shutdown behavior WIP
      Merge branch 'master' into 13822-nm-delayed-daemon
      Merge branch '13822-nm-delayed-daemon' refs #13822
      13804: Fix tests
      13804: Update comments for comments for "consecutive_idle_count"
      Merge branch 'master' into 13804-no-shutdown-wanted-nodes
      Merge branch '13804-no-shutdown-wanted-nodes' refs #13804
      13868: Fallback behavior when arvados_node_size tag is missing
      Merge branch '13868-invalid-node-size' refs #13868
      13879: Arvbox support for arvados-controller
      13546: crunch-job has timeout on srun_sync
      13879: Fix cut and paste typo.
      Merge branch '13879-arvados-server' closes #13879
      Merge branch '13546-crunch1-timeout' refs #13546
      13791: Health check endpoint docs wip
      13791: Admin documentation for monitoring wip
      Arvbox fix correct permissions to stop one-shot services. no issue #
      13570: Add section describing RAM allocation behavior.
      13791: More detail about monitoring
      13791: Tweak doc ordering.
      13570: Add formulas.  Add link from cwl-style to crunch discussion.
      13570: Doc tweaks
      Merge branch '13570-crunch-ram-doc' closes #13570
      arvbox uses keep config file instead of cli switches.  no issue #
      13892: Add jobs API deprecation notice to arvados-cwl-runner
      Merge branch '13791-monitoring-docs' closes #13791
      13690: Walkthrough of Composer on Arvados
      13690: Fill in some missing steps.
      13690: Add/fix links
      Merge branch '13690-composer-user-guide' closes #13690
      13892: Update disable_api_methods with the full list
      13892: Make warning more obvious.
      Merge branch '13892-deprecate-jobs-api' closes #13892
      13609: Update version to 0.0.5
      13609: Improve README fomatting a bit.
      13609: Spelling fix
      13609: Note that move/copy between collections isn't supported
      Merge branch '13609-r-sdk-copy' refs #13609
      13961: Separate config for cloudlist_poll_time, nodelist_poll_time, wishlist_poll_time
      13961: Use poll_time as fallback
      Merge branch '13961-separate-polling' refs #13961
      Merge branch '13399-renice-too-low-priority-jobs' refs #13399
      13931: Add test that file sizes are available.
      13931: Support for getting file sizes of files in Keep
      13931: Update cwltool for add_sizes fix
      13931: Add keep size test to test list and fix Dockerfile for dev jobs image
      13931: Update tests for sizes being added
      Merge branch '13931-cwl-size' refs #13931
      13931: Add dependencies required by latest cwltool
      13931: Fix "don't rebuild packages that already exist" logic
      13931: package deps for centos7
      13931: Reorganize for consistency.
      Merge branch '13931-cwltool-deps' refs #13931
      13146: Shared endpoint wip
      Fix version of 'typing' dep refs #13931
      13976: Filter out keep references with secondaryFiles from list of files to upload
      13976: Add code comment about only_real in upload_dependencies
      Merge branch '13976-acr-keep-references' refs #13976
      Fix version dependency on 'typing' refs #13931
      13931: un-uriquote keep: URIs when converting to (collection, path) tuple
      Merge branch '13931-uri-unquote' refs #13931
      Merge branch 'jrandall/patch-4' refs #14017
      13146: Support for query filters, "includes"
      13146: Update comment
      13146: Add tests for shared endpoint.
      13146: FUSE uses the "groups.shared" API for shared/ directory.
      13146: FUSE tests pass
      13146: Add "include" parameter and rename return field to "included"
      13146: Apply select clause to included objects
      13146: "included" is returned as_api_response.
      13146: Don't return "included" field unless non-nil
      13146: Clarify documentation for shared endpoint
      13973: Set priority of child container requests based on parent
      Merge branch '13146-shared-rails' refs #13146
      14082: Datatype validation for command, environment, mounts
      14082: Fix test
      Merge branch '14082-numbertype' refs #14082
      13973: Tweak test for inheriting CR priority
      14075: Fix cwl unit tests
      14075: Tweak ruamel.yaml verision range in setup.py
      Merge branch 'master' into 14075-uploadfiles
      Merge branch '14075-uploadfiles' refs #14075
      Merge branch '13973-child-priority' refs #13973
      Build ruamel.yaml 0.15.26 for a-c-r
      'trollop' gem renamed to 'optimist' fix build no issue #
      Fix test for changed expectation refs #13973
      Revert ruamel.yaml build to 0.14.12 refs #14075

Tom Clegg (315):
      13111: Mount collections on demand in mnt/by_id/.
      13111: Fix inode not updated during rename().
      13111: .arvados#collection special file.
      13111: Rearrange source files.
      13111: Move code from fs_collection to fs_base and fs_filehandle.
      13111: Save on sync().
      13111: Clear up some type assertions.
      13111: Combine newDirnode and newFIlenode to just newNode.
      13111: Tidy up, add comments.
      13111: Fix collectionfs crash.
      13111: Sync name when changing parents.
      13111: Update parent dir modtime when adding/removing/renaming.
      13111: Port "rename a to a/b/c" fix from 11377b3b.
      13111: Move SetParent() responsibility from Child() to caller.
      13111: Merge branch 'master' into 12308-go-fuse
      13111: Merge branch 'master' into 12308-go-fuse
      13111: Improve deadlock prevention. Prevent unsupported renames.
      13111: Fix nil pointer dereference at sitefs root.
      13111: Merge branch 'master' into 12308-go-fuse
      13111: Add projects hierarchy under "home".
      13111: Add /home test case.
      13111: Add spying client for testing deferred loading.
      13111: Merge branch 'master' into 12308-go-fuse
      13111: Propagate errors in Readdir().
      13111: Propagate errors in Child().
      13111: Reload project dir if fsync(2) was called since last load.
      13111: Merge branch 'master' into 12308-go-fuse
      13111: Test webdav with http basic authentication.
      13111: Add tests for /users/ paths.
      13111: Add /users virtual dir to siteFS.
      13111: Expose read-only /users/ tree.
      13111: Merge branch 'master' into 13111-webdav-projects
      13111: Update tests to accept username in API response.
      13100: Use collection FS for output and logs.
      13111: Merge branch 'master' into 13111-webdav-projects
      13111: Test /users/name and /users paths with cadaver.
      13111: Clarify Child() comment.
      13111: Skip collections named "." or "..".
      13111: Serve "site filesystem" at keep-web root URL.
      13111: Redirect /dir to /dir/ at siteFS paths.
      13111: Update comment.
      13111: Note safe use of append().
      13100: Update comment.
      13100: Handle writable collections mounted below output dir.
      13100: Handle output symlinks to writable collections.
      13100: 12606: Test chain of relative symlinks.
      13100: Merge branch 'master' into 13100-crunch-run-output
      Merge branch '13100-crunch-run-output'
      13142: Raise open files limit.
      Merge branch '13142-dispatch-nfiles'
      13348: Don't submit child jobs/containers with submit=True.
      Merge branch '13348-recursive-submit'
      13111: Ignore projects and collections with "/" in name.
      13111: Avoid double WriteHeader after implicit 200 on first Write.
      13111: Fix skipped error check when Write called before WriteHeader.
      13111: Serve /by_id/ dir.
      13111: Follow attachment-only policy at /by_id/ and /users/ paths.
      13111: Update docs.
      13111: Avoid multi-page API reqs when looking up entries by name.
      13111: Accept chunked responses to GET requests.
      13111: Merge branch 'master' into 13111-webdav-projects
      12708: Pull blocks to desired storage classes.
      13111: Fix bogus form-encoding in example URL.
      13111: Merge branch 'master' into 13111-webdav-projects
      Merge branch '13111-webdav-projects'
      13111: Access projects via /by_id/$uuid/.
      13111: Update arvbox and package-build images to Go 1.9.5.
      Update mock to match new arvados.File interface.
      12167: Propagate X-Request-Id in API calls.
      12167: Log keep-web requests and responses as JSON.
      12167: Fix impossible fixture.
      12167: Propagate X-Request-Id in keep requests.
      12167: Propagate X-Request-Id in API calls.
      13111: Merge branch 'master'
      13111: Update to Go 1.10.1.
      Merge branch '13111-projects-by-id'
      12708: Precompute len(slots) to reduce mallocs. Rename setupCaches.
      12708: TODO notes about memory use and class combinations.
      12708: Merge branch 'master' into 12708-balance-storage-classes
      13399: Log when a slurm job's priority is unrealistically low.
      Merge branch '13399-log-low-priority'
      Merge branch '12708-balance-storage-classes'
      Avoid dangling child procs in test suite.
      13111: Disable siteFS routes for anonymous requests.
      Fix wrong event_type when getting buffered live logs for crunch2.
      Fix #ZgotmplZ in dir listing page when filename contains a colon.
      12167: Pass request ID via keepclient instead of custom code.
      Merge branch '13111-no-anonymous-sitefs'
      12626: Add arvados/v1/users/merge API.
      12626: Check for scoped tokens.
      12626: Log UUID of auth record (if any) instead of supplied token.
      12167: Merge branch 'master'
      12167: Generate a new request ID if caller explicitly uses "".
      12167: Generate only one request ID per get/put.
      12167: Test X-Request-ID headers.
      Use released version of llfuse for tests.
      Fix wrong temp path in govendor recipe.
      Skip explicit libcloud preinstall when not using a fork.
      Fix gopath/govendor recipe.
      Skip some unnecessary pip invocations.
      Avoid broken llfuse 1.3.4 ("pip install" fails).
      12626: Update pg_terminate_backend recipe for PostgreSQL >= 9.2.
      12626: Be more selective/explicit about changing various UUIDs.
      12626: Change variable dst_user -> new_user for consistency.
      12626: Improve error message.
      12626: Advertise required params for users#merge in discovery doc.
      12626: Add tests for missing new_owner_uuid.
      Merge branch 'tests-cleanup'
      Merge branch '12167-go-request-id'
      13383: Add PullWorkers, TrashWorkers, EmptyTrashWorkers configs.
      13383: Fix warning about expected 404 error.
      13383: Implement EmptyTrashWorkers in azure and filesystem backends.
      Merge branch '13383-trash-workers'
      13278: Release jobs that are pending because "launch failed".
      Merge branch '13278-launch-failed'
      12626: Update permission graph after user merge.
      12626: Copy omniauth info into auth account, not target account.
      12626: Merge branch 'master' into 12626-merge-accounts
      Fix run script syntax.
      Merge branch '12626-merge-accounts'
      12708: Log desired/short/surplus figures for each storage class.
      Avoid http client race in test case.
      Avoid copying mutex after first use.
      13446: Enable TLS if given TLSCertificateFile and TLSKeyFile.
      9918: Don't mask http/network error with response body format error.
      Merge branch '9918-index-error'
      Merge branch '13446-keepstore-tls'
      13473: Drop slurm resource constraints when using instancetype=X.
      13473: Tidy up slurmConstraintArgs func.
      Merge branch '13473-no-cpu-constraints'
      13164: Lock tables when updating multiple containers + CRs.
      12737: Fix serialized column setup.
      12737: Rename config serve_static_assets to serve_static_files.
      Merge branch '13164-container-locking'
      Merge branch '12708-report-storage-classes'
      13164: Lock tables in order for container updates.
      13164: Lock tables in order when locking a container.
      Merge branch '13164-container-deadlock'
      9918: Add RequestTimeout config for keep-balance.
      13513: Fix deadlocked workers.
      Merge branch '13513-balance-deadlock'
      12061: Change busywait approach to preserve assertion messages.
      12061: Don't round down to 1s in timestamp parser.
      13517: Fix leaked goroutine and buffer on failed S3 write.
      12061: Explicitly join actor thread after stopping.
      Merge branch '12061-flaky-tests'
      Merge branch '13517-buffer-leak'
      12061: Allow sub-second poll times and sub-minute shutdown windows.
      12061: Poll aggressively in integration tests.
      12061: Don't wait around for test daemon to shutdown gracefully.
      12061: Use shorter retry-after sleeps in integration test.
      12061: Fix nodes marked busy while expecting shutdown decision.
      12061: Reduce per-test timeout.
      12061: Remove unreliable check.
      12061: Disable nodemanager integration tests.
      Merge branch '12061-nm-integration-tests'
      13164: Remove locking. Clean up after races in background instead.
      Merge branch '13164-cr-locking'
      13456: Restart driver after each test.
      More gopath symlink hacks.
      Merge branch '13456-phantomjs-died'
      13427: Ignore readonly devices mounted read-write elsewhere.
      13427: Handle same backend device mounted RW in multiple places.
      13427: Fix replication stats reporting for multiple-mounted devices.
      13427: More symlink hack
      13427: Use same index result N times if a device is mounted N times.
      13497: Add controller, proxy to Rails API.
      13497: Use basename($0) as subcommand, if it is one.
      13497: Add arvados-server command.
      13427: More symlink hack
      13497: Build arvados-server and arvados-controller packages.
      Merge branch '13427-multiple-mounts'
      13594: Remove table lock.
      13407: Take keepstore-reported volume replication into account.
      13407: Fix stats reporting for mounts with replication>1.
      13407: Test stats reporting for volumes with replication>1.
      Merge branch '13407-volume-replication'
      13497: Move common system service code to lib/service.
      13497: Proxy requests to Rails API.
      13497: Add timeout for proxy requests.
      13497: Send request body when proxying.
      13497: Don't propagate connection-oriented headers when proxying.
      13497: Route API traffic through controller in test suites.
      13497: Remove alternate integration test glue in Workbench.
      13497: Set usable path for nginx request body buffering.
      13497: Add controller to health check.
      13497: Accept host key on command line.
      13497: Add systemd unit to arvados-controller package.
      13497: Send test server logs to {workspace}/tmp/*.log.
      13497: Suppress nginx "info" logs from console.
      13497: Quote file paths in test suite nginx.conf.
      13497: Bump version numbers for Go packages when lib/ changes.
      13497: Support "run-tests.sh --only go".
      13497: Abort startup if Rails API cannot be found.
      13497: Rename SystemNodes to NodeProfiles in config.
      13497: Load env vars from /etc/arvados/environment in controller.
      13497: Merge branch 'master' into 13497-controller
      13164: Fix priority of containers that have priority=0 due to races.
      13497: Merge branch 'master' into 13497-controller
      13497: Ensure every test run produces something in tmp/*.log.
      Merge branch '13497-controller'
      Merge branch '13164-fix-zero-priority-after-race'
      Ensure $WORKSPACE/tmp/ exists. Fix unchecked error.
      13493: Proxy requests to remote clusters.
      13493: Test PATCH request.
      13164: Fix priority of containers that have priority=0 due to races.
      13493: Salt tokens when forwarding requests to remote clusters.
      13493: Move proxy and federation code to their own source files.
      13493: Verify TLS unless config specifies insecure.
      13493: Merge branch 'master' into 13493-federation-proxy
      13674: Change InstanceTypes config from array to hash.
      13674: Don't build heavy error unless needed.
      13569: Accept metric/binary-prefixed units in RAM/scratch configs.
      13569: Update dispatchcloud to use arvados.ByteSize.
      13547: Fix up formatting.
      13547: Avoid leaking insecure flag between different configs.
      13569: Accept all JSON numbers (decimals, exponents), not just ints.
      Merge branch '13569-size-prefixes'
      13547: Merge branch 'master' into 13547-respect-insecure-flag-when-talking-ssl-to-keep
      13547: Test insecure-tls flag passed through to pycurl.
      Fix crash in error handler.
      Fix startup warning by setting log_level explicitly.
      13493: Remove old api_token query param when mangling tokens.
      13493: Comment+cleanup tests.
      13493: Check forwarded token.
      Fix "order" param docs for groups#contents.
      13493: Test DELETE and PUT methods.
      13493: Fix excess newline.
      13493: Return 502 Bad Gateway on proxy connection/proto failure.
      13493: Add PostgreSQL connection to cluster config.
      13493: Connect to database.
      13493: Fix HMAC usage.
      13493: Upgrade locally-issued v1 tokens to v2 when proxying.
      13493: Test OPTIONS method.
      Fix: subcommand not recognized in $0 when no CLI args are given.
      13497: 13779: Forward Host header, and don't follow redirects.
      13497: Merge branch 'master' into 13497-controller
      13497: Fix bypassing controller in test suite after calling reset().
      Fix slow tests skipped even when not in --short mode.
      13497: Fix Rails redirect targets.
      13497: Use X-Forwarded-Proto as scheme for keep-web redirect URLs.
      13497: Avoid misdirecting POST to GET when request has double slash.
      13497: Improve nginx logging in test suite.
      13497: Add comments on odd Nginx config. Less confusing vhost name.
      13497: Merge branch 'master' into 13497-controller
      Merge branch '13497-controller'
      13493: Merge branch 'master' into 13493-federation-proxy
      13493: Fix proxy-redirect implementation.
      13198: Move metrics to httpserver package.
      13493: Update CORS OPTIONS test.
      13198: Test case for keep-web metrics.
      13198: Track & serve keep-web metrics.
      13198: Document metrics endpoints.
      13399: If slurm refuses to renice past 10K, stop trying.
      13399: Recognize held jobs with priority between 1 and 20K.
      13399: Link to wiki in warning message.
      Merge branch '13399-nice-logs'
      13788: Fix concurrent map write.
      Merge branch '13198-keep-web-metrics'
      Merge branch '13788-concurrent-map-write'
      13493: Merge branch 'master' into 13493-federation-proxy
      Merge branch '13493-federation-proxy'
      13198: Add cache stats to metrics.
      13497: Add controller to install docs.
      13497: Add controller to upgrade notes.
      13497: Link issues and commit hashes to redmine.
      Merge branch '13198-keep-web-metrics'
      Merge branch '13497-controller'
      13934: Add comment to guard dispatch bugfix.
      13939: Test orders with no table name qualifier.
      13939: In groups#contents, apply unprefixed orders to all tables.
      Merge branch '13934-guard-comment'
      13399: Don't try to bury jobs below very-low-priority jobs.
      13752: Dump structure with postgresql 9.6.
      13399: Adjust locking strategy to avoid starvation.
      13399: Don't spam logs about low priority jobs that are running.
      13752: Remove column size limit.
      13823: Build packages for ubuntu1804.
      13823: Upgrade ws4py to version that ships in ubuntu 1804.
      13823: Switch to stock fpm.
      13823: Use pip instead of easy_install if easy_install3 is missing.
      13823: Accept ruamel.yaml 0.15.x, build 0.15.34.
      Merge branch '13399-lock-starvation'
      13198: Remove old counters from status.json.
      Merge branch '13198-keep-web-metrics'
      Merge branch '13752-index-all-filenames'
      Merge branch '13939-unqualified-contents-orders'
      13558: Merge branch 'master' into wtsi-hgi-13558-debug-log-tag-req-id
      Merge branch 'wtsi-hgi-13558-debug-log-tag-req-id'
      13399: Don't block job submissions during release/renice batches.
      Merge branch '13399-nb-squeue-check'
      Note slow migration in upgrade notes.
      13198: Note keep-web metrics in admin docs.
      Merge branch '13198-metrics-docs'
      13752: Don't rewrite entire file_names column during the migration.
      Merge branch '13752-index-all-filenames'
      13959: Default 5-minute timeout for API calls.
      13959: Use logrus for crunch-dispatch-slurm logging.
      13933: Fix formatting.
      13933: Update error text check.
      Merge branch '13933-dispatch-batch-size'
      13959: Use logrus for crunch-dispatch-local logging.
      13933: Update error message expectation.
      13959: Merge branch 'master' into 13959-timeouts-and-logging
      14020: Allow WebDAV headers in CORS requests.
      13924: Add index for PDH lookups.
      13924: Drop redundant index.
      Merge branch '14020-webdav-cors'
      Merge branch '13924-collections-index'
      13959: Merge branch 'master' into 13959-timeouts-and-logging
      13959: Fix test regexps.
      13959: Merge branch 'master' into 13959-timeouts-and-logging
      Merge branch '13959-timeouts-and-logging'
      13456: Use HTTPRequestTimeout in test suite.
      Merge branch '13456-controller-timeout'

Ward Vandewege (44):
      Update instructions to get arvados node manager Azure credentials to v2
      Remove remaining build script references to ubuntu1204, which we do not
      libcloud 2.3.0 now requires python-requests 2.4.3 or higher, otherwise
      Documentation: update PostgreSQL installation instructions
      Merge branch 'thehyve/fix-crunch-documentation' Fix a typo in Crunch Dispatch installation manual
      Fix another ineffassign warning in our golang code.
      9918: Merge branch 'master' into 9918-index-timeout
      9918: Merge branch '9918-index-timeout'
      Add Veritas Genetics to the AUTHORS file.
      2411: another pass at adding copyright headers where missing.
      2411: Add support for R source files to build/check-copyright-notices
      2411: add copyright headers to our R files.
      2411: Add license header to sdk/R/README.Rmd
      2411: Add a bunch of exceptions to .licenseignore for our R sdk
      2411: Merge branch '2411-another-pass'
      13547: make the Python SDK respect the ARVADOS_API_HOST_INSECURE environment
      13547: if an instance of KeepClient has an api_client object with the insecure
      Add basic documentation for the new Arvados on Kubernetes install option.
      A series of documentation improvements based on review feedback.
      13650: Merge branch 'master' into 13650-document-arvados-kubernetes
      13650: Merge branch 'master' into 13650-document-arvados-kubernetes
      13650: Merge branch '13650-document-arvados-kubernetes'
      Further doc improvements for Arvados on Kubernetes.
      13650: Merge branch '13650-document-arvados-kubernetes'
      Minikube is not supported yet by our Arvados on Kubernetes Helm Chart.
      Merge branch '13547-respect-insecure-flag-when-talking-ssl-to-keep'
      Fix typo.
      Reflect the new name and url for the public evaluation and trial instance,
      Fix a few typos.
      We no longer build for Ubuntu 12.04
      Merge branch 'master' into 13823-bionic
      Remove spurious error output in package testing script.
      Remove spurious error output in package testing script.
      Add support for package testing to our ubuntu 18.04 packages.
      Merge branch '13823-bionic'
      Fix for run-tests.sh: do not run sdk/R install if the --skip sdk/R option is
      Bugfix for run-tests.sh: run sdk/R install if docs are being built,
      Improvement for run-tests.sh: support multiple occurrences of the
      Switch to use ha.pool.sks-keyservers.net everywhere, which is more reliable
      We need to build ws 0.4.2 for ubuntu1404 and ubuntu1604
      Documentation fix: make the use of $uuid_prefix consistent.
      Make sure to bump the dependency on the arvados gems to the 1.2 series.
      Change dependency from google-api-client to cure-google-api-client so
      14153: Merge branch '14153-support-ruby-2.3.7'

strangepleasures (3):
      Fix LocalJumpError in arvados-login-sync. Arvados-DCO-1.1-Signed-off-by: Pavel Mikhailovskii <pavel at thehyve.nl>
      Merge remote-tracking branch 'upstream/master'
      Simplify filtering in arvados-login-sync. Arvados-DCO-1.1-Signed-off-by: Pavel Mikhailovskii <pavel at thehyve.nl>



More information about the arvados-commits mailing list