[ARVADOS] created: 1.1.0
Git user
git at public.curoverse.com
Tue Oct 24 09:40:31 EDT 2017
at 17052ce4d0305de11b338689d1b5b3765facf76c (tag)
tagging b25f2bdceae4750c8c803853874227dc2d175767 (commit)
tagged by Fernando Monserrat
on Tue Oct 24 10:40:06 2017 -0300
12389: Tag release for roche 1.1.0
Arvados-DCO-1.1-Signed-off-by: Fernando Monserrat <fmonserrat at veritasgenetics.com>
Adam Savitzky (2):
Patch socket configuration for Apple systems
Monkey patch socket constants on apple only
Brett Smith (1078):
doc: Suggest setting User in .ssh/config.
doc: Highlight `you` in the first script job spec.
doc: Update job1's expected output.
doc: Consistent style for `you` in job-debug.
doc: Fix hello-world-fixed.py name in job-debug.
doc: Bold `you` in tutorial-parallel's job spec.
doc: Small bugfixes for first pipeline template.
doc: Consistent style in new pipeline tutorial.
doc: Consistent style in outside program tutorial.
doc: Improve flow of CLI pipeline run.
doc: Make CLI pipeline run up-to-date.
doc: Improve "How Keep Works" flow.
doc: Note warning for get one collection block.
doc: Make Workbench install guide up-to-date.
api: Improve compute_node_domain handling.
js: Add identifiers to X-Editable templates.
workbench: Add Capybara gem for integration tests.
workbench: Add integration test helper module.
workbench: Use Poltergeist for integration tests.
workbench: Add VM integration tests.
workbench: Add basic login integration tests.
workbench: Document integration test dependencies.
workbench: Make integration test settings default.
workbench: Fix typo in application.default.yml.
Merge branch '2288-integration-tests'
workbench: Improve integration test environment.
workbench: Improve API server integration launch.
workbench: Tweak integration test environment.
workbench: Make SERVER_PID_PATH a constant.
workbench: Expound Bundle+integration interaction.
workbench: Bundle integration env bugfix.
workbench: Add developer docs.
workbench: Add ids to other navbar dropdowns.
workbench: Add basic smoke test.
workbench: Document deployment Bundle testing.
workbench: Improve integration test Rails launch.
workbench: Timeout integration test Rails launch.
Merge branch '2288-smoke-test'
doc: Make index adorable.
doc: Style and consistency in user/index.
doc: Fix ssh-access typos and consistency.
doc: Fix api-tokens typos and consistency.
doc: Fix tutorial-keep typos and consistency.
doc: tutorial-pipeline-workbench style+clarity.
doc: tutorial-firstscript style consistency.
doc: tutorial-new-pipeline clarity.
doc: Tweak running-external-program style.
doc: Use $USER in tutorial JSON.
doc: Highlight "you" in reference JSON.
doc: Make running-pipeline-command up-to-date.
doc: tutorial-job-debug update and style.
doc: Fix typos in tutorial-parallel.
doc: Keep topic style tweaks.
doc: job-and-pipeline-ref grammar fixes.
doc: tutorial-job1 updates and style tweaks.
doc: Implement Tom's suggested improvements.
doc: Style and consistency in user/index.
doc: Fix ssh-access typos and consistency.
doc: Fix api-tokens typos and consistency.
doc: Fix tutorial-keep typos and consistency.
doc: tutorial-pipeline-workbench style+clarity.
doc: tutorial-firstscript style consistency.
doc: tutorial-new-pipeline clarity.
doc: Tweak running-external-program style.
doc: Use $USER in tutorial JSON.
doc: Highlight "you" in reference JSON.
doc: Make running-pipeline-command up-to-date.
doc: tutorial-job-debug update and style.
doc: Fix typos in tutorial-parallel.
doc: Keep topic style tweaks.
doc: job-and-pipeline-ref grammar fixes.
doc: tutorial-job1 updates and style tweaks.
doc: Implement Tom's suggested improvements.
Merge branch '2470-doc-refresh'.
doc: Fix Keep links in tutorial-job1.
docker: Improve build.sh's Makefile finding.
docker: Don't try shell expansion in Makefile.
docker: `chmod go=rX` on generated files.
docker: Tolerate existing tmp in api-image.
docker: Install staged source on Docker images.
api: Rename Log.info to .properties.
api: Make a Log from other model changes.
api: Give all Job fixtures script_versions.
Merge branch 'master' into 2375-log-table
api: Test that we don't make logs about logs.
api: Let admin users delete logs.
api: Have users own the logs they generate.
api: Raise errors from saving logs.
api: Don't log common changes to API tokens.
api: Improve test for selective API auth logging.
api: More robust tests for log properties.
api: Don't include API tokens in logs.
Merge branch '2375-log-table' into master
workbench: Add pipeline instance tag render test.
api: Shorten name of authorized_keys index.
api: Test VM login scopes.
api: Introduce path-based API token scopes.
api: Migrate VM auth scopes to new system.
api: Support scope searching in API token index.
api: Support filters in API client auths index.
Merge branch '1904-object-scopes'
api: Shorten name of authorized_keys index.
api: Test VM login scopes.
api: Introduce path-based API token scopes.
api: Migrate VM auth scopes to new system.
api: Support scope searching in API token index.
api: Support filters in API client auths index.
api: Make JSON the default response format.
api: More consistency in test helper methods.
api: Introduce reader_tokens for extra access.
api: Accept JSON-formatted reader token array.
Merge branch '1904-api-reader-tokens'
workbench: Remove trailing whitespace.
workbench: Add Collections controller tests.
workbench: Add initial reader tokens support.
workbench: Reader tokens show collection files.
Merge branch '1904-workbench-reader-tokens'
docker: Build base container for Crunch jobs.
crunch-job: Support runtime Docker image.
crunch: Install Docker images on demand.
Merge branch '2492-docker-crunch-jobs'
2492: Crunch respects the maximum length of Docker image hashes.
2492: Properly escape Docker image name in Crunch.
Bugfix: Don't crash trying to render a new log buffer.
2739: Ensure crunch-job spawns all new tasks.
Merge branch '2739-fix-100-task-limit-wip'
2759: Workbench shouldn't try to make links from Link names.
2803: Remove unused setup.py.src.
2803: Split arvados and arvados_fuse Python modules.
2803: FUSE driver is now under the AGPL.
2803: Relax Python library requirements.
Merge branch '2803-python-package-split-wip'
2803: Further relax Python SDK requirement for httplib2.
2803: Make all Python required versions match wheezy.
2753: Remove older, unused Workbench provenance code.
2753: Add event_at data to Log fixtures.
2753: Move Workbench object list rendering to application_helper.
2753: Show immediate relations on Workbench Collection page.
Merge branch '2753-collection-metadata'
2753: Make Collection filenames with existing helper.
2753: Add tests for CollectionsHelper.file_path.
2753: Add Workbench Collection#files_tree.
2753: Workbench renders "0 bytes" for empty file sizes.
2753: Workbench shows Collection files in a tree.
2753: Replace Workbench file tree backgrounds with hover effect.
2753: Refactor Collection view to use library CSS.
2753: Move file tree CSS to collections.css.
2753: Add space between Workspace file tree buttons and scrollbar.
Merge branch '2753-collection-tree-view'
1904: Make Keep file EPERM test stricter.
2753: Properly close entire Collection file tree in Workbench.
2764: Reset browser session between Workbench integration tests.
2765: Tear out general Workbench reader_tokens support.
2764: Introduce show_file_links route.
2764: Add wget-friendly Collections file page.
2764: UI overhaul to the Collection download page.
Merge branch '2764-wget-collections'
2863: Remove debugging print from Python KeepClient.
2753: Fix Collection inline file view.
2752: Clean up arv-put imports.
2752: Refactor Python SDK test_collections.
2752: Clean up after Python SDK Collections tests.
2752: Add tests for CollectionWriter.write_directory_tree().
2752: Remove trailing whitespace in arv-put.
2752: Add method CollectionWriter.write_file().
2752: Reuse CollectionWriter.write_file() where possible.
2752: Implement CollectionWriter with a work queue.
2752: Add arvados.collections.ResumableCollectionWriter.
2752: Move Python tests to dedicated subdirectory.
2752: Move arv-put functionality to arvados.commands.put.
2752: Extract common Python test methods.
2752: Add basic test for arv-put.
2752: Generate cache filenames from arv-put arguments.
2752: Add ResumeCache to arv-put.
2752: Add ResumableCollectionWriter serialization to arv-put.
2752: Factor out arv-put byte tally, and test.
2752: Separate and test progress reporting functions.
2752: Refactor progress reporting in arv-put's CollectionWriter.
2752: Add ResumeCache.restart().
2752: arv-put resumes interrupted downloads from cache.
2752: Add KeepLocator class to Python SDK.
2752: arv-put shouldn't resume from expired Keep locators.
2752: arv-put saves and restores write progress information.
2752: Improve arv-put initialization from cache.
2752: Avoid hiccup on arv-put with short stdin.
2752: Quiet arv-put stdin integration test.
2752: Fix arv-put's write percentage output.
2752: arv-put ResumeCache constructor only accepts path argument.
2752: arv-put displays a notice when it resumes.
2752: arv-put displays progress quickly after resuming.
2752: Don't duplicate arv-put work after resume.
2752: arv-put only caches state on failure.
2752: Remove unused CollectionWriter checkpoint hook.
2752: Resumed collection writer doesn't do_queued_work immediately.
2752: arv-put handles exit signals.
2752: arv-put checkpoints after a PUT to Keep.
2752: Avoid ResumeCache name collisions from concatenating paths.
2752: arv-put always displays 100% progress at the end.
2752: arv-put writes newline after 100% human progress report.
2752: arv-put explains resumed uploads in more detail.
2752: arv-put doesn't update the resume cache when aborted.
Merge branch '2752-arv-put-resume'
2752: Include arvados.commands submodule in built packages.
2752/2755: Move test_cmdline code to test_arv-put.
2752: Rename test_arv_put.py.
2752: arv-put works when it can't write a cache file.
2755: Adjust arv-put test invocation.
2752: Show arv-put output on stdin test failure.
2893: API server accepts all serialized parameters as JSON.
2953: crunch-job strips permission hints from output manifests.
2933: Update Docker to use new Python package build process.
2933: Add Keep signing key configuration to Docker.
2882: Workbench generic create controller adds JSON href attribute.
2880: API server saves node statistics from pings.
Merge branch '2880-compute-ping-stats'
2880: crunch-dispatch must stop putting symbols in the database.
2880: Improve error reporting when crunch-dispatch updates node status.
2880: Don't dispatch Jobs until runtime constraints are met.
Merge branch '2880-crunch-dispatch-node-constraints'
2879: Factor out arv-put's cache creation to a utility library.
2879: Split arv-put's ArgumentParser into module segments.
2879: Move Python is_hex function to arvados.util.
2879: arv-put main() returns what it prints.
2879: Add arv-keepdocker command.
2879: Docker Keep installer returns 0 if image already installed.
2879: Clarify Docker image search behavior in Keep installer.
2879: arv-keepdocker reports a better error for ambiguous hashes.
Merge branch '2879-docker-image-installer'
2879: Store Docker image repo+tag together in one tag.
2879: Fix SLURM node separator when installing Docker images.
2879: Let ArvadosModel.readable_by callers specify table name.
2879: API server maps Job Docker image constraints to Collections.
2879: crunch-job uses Job Docker information provided by API server.
Merge branch '2879-job-api-docker-images'
crunch-job passes ARVADOS envvars to Docker.
2879: API server can find_or_create Jobs based on filters.
2879: API server can find_or_create Jobs based on Docker image.
2879: Use Job-specific filters in #index too.
2879: Support tag search in "in docker" Job filters.
2879: Document new Job API filters.
2879: Job Docker filters respect reader tokens.
2879: Docker filters on Jobs accept an array of search terms.
Merge branch '2879-docker-image-job-reuse'
2937: Make sure Collection share links end with /.
Refactor show_file_links.html.erb to reduce duplication.
Make Job runtime constraints documentation up-to-date.
3031: Improve "Docker image not found" error message.
2891: Workbench integration tests cope when screenshots aren't supported.
2891: Workbench handles expired tokens more consistently.
2891: Workbench error page shows login status.
2891: Refactor Workbench API token filters.
2891: Rename and comment Workbench API token filters.
Merge branch '2891-workbench-errors-retain-login'
Fix bad interaction between llfuse and daemonization in arv-mount.
Escape HTML entities in Workbench Job log viewer.
2891: Fix bug in Workbench error page test.
2891: API server ApplicationController style updates.
2891: API server assigns error tokens.
2891: Workbench API client raises structured exceptions.
2891: Workbench returns 404 when API object doesn't exist.
2891: Add index link to Workbench 404 responses.
2891: Add Workbench test for expired API token.
2891: Workbench displays more info about API errors.
Clean redundant conditional from Workbench layout.
2891: Guard against API server errors in Workbench layout.
2891: Workbench converts API HTTP errors to API exceptions.
2891: Workbench current_user copes when API server is unreachable.
2891: Workbench renders login exceptions earlier.
2891: Workbench reliably reloads model columns after failure.
2891: Trying to show a non-UUID in Workbench renders 404.
2891: Workbench details the requested item on its 404 page.
Merge branch '2891-improve-workbench-errors'
3204: Remove redundant pull-right from Workbench Collections search.
3078: arv-put always writes progress before starting work.
3078: arv-put supports dynamic stderr.
3078: Add arv-put options to add the upload to a Project.
3176: Fix arv-put's resume cache writing.
Merge branch '3078-arv-put-name-links'
3209: arv-run-pipeline-instance sends instance body as JSON.
Start API pipeline integration test.
arv-run-pipeline-instance doesn't submit nil Job parameters.
3201: Only Projects can be moved to "My Projects."
3201: Simplify logic to decide "My Projects" is selectable.
Give API server arv-run-pipeline-instance bugfix.
3113: API server assigns task qsequence from a PostgreSQL sequence.
Merge branch '3113-qsequence-serial'
3113: Make job task test more robust.
2044: Non-admins get all users' basic info from index API.
2044: API filters can search on boolean columns.
2044: Improve API test fixtures around projects.
Merge branch '2044-api-users-index-wip'
3195: Improve error handling in Job creation API.
Improve API server's git rev-list calls.
Merge branch '3195-bugfix-job-min-script-ver'
3195: Add job reuse test.
3252: Make Python tests runnable from setup.py.
3252: Add Python package documentation.
Update Python package test procedure.
Merge branch '3252-py-sdk-install-docs-wip'
3252: Fix README location in setup.py scripts.
3252: Include README in Python source manifest.
3303: Add logfile support to arv-mount.
2044: All users can give all other users folder permissions.
2044: API server lets group managers see group permission links.
2044: Workbench supports API select parameter.
2044: Add more helpers to get FA icons for objects.
2044: Workbench can query the API /permissions method.
2044: Workbench route to share a project with others.
2044: Add Workbench interface to manage project sharing.
Merge branch '2044-workbench-project-sharing'
2044: Workbench resource lists support Enumerable-style select.
2044: Make project sharing tests more forgiving of Ajax timing.
2044: Improve reporting when project sharing fails.
3200: Workbench copes better when a tab fails to load.
3200: X-Editable renders errors even without a message array.
3200: Improve error handling in Workbench's API client.
Merge branch '3200-ajax-error-handling'
3321: Workbench copes when pipeline components have an odd form.
Merge branch 'staging'
3321: Add test for rendering unusual pipeline templates.
3415: Python SDK can pass extra arguments to the client constructor.
3415: API exceptions from Python SDK include more error information.
Merge branch '3415-py-sdk-api-errors-wip'
3427: Restore arv-put's resume functionality.
3388: Workbench only lists user groups when sharing a project with groups.
3436: Don't show Projects pulldown on project sharing popups.
Re-disable preview pane in project sharing popup.
3482: crunch-job logs locator of job output sans permission hints.
3438: Show input title when choosing Collections for pipeline instances.
3527: Find Docker images natively inside Collections.
3527: Support using a Docker image from any stream in a Collection.
Merge branch '3527-infer-docker-hash-wip'
3412: Clean API Collections controller trailing whitespace.
3412: Refactor common manifest munging in API Collections controller.
3412: Make non-database fields selectable from API server.
3412: Make manifest_text selectable from Collections index API.
Merge branch '3412-full-collections-index'
3457: Revamp Python SDK and tool logging.
3457: Make arv-get display interface messages directly.
Merge branch '3457-pysdk-logging-wip'
3405: Log errors about launching pipeline instances.
3405: Mark pipeline instances failed when they're aborted.
Merge branch '3405-pipeline-error-logging-wip'
3421: Make Workbench project contents tables variable width.
Merge branch '3421-wide-project-tables-wip'
3593: Another round of hardening Workbench's render_exception.
Merge branch '3593-workbench-exception-rendering'
3593: Add explanatory comment about Workbench's user cache.
3499: Improve style consistency throughout docs.
3499: Update pipeline tutorial for recent UI changes.
3499: Update Keep tutorial for recent UI changes.
Merge branch '3499-home-project-docs-wip'
2800: Introduce config.flag_is_true() to Python SDK.
2800: Improve spec conformance of Python SDK KeepLocator.
2800: Remove global state from KeepClient.
2800: Migrate from Keep to KeepClient in the Python SDK.
2800: Use dedicated logger in Python SDK api module.
2800: Make run_test_server put Keep tempfiles in tests/tmp/.
2800: Introduce TestCaseWithServers to Python SDK.
2800: Use local KeepClients in Python SDK tests.
Merge branch '2800-pysdk-no-global-keep-client-wip'
2800: Delay API client creation in Python CollectionReader.
2800: run_test_server only uses tests/tmp for real Python tests.
Fix run_test_server Keep blob filename handling.
3147: Move fake Python HTTP response generation to arvados_testutil.
3147: Add RetryLoop to the Python SDK.
3147: Add check_http_response_success to Python SDK.
3147: Remove old Keep signing support from Python SDK.
3147: Add retry support to Python SDK's KeepClient.
3147: Fix unwanted integer division in KeepClient logs.
Merge branch '3147-pysdk-retries'
Merge branch '3704-broken-filters-param'
3704: Treat project filters consistently in Workbench chooser.
3720: Clean up+skip Ruby SDK tests.
3720: arvados-cli Gemfile refers to gemspec.
3720: Refactor manifest parsing from API server to Ruby SDK.
3720: Limit Workbench file rendering for large Collections.
Install & test Ruby SDK, then use that version for other tests.
Merge branch '3720-big-manifest-performance-wip'
3720: Update arvados Gem versions.
#3830: Make crunch-job's git calls more robust.
3842: Keep::Manifest concatenates file information from manifest.
3846: Improve timeout handling in PySDK KeepClient.
Prevent PySDK CollectionReader from sending UUIDs to Keep.
Clean up manifest whitespace in split-fastq.
3147: PySDK tests use mock>=1.0 and easier mock side_effect.
3147: Move PySDK HTTP test infrastructure to testutil.
3147: PySDK StreamReader instantiates a KeepClient late.
3147: Fix variable name typo.
3147: Add retry_method to the Python SDK.
3147: Make PySDK KeepClient.get and put retry_methods.
3147: Add retry support to PySDK StreamReader classes.
3147: Add retry support to PySDK Collection objects.
3147: Add retry support to Python CLI tools.
3147: Add retry support to PySDK list_all utility.
3147: FUSE driver requires a Python SDK with retry support.
3147: Add retry support to FUSE driver.
3147: crunch-job retries data uploads more for longer-running jobs.
Merge branch '3147-py-collection-retries-wip2'
3147: Rebuild the Python SDK to satisfy FUSE version requirement.
3147: FUSE requires a built Python SDK.
run-tests.sh uses virtualenv's activate script.
2881: API server associates nodes with their assigned jobs.
2881: crunch-dispatch updates nodes' job assignments.
Merge branch '2881-node-has-job-wip'
3848: Python packages build with git tag by default.
3848: Package version numbers use UTC timestamps.
3848: Stop manually tagging Python builds.
Merge branch '3848-egg-default-git-tag-wip'
3866: PySDK KeepClient accommodates services with IPv6 addresses.
3848: build-packages respects egg-info versions.
Merge branch '3848-build-packages-pyversion'
arv-put parses arguments before instantiating an API client.
3634: Update user setup tests for preserved tab state.
4012: crunch-job retries all API operations.
Merge branch '4012-crunch-job-api-retries-wip'
2881: Add Node Manager service.
Merge branch '2881-node-manager'
Update install docs for keep→keepstore rename.
2881: run-tests tests the Node Manager.
4139: Add environment configuration knobs for Node Manager tests.
4139: Add *.egg to Node Manager's .gitignore.
4139: Add README to Node Manager.
2881: Skip Node Manager tests for now.
2881: Re-enable Node Manager tests.
4139: Speed up Node Manager tests.
4139: Node Manager README links to compute node lifecycle page.
Merge branch '4139-blocking-node-manager-tests-wip'
4126: a-r-p-i dereferences parameter values when creating jobs.
4126: API server uses fixed a-r-p-i (again).
4196: arv-keepdocker uses Docker 1.0 pull syntax.
4126: API server uses fixed a-r-p-i.
Python SDK is not compatible with apiclient 1.3.
4189: Document the API server's writable_by field.
4189: Workbench tests clear more state during teardown.
4189: Workbench lets users with write permission rename objects in projects.
Merge branch '4189-workbench-project-admin-attr-editing-wip'
4189: Diagnostics suite doesn't try to clear Rails cache.
4239: Update Python SDK for google-api-python-client 1.3.
4239: Build more dependencies of new google-api-python-client.
4139: Node Manager callback actor avoids redundant delivery calls.
4139: Improve logging in Node Manager poll actors.
4139: Improve scheduling of Node Manager polls.
Merge branch '4139-node-manager-polling-logging-wip'
4139: Node Manager supports subscribing to shutdown events.
4139: Node Manager more closely tracks nodes it boots.
Merge branch '4139-node-manager-race-fix-wip'
4139: Fix race condition in Node Manager tests.
4137: Fix bad state reference in Node Manager daemon.
4139: Remove cloud node setup code from API server.
4139: Node create API requires a node parameter.
Merge branch '4139-clean-api-node-cruft-wip'
Fix off-by-one error in Node Manager server calculations.
Fix race condition in Node Manager shutdown tests.
Fix Node Manager test bug introduced by last commit.
4307: Workbench: only pipeline components are not editable after start.
4388: Ruby Keep::Manifest skips blank lines in manifests.
4412: arv edit consistently uses :body_object in API requests.
4412: Remaining Ruby clients use :body_object for API requests.
Fix `arv keep copy` → `arv copy` wrapper.
4303: crunch-job logs when it retries API calls.
Merge branch '4303-crunch-retry-logging-wip'
4357: Node Manager boots new nodes when up nodes are busy.
Merge branch '4357-node-manager-busy-nodes-wip'
4408: Workbench Collection sharing allows file downloads again.
Merge branch '4408-collection-sharing-login-fix-wip'
PySDK CollectionReader can get empty collection.
3603: Fix context methods for PySDK Collection objects.
3603: Clean up PySDK imports.
3603: PySDK Collection objects support file-like APIs.
Merge branch '3603-pysdk-file-api-wip'
Refs #3857: Build modern requests package for PySDK.
Bugfix bad exception name in PySDK util.is_hex.
4380: Reorganize arvnodeman.computenode.
Merge branch '4380-node-manager-computenode-reorg-wip'
4027: Add arvados_sdk_version runtime constraint to API server.
Merge branch '4027-api-sdk-version-wip'
4027: Build packages for Ruby and Perl SDKs.
Merge branch '4027-sdk-debian-packages-wip'
4027: API server considers arvados_sdk_version for Job reuse.
4027: Test updating job's SDK version after it's already set.
Merge branch '4027-sdk-constraint-reuse-wip'
4380: Node Manager monitors respond to shutdown_eligible message.
4380: Node Manager shutdown actor is more robust.
4380: Fix retry scheduling for Node Manager change methods.
4380: Add SLURM dispatcher to Node Manager.
Merge branch '4380-node-manager-slurm-drain-wip'
4027: API Jobs that specify SDK version must also use Docker.
Merge branch '4027-api-sdk-requires-docker-wip'
4501: Restore FUSE FuseTagsUpdateTest.
4501: FUSE exposes by_id subdirectory when run with --by-id.
Merge branch '4501-fuse-by-id-subdirectory-wip'
4591: Avoid capturing critical exceptions in Websockets server.
4591: Websockets server fetches fewer logs at a time.
Merge branch '4591-websockets-raise-oom-wip'
4705: Add CollectionReader.api_response() to PySDK.
4705: Refactor mount creation in FUSE tests.
4705: Fix FUSE exception logging.
4705: FUSE driver can read Collections that only exist in Keep.
Merge branch '4705-fuse-collections-from-keep-wip'
4291: Clean up HTTP methods in Workbench URL generators.
4291: Workbench uses .prop() to set element disabled property.
4291: Add Workbench controller tests for collection sharing.
4676: Collection sharing popup is always JavaScript.
4291: Workbench Collection sharing buttons are actual buttons.
4291: Workbench says explicitly when collection sharing is disabled.
Merge branch '4291-workbench-collection-sharing-button-wip'
4027: Revamp SSH use in our Docker images.
4027: Bugfix update-gitolite.rb in Docker.
4027: arvados/jobs includes virtualenv.
4027: Crunch installs jobs' requested arvados_sdk_version.
4027: crunch-job logs its own version information.
Merge branch '4027-crunch-sdk-install-wip'
4027: Update arvados-cli in API server bundle.
4380: Node Manager SLURM dispatcher proceeds from more states.
4293: Node Manager shuts down nodes that fail to boot.
Merge branch '4293-node-manager-timed-bootstrap-wip'
4027: Document arvados_sdk_version's virtualenv requirement.
4792: arv-ls normalizes the collection before listing.
Merge branch '4792-arv-ls-normalize-wip'
4481: Update tutorial Crunch scripts to use newer PySDK methods.
4481: Refresh Crunch script tutorial page.
Merge branch '4481-update-user-docs-TC'
4818: Node Manager unpairs Arvados node when cloud node shuts down.
4818: Add missing timeout in Node Manager test.
4800: run-command calls sys.exit() with an integer.
4670: Node Manager handles more libcloud exceptions.
4670: Add a post-create hook to Node Manager for EC2 tagging.
Merge branch '4670-node-manager-robust-tags-wip'
4844: Node Manager doesn't treat min_nodes as min_nodes_idle.
4836: Trigger Workbench infinite scroll load on tab show.
Merge branch '4836-first-tab-load-wip'
Python packages use python-daemon<2.
3835: Improve error reporting in PySDK Keep client.
3835: PySDK raises NotFoundError when all Keep services report such.
Merge branch '3835-pysdk-keep-exceptions-wip'
4840: Restore checkbox to project Pipelines and Jobs tab.
4840: All Workbench selection links toggle dropdown.
4840: Workbench selection actions are submitted by POST.
Merge branch '4840-workbench-selections-action-forms-wip'
5029: Fix crunch-job logging when SDK requested+virtualenv unavailable.
5029: API server uses bugfixed crunch-job.
4967: crunch-job unmounts old Keep mounts more robustly.
Merge branch '4967-crunch-mount-cleanup-wip'
4967: API server bundle uses bugfixed crunch-job.
4967: Fix API server Gemfile.
5087: crunch-job installer feeds tar incrementally.
Merge branch '5087-fix-crunch-sigpipe-wip'
4878: Refactor Workbench "Re-run job" button for more code reuse.
Merge branch '4878-rerun-job-bcs-wip'
5074: Improve discovery document caching in Ruby SDK.
5074: CLI SDK uses Google API client extensions from Ruby SDK.
Merge branch '5074-ruby-sdk-discovery-cache-wip'
4562: Add Crunch tools overview to the documentation.
Merge branch '4562-crunch-tools-docs-wip'
5037: Add test for new PySDK API cache behavior.
5037: Fix race condition in PySDK Websockets test.
5037: Remove unnecessary cache=False from arv-copy.
5037: Fix tearDown error handling in PySDK Websockets test.
5087: API server uses a crunch-job with bugfix.
5131: run-command consumes escape backslashes.
Merge branch '5131-run-command-consume-backslashes-wip'
5131: Document run-command's escape sequences.
5135: Workbench consistently orders modal buttons.
5141: Workbench permits float values for number pipeline inputs.
Merge branch '5141-workbench-numeric-inputs-wip'
3021: Remove obsolete accommodation for buggy python-daemon.
Relax version limit of python-daemon dependencies.
5175: Do not add +sha1 tag to Python packages ever.
5175: Include README for Node Manager packages.
5175: Avoid egg_info name conflicts with pip.
5176: Add more Ruby SDK tests for empty manifests.
4464: Remove API group contents include_linked parameter.
4464: Document API group contents parameters.
4464: API group contents no longer include manifests.
4464: Simplify API collection list method docs.
Merge branch '4464-api-project-contents-wip'
Revert 11339c91. Don't use virtualenv to build packages.
5176: Update API Collections test to accommodate new fixtures.
Merge branch '5176-escape-filenames'
5176: Update Gemfiles with current arvados gem.
5160: All users have API read permission to anonymous group.
5160: Fix typo in Workbench default config comments.
5160: Make anonymous group fixture match database seed.
5160: Workbench hides anonymous sharing when browsing disabled.
Merge branch '5160-share-with-anonymous-group-wip'
Node Manager daemon tests avoid checking monitor.is_alive().
4138: Simplify Node Manager GCE credential handling.
4138: Refactor common Node Manager driver initialization to base driver.
4138: Fix noop Node Manager EC2 driver tests.
4138: Refactor out Node Manager DriverTestMixin.
4138: Revamp Node Manager driver proxying in BaseComputeNodeDriver.
4138: Prepare Node Manager GCE driver for production.
Merge branch '4138-node-manager-gce-wip'
4759: Update Node Manager to parse new Arvados API timestamps.
5277: Build .deb for new ciso8601 dependency.
5283: Improve reliability of crunch-job output collation.
5283: crunch-job doesn't use freeze logic after a job fails.
5283: Log more crunch-job output handling.
Merge branch '5283-crunch-collation-safety-wip'
5313: Rename Node Manager's `user-data` GCE tag to `arv-ping-url`.
5313: Node Manager's GCE driver destroys boot disks reliably.
Merge branch '5313-node-manager-gce-fixes-wip'
4751: Node Manager considers ping times for stricter node pairing.
Merge branch '4751-node-manager-stricter-node-pairing-wip'
5313: Revert Node Manager's GCE boot disk destroy code.
5313: Rely more on datacenter constructor in Node Manager GCE driver.
Merge branch '5313-node-manager-gce-fixups2-wip'
5319: Fix incorrect portable_data_hash in test fixture.
5319: Add API migration for manifests with bad portable data hashes.
Merge branch '5319-fix-hinted-collection-hashes-wip'
5319: Include CurrentApiClient in migration model.
5313: Build a .deb for the PyCrypto Python package.
Add missing re import to PySDK arvfile module.
5104: Refactor Ruby SDK test infrastructure into sharable module.
5104: Add Collection class to the Ruby SDK.
Merge branch '5104-ruby-sdk-collections-wip'
5034: Add Keep::Manifest#files_size to Ruby SDK.
5034: Improve Workbench collection display.
Merge branch '5034-workbench-collection-summary-wip'
5034: Update Workbench Gemfile post-merge.
5034: Update Workbench integration tests.
5319: Improve performance of Collection PDH fix migration.
Merge branch '5319-collection-pdh-fix-performance-wip'
5313: Node Manager has cloud-specific logic to get node FQDNs.
Merge branch '5313-node-manager-node-naming-tag-wip'
5319: Improve collection PDH fix performance with LIKE searches.
Merge branch '5319-collection-pdh-fix-performance-2-wip'
5319: Bound search in collection PDH fix migration.
4253: Add a username attribute to users.
4253: Clean up some user setup methods.
4253: Use new username to set up repository and VM logins.
Merge branch '4253-user-usernames-wip'
5502: Adjust id and name of Node Manager cloud object mocks.
5502: Node Manager attaches a local SSD to GCE compute nodes.
Merge branch '5502-node-manager-gce-ssd-wip'
4253: Users can manage their own repositories.
4253: Sync up Docker with our production Gitolite setup.
4253: Let Workbench Manage Account partials render their whole pane.
4253: Add Workbench interface to create repositories.
Merge branch '4253-user-repos-wip'
4253: Add script to migrate existing Gitolite installations.
Merge branch '4253-gitolite-migration-wip'
5627: Python file-like objects use SEET_SET as the default whence.
4253: Gitolite migration makes a name symlink for arvados repository.
Merge branch '5416-repo-dir-is-uuid'
5518: PySDK arvados.util must import arvados.
Fix debug log formatting in PySDK.
Add missing global declaration in arv-copy.
Tighten up DNS check in arvdock.
5352: crunch-dispatch treats node allocation failure as temporary.
Merge branch '5352-crunch-dispatch-salloc-tempfail-wip'
5653: arv-copy copies multiple commits from the same repository+pipeline.
Merge branch '5653-arv-copy-multiple-branches-wip'
Include arvados/collection in the Ruby SDK gem.
5642: Explicitly make all swap available under Docker in crunch-job.
5642: crunch-job uses higher memory limit when running few tasks.
Merge branch '5642-crunch-job-swap-limits-wip'
5614: Add Collection#exist? to Ruby SDK.
5614: Add Collection#each_file to Ruby SDK.
5614: Ruby SDK cp_r method creates streams as needed.
5614: Use assert_includes more in Workbench tests for better diagnostics.
5614: Workbench renders error flash separately from others.
5614: Improve Workbench combine collections performance.
Merge branch '5614-workbench-optimize-combine-collections-wip'
5614: Update Workbench Gemfile for new Ruby SDK.
5714: Node Manager setup process retries Arvados errors.
5714: Avoid Node Manager race conditions around stop_if_no_cloud_node.
Merge branch '5714-gce-setup-bugfixes-wip'
5717: crunch-job uses fewer slots when few tasks at this level.
Merge branch '5717-crunch-dynamic-max-tasks-per-node-wip'
5742: User setup process uses username for VM login, not repo name.
run_test_server doesn't assume ARVADOS_API_TOKEN is set.
5554: Add Rake task to delete old job log records.
5554: Clean up erroneous fixture pipeline state.
5554: Workbench links finished job logs from pipeline instance log tab.
Merge branch '5554-delete-job-log-rows-wip'
5490: Refactor API server job_reuse_controller_test.
5490: Clean up API server previous job fixtures.
5490: Test API server handling of git hash-like branch names.
5490: Test API server finds Docker images with hash names.
5490: Require exact matches in API server's default job reuse filters.
Merge branch '5490-crunch-tighten-job-reuse-wip'
5834: Limit how much manifest text data API server will load for index.
Merge branch '5834-api-max-response-size-wip'
5752: arv-copy preserves properties of docker_image links.
Merge branch '5752-arv-copy-preserve-docker-timestamps-wip'
5842: Node Manager shuts down booted nodes if they can't do compute work.
5842: Node Manager only considers nodes busy if they're working.
Merge branch '5842-node-manager-tighten-successful-boot-criteria-wip'
3793: Add Docker image cleaner service for compute nodes.
Merge branch '3793-clean-docker-images-wip'
5736: Fix typo in Node Manager test comments.
5736: Node Manager cleans node records after shutting down a paired node.
Merge branch '5736-node-manager-easy-slot-cleanup-wip'
3793: Add services/dockercleaner to run-tests.sh.
5915: Workbench tries the anonymous reader token first for collection wget listing.
3793: Build Docker cleaner service in run-build-packages.sh.
Merge branch '3793-docker-cleaner-wip'
Improve packaging of third-party Python 3 packages.
3793: Fix how python3 fpm arguments are added.
3793: Add services/dockercleaner to run-tests.sh.
3793: Build Docker cleaner service in run-build-packages.sh.
Improve packaging of third-party Python 3 packages.
3793: Fix how python3 fpm arguments are added.
Update tutorial pipeline template definition.
Update tutorial pipeline page to match new definition.
6095: arv-copy preserves order of copied JSON.
Merge branch '6095-arv-copy-preserve-object-order-wip'
Fix directory when dropping API server database connections.
5790: Fix PySDK Docker image listing comparing ints and datetimes.
6152: Use Python SDK for compute node installation.
Merge branch '6152-compute-node-no-arv-wip'
5790: Improve Docker image listing in Python SDK.
Merge branch '5790-copy-most-recent-docker-image-wip'
6149: crunch-job installer handles egg_info errors better.
Merge branch '6149-quiet-egg-info-stderr-wip'
6320: Add tests for API virtual machines login method.
6320: API virtual machines login method include groups information.
Merge branch '6320-api-logins-include-groups-wip'
4410: crunch-dispatch logs crunch-job exit later.
4410: Crunch retries jobs when all SLURM nodes fail.
Merge branch '4410-slurm-fails-are-tempfails-wip'
Remove unused modules from PySDK setup.py.
6320: Update Workbench test for active user VM setup.
6388: Remove Debianisms from run-build-packages.
6388: Add distro targets to run-build-packages.
6388: run-build-packages records python2.7 dependency where needed.
6388: Remove version munging from Python package builds.
6388: Backport pycurl package everywhere.
6388: Fix Arvados Perl SDK package build.
6388: run-build-packages introspects more Python environment.
Merge branch '6388-centos-packages-wip'
6388: Remove old --apt-server switch from run-build-packages.
6388: Build PyCURL with NSS on CentOS 6.
6388: Build specific backport versions for Python OAuth dependencies.
6388: CentOS 6 needs Python daemon and llfuse backported.
6388: Improve portability of Python backports in run-build-packages.
Merge branch '6388-python-backports-wip'
6388: CentOS needs backports for python-daemon and its dependencies.
6432: Python SDK can find and use CA certs on Red Hat.
Merge branch '6432-pysdk-more-ca-certs-wip'
Fix bad CollectionReader reference in arv-put.
Invoke crunch-job with `/usr/bin/env perl`.
Bugfix config:check error detection in Rails upgrade scripts.
6388: Rails package upgrade scripts support Red Hat systems.
6388: Extend API server upgrade script to do initial database setup.
Merge branch '6388-package-scripts-redhat-support-wip'
6388: Add CentOS 6.6 instructions to install docs.
6438: Modernize Rails server installation guides.
6455: Include more steps in Crunch install docs.
Merge branch '6438-multiplatform-install-guide-wip'
5989: API repo perms method always returns all repos.
Merge branch '5989-api-all-repos-permissions-wip'
6389: Dockerfiles include the CMD to build packages.
6389: Add Makefile to support Docker builds.
6388: Add Dockerfile to build packages on CentOS 6.
Merge branch '6389-add-dockerfiles'
6358: Declare FUSE driver's dependency on llfuse >= 0.40.
API server installation doc clones arvados over the network.
6656: python-arvados-python-client Recommends: git.
6658: Fix exception name errors in PySDK arvados.util.
6661: Install docs refer to arvados-git-httpd package and binary.
6497: Separate package build and uploader scripts.
Merge branch '6497-separate-package-upload-wip'
Add standard Python .gitignore for CWL SDK.
6750: arvados-api-server-upgrade only installs omniauth.rb if it exists.
6750: API install guide doesn't tell users to copy omniauth.rb.example.
6497: Bugfix distro names for DebianPackageSuites.
6682: Add script to orchestrate builds for multiple targets.
Merge branch '6682-use-docker-build-cmd-wip'
6691: Build and ship packages for Ubuntu 14.04.
Merge branch '6691-trusty-packages-wip'
Rebuild Python packages before building their distro packages.
6778: Remove git install from PostgreSQL install instructions.
6691: Add Ubuntu trusty packages to our install guide.
6691: Add reference to Ubuntu trusty in Ruby install instructions.
Fix duplicated id in SSO install documentation.
RVM setup instructions tell you to install Bundler after activation.
Fix stale "Install Ruby and Bundler" links.
arv-copy git check doesn't catch system exceptions.
Fix type checks in arv-copy recursive collection copying.
6590: Aesthetic improvements to the API install guide.
Merge branch '6590-api-config-docs'
6847: Handle Docker DNS in Docker daemon configuration.
Merge branch '6847-docker-dns-wip'
Merge branch '6221-write-trash-list'
6674: Install docs use Phusion's install instructions exclusively.
6591: Improve install docs around $external_client variable.
6591: Suggest proxy timeouts in install doc Nginx configs.
6676: Install docs recommend running SSO server behind Nginx.
6591: Add ssl_certificate lines to sample install nginx configs.
Merge branch '6591-6674-6676-nginx-docs-wip'
6157: Copyedits to compute node name docs.
Add `bundle exec` needed to run script in keepproxy install guide.
Fix `websocket_address` typo throughout API install guide.
Quote `git_repo_ssh_base` value in git install guide.
Add perl-Crypt-SSLeay as a Perl SDK dependency on Red Hat.
Install guide notes that snakeoil certs break browser uploads.
crunch-dispatch preserves LD_LIBRARY_PATH environment variable.
Haha, no seriously, don't deploy Workbench with snakeoil certs.
API server install guide suggests a more likely websocket_address.
Remove redundant console line from Workbench install guide.
Install Git in shell server install guide.
Highlight dynamic fields in Keep service registration instructions.
Add client_max_body_size to Nginx install configurations.
Add Gitolite UMASK setting to install guide.
Make rvm-exec version in crunch-dispatch run script less specific.
Add docs to Node Manager's base compute node driver.
Include example service script for keepproxy.
Improve crunch-dispatch runit script in install guide.
6938: Install guide covers git server before standard objects.
6950: Improve "make a job" check in Crunch dispatch install guide.
Merge branch '6950-install-guide-job-test-wip'
6998: Set force_ssl: false in Workbench Docker image.
6095: Stop demoting OrderedDicts to dicts in arv-copy.
6095: Brown paper bag fix the last commit.
6940: Use nginx as WWW_OWNER on Red Hat distributions.
arv-tag authenticates to the API server with OAuth2.
arv-tag respects more valid ARVADOS_API_HOST_INSECURE values.
6940: Move arvados-workbench-upgrade.sh down in install guide.
6940: Workbench install guide describes using Nginx with Software Collections.
Merge branch '6940-workbench-centos-install-wip'
6638: Python backports declare all their C dependencies.
6638: Remove C dependencies from our Python SDK.
Merge branch '6638-python-backport-dependencies-wip'
Fix switch typo in arv-mount tutorial docs.
Move arvados-api-server-upgrade.sh down in install guide.
7170: Fix Ruby configure switch typo.
7225: ArvadosFile.flush() commits all underlying blocks.
Merge branch 'pr/1'
Close unclosed quotes in arvados-api-server-upgrade.sh.
Merge branch 'pr/24'
API server limits KeepService changes to admins.
7321: arvados/jobs Dockerfile installs development headers.
7321: arvados/jobs Dockerfile installs python-pip.
7322: Don't backport virtualenv for Arvados.
6967: Remove trailing whitespace in Workbench's application.default.yml.
6967: Extend application.default.yml source_version detection.
Merge branch '6967-application-yml-without-git-wip'
7227: crunch-job only unmounts Keep mounts under $CRUNCH_TMP.
Merge branch '7227-crunch-job-stricter-unmount-wip'
Merge branch 'pr/29'
7263: crunch-job checks for refreshes every two seconds.
7207: Remove `arv keep check` and `arv keep less`.
7356: CentOS 6 needs a Python 3 backport for websocket-client.
7451: Improve virtualenv setup in run-tests.sh.
Merge branch '7451-dont-recreate-venv3dir-wip'
7435: Node Manager stops trying to shut down delisted cloud nodes.
7254: Test arv-put preserves replication when cache load fails.
Merge branch '7435-node-manager-shutdown-cleanup-wip'
Version the CWL runner's dependency on cwltool.
API server needs an arvados-cli with crunch-job --docker-bin.
7341: run-deploy.sh retries failed Puppet agent runs.
Merge branch '7341-deploy-retries-puppet-agent-wip'
Backport pbr < 1.0 on CentOS 6.
7499: Update development link in Workbench "Getting Started" popup.
7324: Tighten exception ignoring in run_test_server start_nginx.
Clean redundant except: blocks in run_test_server.
Merge branch 'pr/28'
Fix multiple exception catching in arv-run.
7587: PySDK retries socket.error exceptions from API requests.
Merge branch '7587-httplib2-retries-wip'
7587: Refactor PySDK API tests to use TestCaseWithServers.
7587: Add test for PySDK API client socket.error retries.
Merge branch '7587-pysdk-retry-test-wip'
7307: Clarify intended failure in arv-git-httpd SplitHostPort test.
Merge branch 'pr/25'
7695: Docs reflect that docker_image can't be a collection UUID.
7668: crunch-dispatch gets node stats from properties field.
Merge branch '7668-crunch-node-properties-wip'
6638/7370: Force new builds of Python backports with dependencies.
7713: Node Manager blackholes broken nodes that can't shut down.
Merge branch '7713-node-manager-blacklist-broken-nodes-wip'
Fix non-packaged API server paths in the install guide.
7591: Workaround bad permissions in the httplib2 backport.
3585: Add --project-uuid switch to a-r-p-i.
6356: crunch-job doesn't create new tasks after job success is set.
Merge branch '6356-crunch-permfail-task-retry-fix-wip'
6846: Workbench navigation bar login returns user to the same page.
Merge branch '6846-workbench-top-nav-login-returns-wip'
7645: Install guide suggests setting client_max_body_size consistently.
7356: Install guide sets client_max_body_size for arv-git-httpd.
Merge branch '7645-doc-client-max-body-size-wip'
7123: Crunch doesn't update job log when arv-put fails.
Merge branch '7123-crunch-no-record-log-failure-wip'
7696: Refactor locator builder method in PySDK tests.
7696: Clean imports in PySDK arvados.keep module.
7696: PySDK KeepClient uses all service types.
7696: PySDK determines max_replicas_per_service after querying services.
7696: Improve PySDK KeepClient.ThreadLimiter.
Merge branch '7696-pysdk-all-keep-service-types-wip'
7313: crunch-job reports an error when a task doesn't record state.
6846: Streamline Workbench 404 page.
6923: Improve Arvados SDK version logging in Crunch run script.
6923: crunch-job logs PySDK version when minimally bootstrapped.
Merge branch '6923-crunch-no-dpkg-query-wip'
Install guide crunch-dispatch runit script exports $HOME.
7870: Teach crunch-dispatch to fail jobs it already locked.
Merge branch '7870-crunch-dispatch-retry-fail-lock-wip'
7839: Stop backporting requests for Python 2.
7313: Fix sprintf formatter in crunch-job warning message.
7835: Node Manager coerces node size prices to float.
Remove redundant exception from Node Manager NETWORK_ERRORS.
7835: Node Manager converts node.size into a size object on GCE.
Merge branch '7835-node-manager-price-type-wip'
7829: Remove unnecessary MANIFEST.in files.
7829: Python bdists include licenses and documentation.
7829: Gems include licenses and documentation.
Merge branch '7829-package-licenses-wip'
7953: Fix Docker image to build CentOS 6.6 packages.
Merge branch '7953-centos-scl-fix-wip'
7829: Restore MANIFEST.in files; add licenses to them.
7829: Add MANIFEST.in for arvados-docker-cleaner.
7953: Fix Docker image to test CentOS 6.6 packages.
7829: Add licenses to Arvados software packages.
Merge branch '7829-package-licenses-wip'
7953: Update test procedure for CentOS 6 packages.
7960: Fix typo in arv-run option reference.
7961: SSO postinst script uses bash.
7901: Fix variable name typo in crunchstat-summary.
7927: Docker cleaner service uses Docker API v1.14.
Merge branch '7927-docker-cleaner-api-version-wip'
8077: Tutorial suggests using the arvados/jobs Docker image.
8082: Ensure consistent column widths in Workbench job banner.
8014: Add #configure anchors to API server and Workbench install guides.
7957: Update Software Collections instructions in install guide.
Merge branch '7957-new-software-collections-install-wip'
8095: Make install guide bundle commands more consistent.
Merge branch '8095-install-guide-rails-sudo-wip'
8104: Pin the pycurl backport version.
7991: Fix permissions of google-api-python-client egg info files.
Merge branch '7991-google-api-python-client-perms-wip'
7991: Update error message for pip build errors.
7991: pip build process detects if --no-use-wheel is supported.
8014: Remove unused sso-server-upgrade.sh script.
8014: Remove unused $VERSION tracking from arvados-sso postinst.
8014: Refactor "not fully configured" messages in arvados-sso postinst.
8014: Simplify COMMAND_PREFIX-setting code.
8014: Refactor web service detection warnings in arvados-sso postinst.
8014: arvados-sso postinst recognizes versioned Nginx service names.
8014: Improve conffile handling in arvados-sso postinst.
8014: Introduce run_and_report function to arvados-sso postinst.
8014: Clean database state detection in arvados-sso postinst.
Fix quoting of local variable assignments throughout.
8014, 8059: Unify Rails package building.
8014: Rails postinst script sets better default access for /etc/arvados.
8014: Rails upgrade scripts are removed; don't run them in deploy script.
Merge branch '8014-rails-postinst-scripts-wip'
8014: Install guide uses new Rails postinst scripts for configuration.
Merge branch '8014-rails-postinst-doc-update-wip'
8008: Quote more variables in package test scripts.
8008: Make dependencies for build-dockerfiles.
8005: DRY up PostgreSQL password auth instructions on CentOS 6.
8005: Document installing Git on CentOS 6 from RepoForge.
8005: Fix bad Textile markup in compute node install guide.
8005: Install guide gets SLURM and MUNGE from RPMs.
8005: Install guide uses runit packages on Red Hat.
8005: Add tar Ruby build dependency on CentOS 6.
Merge branch '8005-centos-3rdparty-installs-wip'
Fix install doc rendering of API Nginx config.
Refresh Gitolite install guide.
8014: Remove more upgrade script references from install guide.
8005: Install guide suggests slurm-munge on Red Hat SLURM nodes.
Add missing ; in keepproxy Nginx config.
SLURM install guide notes slurm.conf path on Red Hat.
Add curl library dependency to shell install guide.
Add fuse dependency to FUSE driver package.
Workbench loads CA certs on Red Hat.
login-sync gets user's home from /etc/passwd.
Improve install guide Nginx+SCL integration.
.deb dependencies from Python packages ignore iteration.
Inhibit udev startup in Debian test Docker images.
Make install guide slurm.conf more Arvados-compliant.
7868: Update API server's arvados-cli version.
8406: Update comment to match new code.
Merge branch '8406-tempfail-after-retry-unlocked'
crunch-job detects more "io aborted" SLURM errors.
Pin PySDK's gflags dependency to <3.0.
Build python-glags backports < version 3.0.
8203: crunch-job tempfails after failing to install a Docker image.
8371: Rails postrm only fires after removing last RPM.
Merge branch '8371-centos-postrm-bugfix-wip'
8413: Rails postinst doesn't chgrp root /etc/arvados.
Merge branch '8413-fix-rails-postinst-chgrp-wip'
8371, 8413: Bump Rails package iteration.
Docs: Fix typo in run-command foreach example.
8373: Expand crunch-job regexp to detect more Keep temporary failures.
8439: API server suggests RW+ Gitolite permissions for writers.
Merge branch '8439-gitolite-nonff-perms-wip'
8203/8690: crunch-job uses `srun --nodes=1`.
8709: Docker cleaner cleans container volumes.
Merge branch '8709-docker-cleaner-volumes-wip'
8752: Skip unreliable Python SDK Keep test.
7852: Bugfix _filepos tracking in PySDK readline method.
Merge branch '7852-readline-cache-fix-wip'
8002: Stop backporting PyVCF.
Merge branch '8002-no-pyvcf-wip'
8800: Document the new queue_position implementation.
Merge branch '8800-queue-query'
8782: Remove WIFEXITED check from crunch-job reapchildren.
Merge branch '8782-reapchildren-after-signal-wip'
8811: crunch-job srun_sync detects and reports SLURM tempfails.
Merge branch '8811-srun-sync-tempfail-wip'
8893: crunch-job doesn't pass empty strings to `docker run`.
Merge branch '8893-crunch-job-crunchrunner-quoting-wip'
8810: crunch-job reports errors when checking if Docker image is loaded.
Merge branch '8810-crunch-improve-docker-loading-wip'
8879: cwl-runner --submit respects --project-uuid.
8879: Clean indentation in CWL SDK tests.
Merge branch '8879-cwl-runner-job-owner-wip'
8872: Bugfix Node Manager's node search after node create failure.
Merge branch '8872-node-manager-create-search-handling-wip'
Merge branch '8904-support-python3.2'
Pin dockercleaner's docker-py requirement to 1.7.2.
8912: Node Manager search_for_now uses overridden methods.
Merge branch '8912-node-manager-patch-nodes-wip'
4083: crunchstat-summary imports _strptime.
8893: Safer quoting of crunch-job's conditional volume switches.
Merge branch '8893-crunch-job-volumes-array-wip'
8963: arv-keepdocker copies metadata links' properties.
Merge branch '8963-arv-copy-link-properties-wip'
9118: Fix arv-put crash when finishing without output.
Merge branch '9118-arv-put-nameerror-fix-wip'
9120: Add tests for BaseComputeNodeDriver's search_for methods.
9120: search_for_now falls back to real driver methods when needed.
Merge branch '9120-node-manager-search-ex-methods-wip'
9135: Add basic tests for Python events listeners.
9135: Clean imports in test_events.
9135: Make EventClient initialization more consistent.
9135: Bring EventClient's public interface closer to PollClient's.
Merge branch '9135-eventclient-run-forever-wip'
9213: Fix google-api-client dependency range in gemspecs.
9213: Improve gem loading in `arv`.
9213: Update arv's `gem install` suggestions.
Merge branch '9213-fix-arv-gems-wip'
9049: arv-copy checks and updates pipeline template filters.
Merge branch '9049-arv-copy-filters-wip'
8959: Remove redundant python-gflags fpm-info.sh.
Update Software Collections package name in CentOS 6 Dockerfiles.
Update Software Collections package name in Install Guide.
9316: Include documentation in CWL SDK.
9242: Refactor Python constant definitions in r-b-p.
9242: Python packages install libraries to the distro path.
Merge branch '9242-python-backport-prefix-wip'
9242: Update Python module paths for CentOS 6.
9242: Restore newer backported versions of Python packages.
9309: Look for fpm-info in backports/$PACKAGE_NAME.
9309: Add packages and tests for CentOS 7.
Merge branch '9309-centos-7-packages-wip'
9309: Separate PostgreSQL setup page in Install Guide.
Merge branch '9309-postgresql-install-guide-wip'
9187: Add priorities to crunch-dispatch-local test containers.
9309: Bugfix Ruby source install instructions for CentOS.
9370: Alphabetize the package_go_binary list.
9370: Package Crunch v2 dispatch services.
Merge branch '9370-crunch2-packages-wip'
9309: Add systemd support to Rails package scripts.
9345: Use same script to test all Rails packages.
Merge branch '9309-centos-7-rails-scripts-wip'
Package build directories get same owner as $WORKSPACE.
9309: Bump Rails package iteration.
9309: Add CentOS 7 to Install Guide.
Merge branch '9309-centos-7-install-guide-wip'
9408: Restore build-essential and ruby-dev to arvbox.
PySDK: Fix spelling of X-Keep-Desired-Replicas header.
Add libgnutls28-dev to arvados/jobs.
9369: Copyedits throughout the CWL docs.
9369: Copyedits throughout the CWL documentation.
9369: Copyedit CLI SDK install guide.
9459: arvbox-demo accepts version arguments for Git checkouts.
Merge branch '9459-arvbox-demo-version-arg-wip'
9262: Stop setting sbatch --priority in crunch-dispatch-slurm.
9659: Bump CWL SDK's versioned dependency on PySDK.
9705: Refactor out partials from compute node install guide.
9705: Add crunch-dispatch-slurm to the Install Guide.
9705: Add docker-cleaner unit file to Install Guide.
9706: Go service postinst improvements.
Bryan Cosca (18):
Added note about job reproducibility and version control with arvados_sdk_version, script_version, and docker_image to Writing a Pipeline template
6858: Added note about job reproducibility and version control with arvados_sdk_version, script_version, and docker_image to Writing a Pipeline template
6858: Added new section 'Re-using your pipeline run' and moved all version control parameters there for ease of reading. Added more explanation of all parameters
6858: Fixed syntax and added notes to re-use the same hashes from previous runs
Merge branch '6858-job-re-run-documentation'
closes #6858
6862: Added a quick guide to git, from git clone to git push.
6862: Moved clone arvados repository from tutorial-submit-job to git-arvados-guide, fixed some headers to include Arvados repository
6862: Fixed typos and syntax in git-arvados-guide
closes #6862
7015: Tested up to Accessing an Arvados VM with Webshell
7015: Checked up to Working with data sets
7015: Checked up to Concurrent Crunch tasks
7015: Finished going through user guide
7015: Removed whitespace
Merge branch '7015-update-user-guide'
6600: Added Retryloop to task_set_output(), current_task(), and current_job() to python SDK
Merge branch '6600-retry-job-helpers'
Colin Nolan (15):
7204: Setup for testing new `arg get` command.
7204: Created black box test cases for `arv get` command.
7204: Completed supporting methods for `arv get` test cases.
7204: Minor refactoring of methods supporting `arv get` test cases.
7204: Fixed bugs in `arg get` tests.7204: Fixed bugs in `arg get` tests.7204: Fixed bugs in `arg get` tests.7204: Fixed bugs in `arg get` tests.7204: Fixed bugs in `arg get` tests.7204: Fixed bugs in `arg get` tests.7204: Fixed bugs in `arg get` tests.
7204: Stubbed more test cases for `arg get` to cover use for only getting specific fields.
7204: Completed additional tests for `arg get`.
7204: Fixed bugs in tests for `arg get`.
7204: Added user documentation for `arv get` command.
7204: Completed fixing bugs in tests for `arv get`.
7204: Removed assertion of success exit code for `arv get --help`.
7204: Implemented changes to `arv get` test suggested by @tomclegg.
7324: Implemented deletion of previous nginx access log fifo before creation,
9897: Logs errors encountered by block prefetch workers.
9897: Changes as per discussion in #9897.
David (4):
Update prerequisites
Use postgresql options to reduce questions
Update install-workbench-app.html.textile.liquid
Update install-workbench-app.html.textile.liquid
Guillermo Carrasco (1):
Create temporary directory if it does not exist
Javier Bértoli (7):
Update documentation about get_anonymous_user_token.rb to prevent duplicates
Merge branch '10556-get_anonymous_user_token.rb-multiple-similar-records'
Ensure chown does not fail on disappearing files and dirs
Add RPM repository GPG key and modify CentOS information accordingly
Merge branch '8326-add-centos-repo-gpg-key'
Add parameter for FPM so it removes old directories on upgrade
Merge branch '12271-fpm-cleanup-on-rpm-upgrade' closes #12271
Jiayong Li (10):
Add --output-tags to arvados-cwl-runner and tests
Change output_tags to an attribute instead of kwarg, modify tests
Fix typo and modify test
Add num_tries to adding tags and tests for container API
Merge master to output-tags branch and resolve conflict
closes #10435
Change statfile in run.py to be able to raise OSError, and change the mock in test_pathmapper.py accordingly
refs #11362 Merge branch '11362-missing-input-sdk/python'
Change visit in pathmapper.py to raise OSError if input file is not found, change test_pathmapper.py to test it, and update python client version in setup.py
closes #11362 Merge branch '11362-missing-input-sdk/cwl'
Joshua C. Randall (39):
Merge pull request #1 from curoverse/master
Adds useful log warnings re: repo failures
Makes it possible to set docker_bin in crunch-job
Merge pull request #2 from wtsi-hgi/feature/arv-view
Blacklists pbr version 1.7.0 as it is not working with mock 1.3.0
Puts pbr 1.7.0 on blacklist for services/dockercleaner
Adds setuptools/pip upgrade in python3 virtualenv
7307: Fixes mocked up CGI output and adds comment regarding SplitHostPort error
7310: adds git v2 style regex to match http 500 error
7324: fixes /dev/stderr issue by creating and catting a fifo for nginx access_log
7262: blacklists pbr >= 1.7.0
Adds sanity check on number of collections retrieved
Changes GetCollection loop to more reliably fetch all collections
Makes changes suggested by tomclegg in review of PR #39
Modifies sanity check to query for the total expected number of collections
adds safety quotes and '_' before wildcard as suggested in PR #6 comments
adds safety quotes and '-[0-9]' before wildcard as suggested in PR #6 comments
cleans up debian/changelog generation per tomclegg's comment
uses variable for curl output filename and makes curl foLlow redirects
raises datamanager BatchSize from 50 to 1000
adds missing var collections SdkCollectionList
Fixes 8508 by creating a bad path explicitly
Adds better error reporting for problems with request body file
Adds checking for request body options that are both valid JSON and readable files
Renames 'redundancy' to 'replication_desired'
implement unsigned_manifest_text column
always omit unsigned_manifest_text when showing collection not in an index
add support for unsigned manifests to SizedDigests() in Go SDK
change keep-balance to request unsigned_manifest_text from API server
Adds no_count option to list method
Add --no-count option to CLI reference docs
Switch from no_count to count to avoid issues with arv CLI / trollop
Update CLI docs for --no-count
Add Count to Go SDK ResourceListParams
Change keep-balance to only request Count at beginning and end of collection retrieval
Fixes API server to return default columns for collections
update signature of SdNotify for go-systemd v14
fixes formatting on collection.go
fix s3_volume so that it can create empty recent/* objects on Ceph RadosGW
Joshua Randall (24):
Adds error handling for command not found
Adds `arv view` subcommand
Removes unnecessary check that stdout is a TTY
Moves printing of `arv edit` messages out-of-band
Merge remote-tracking branch 'hgi/develop' into fix/7205-arv-edit-tty-check
Changes new feature name to `arv get`
Makes bin_path local
Adds an abort if global_opts[:format] is not recognized
Merge branch 'fix/7205-arv-edit-tty-check' into feature/arv-view
adds numerous crunch-job performance improvements
pins python-llfuse version to 0.41.1 for fpm on all platforms
Unpins llfuse version and adds fuse 2.9.2 backport
adds centos6 backport of fuse 2.9.2
Adds additional .gitignore entries
Fixes reference to $js->{killtime} instead of $procinfo->{killtime}
8087: implements reading of request body from file
7263: Updates reapchildren to perform API query for new tasks in batches
makes -collection-batch-size an option as requested by @rchippada
8087: makes changes suggested by radhika
updates llfuse requirement and API to >=0.42
adds missing fix for llfuse 0.42+
adds backports specific to centos6
updates setattr for llfuse-python 0.42
updates test_mount to use st_mtime_ns instead of st_mtime
Lucas Di Pentima (476):
9408: Added apt-get flag to avoid installing 'Recommended:' packages, it's better to have an explicit dependency list. Also, added bzip2 that was being pulled implicitly and it's needed.
Merge branch '9408-arvbox-dockerfile-pulls-unnecessary-packages'
First try, don't think it'll work
9417: Replaced assertTrue() with assertEqual() on RetryNeedsMultipleServices's tests
Merge branch '9417-asserttrue-for-value-comparison'
9180: Changed some of the logic on ThreadLimiter and made unit tests to validate the new behaviour
Merge branch '9180-avoid-overreplication-keepclient'
Ready to start testing code
One last method before calling it a day
Some tests done on new code
PySDK put() refactoring ready, all local tests worked OK
9446: Applying Peter's review suggestions. refs #9446
Merge branch '9446-refactor-keep-parallel-write-strategy'
9463: Initial coding with tests
9463: Resume cache implementation (WIP)
9463: Lots of progress today, resume upload code written, many tests to do!
9463: Polishing the last details to make the integration tests work ok
9463: Added desired replication level to collection record on API Server when using Collection class
9463: Added to Collection a way to ask _BlockManager how many copies need to be written when uploading blocks to Keep
9463: Fixed some integration tests broken because of the replication_desired addition to Collection
9463: Finished first draft on arv-put command use of Collection class. Also, partial uploads resuming is working. Tests were written to validate this. Refs #9463 #8910
9463: New approach on this arv-put revamping: the ArvPutUploadJob class writes to a Collection and start a thread to periodically poll the upload status
9463: blank spaces cleanup
9438: Added a new var DefaultSecureClient and assigned a 5 minute timeout to both InsecureHTTPClient and DefaultSecureClient
9438: Eliminated redundant casting
Merge branch '9438-http-default-timeout'
9463: Using finally: on except: clause to stop the updater thread on error or finished work. When counting the collection size, stopped ignoring items that are not Collection or ArvadosFile, in case some other class is used in the future. When getting block locators (for example when called with --raw), don\'t make sure the blocks are flushed to Keep before asking for segments
9463: Updated integration test to mock current implementation
9463: Finished writing tests for ArvPutUploadJob
9551: Added support for env var ARVADOS_KEEP_SERVICES as a special case of a proxy.
9551: Change proxy parameter priority over environment variables. Updated docstring to mention ARVADOS_KEEP_SERVICES.
9551: Proxy URLs validation. Iteration optimization builind _keep_services list
9551: Replaced ARVADOS_KEEP_PROXY with ARVADOS_KEEP_SERVICES new var on tests. Added proxy URI validation test.
Merge branch '9551-python-keep-services-env'
9463: Use a constant as a template for initializing the empty cache
9463: Optimizations on _write_ffile()
9463: Added logging messages to warn the user about anomalous cases
9463: Temp files & dirs creation/cleanup moved to setUp and tearDown stages. Replaced usage of reporter function with a second wrapper function to make the test cleaner
9463: Parameter naming unification on _BlockManager. Corrected mocked discovery doc on test.
9463: Unified use of 'replication_desired' param on Collection class at instantiation.
9463: Unify replication_desired & write_copies parameters to only one, passing it at Collection's constructor and let the class decide which value is best from default when None is passed.
9617, 9618: Added validations and wrote the corresponding test. Also, corrected some other tests to conform this new requirement.
9617: Added validations on updates & tests on sucessful cases.
9617: Corrected style following suggestions and also compacted the code to avoid repetition.
9617: Added one more invalid constraint case to the tests, also check for container_uuid on successful test
9617: Reorganized code reusing the invalid_constraint list on both create & update tests. Separated the successful test case to be on its own.
9617: Corrected variable & test naming
Merge branch '9617-reject-container-reqs-without-vcpus'
9617, 9618: Added runtime constraints to container requests fixtures. Wrote a new test to load a CR from a fixture and update some attribute to test for the new validations
Merge branch '9617-reject-container-reqs-without-vcpus'
9463: On failure, commit the cache file too, before exiting. Wrote missing test about failed partial upload + resuming using mock function instead of injecting code manually.
9463: Mock usage corrections on a test that was producing failures on other tests
9333: Attribute validation on "select" to avoid invalid SQL statements
9463: Get Collection's manifest text including only those committed blocks, so that it can be correctly saved to the cache. Added a test for this new feature.
9463: Corrected test to prove that the manifest is correct when uncommitted blocks exist
9333: Treating the "attribute is not a string" error case the same as the other invalid attributes. Moved the checks to ArvadosModel. Updated tests.
9333: Removed superfluous quote marks on exception message
Merge branch '9333-api-server-invalid-sql'
9333: Adding collection model specific sttributes to be ignored on the validation mechanism to fix workbench integration test failures
Merge branch 'master' into 9333-api-server-invalid-sql
Merge branch '9333-api-server-invalid-sql'
9463: Setting up the checkpointer thread as a daemon so that it finishes when the command is cancelled through ctrl-c
9804: Removed the "Activity" pane from Collection View @ Workbench. Also removed a related query on the controller's show method, and a test.
9804: Tweaked the collection#show's pane widths. Fixed some integration_performance tests that were failing after the activity pane removal and removed an unused query at collection controller.
Merge branch '9804-drop-activity-pane'
9687: For a ContainerRequest being committed, if its container has the state "Complete" but exit_code not zero, return "Failed" as a state_label so that Workbench won't render its status as "Complete". Added fixture & test to check for this correction.
9463: Several changes to the PySDK, as described below:
9854: Added ArvadosModel::UnresolvableContainerError exception to return a 422 http status. Modified container resolving methods to raise this new exception instead of RecordNotFound. Updated related tests.
9463: Updated CWL tests to match Collection new behaviour.
Merge branch '9463-pysdk-amended'
Merge branch '9854-container-validations-produce-bad-errors'
9687: Updated state_label method changes to something more simple and compact. Added exit_code:0 to complete container fixtures. Added explicit test case to check for a not Failed container.
9687: Added method to access container's exit_code. Updated test to explicitly test for exit_code in addition to state_label.
9687: Added exit_code method to work_unit interface declarations. Simplified state_label method using the new exit_code accessor.
Merge branch 'master' into 9687-container-request-display
Merge branch '9687-container-request-display'
9858: Check if port-1 is available, retry if not when asking an available port for selenium.
9858: Added Errno::EACCES exception handling just in case it tries to open a low port as a user.
Merge branch '9858-selenium-locking-port-binding'
9623: Sort Container serialized hashed attributes for efficient comparison. Copied Job.deep_sort_hash method into Container assuming Job will be deprecated in the future.
9623: Added method to find a reusable container, used by ContainerRequest#resolve. Added some tests for this new method.
9623: Check for reusable containers in Completed state that has existing output and log data. Added additional tests to check for correct container reuse preferences.
9623: Cleaned up references to local constants about Container's states. Added optimizations on the serialized attributes sorting.
9623: Fixed test by avoiding Container reusability when calling create_minimal_req! multiple times with the same attributes.
9623: Added CR fixtures to make services/crunch-dispatch-local tests run OK again.
9623: Removed fixtures additions meant for Container reuse tests because some of them made other integration tests to fail.
9623: Rewrote tests to avoid using fixture records.
9623: Restored changes to fixture file
Merge branch 'master' into 9623-reuse-containers
9623: Moved deep_sort_hash method to ArvadosModel as it's used by both Job and Container classes.
9623: Container fixtures corrections.
9623: Fixed progress values to be between 0.0 and 1.0
9623: Changed query strategy for better performance. Also fixed the case where state = complete and several containers were found with different outputs.
9623: Added test cases on container request to check for container reuse flow. Also added more container reuse tests to cover missing cases.
9623: Several fixes addressing review comments:
Merge branch 'master' into 9623-reuse-containers
Merge branch '9623-reuse-containers'
10029: Added a session id for every new workbench request, to be sent as a parameter to the API server.
10029: Added client_session_id to Workbench's client request log.
10029: Added test to check for client_session_id parameter existence and format.
10037: Avoid showing progressbar on Workbench's Dashboard when a container is not running, instead show it's state label.
10029: Some changes/improvements described as follows:
10037: Added test to check for progress bar showing only on running containers
10037: Fixed test because of the fixture dates changes made to allow some containers to appear on the recent pipelines and processes list on the dashboard
10029: Merge branch 'master' into 10029-wb-send-request-id
Merge branch '10029-wb-send-request-id'
10037: Fixed alignment issues on the dashboard
10037: Test enhancement to check for other statuses on the dashboard, also adding pipeline instances on those checks.
10037: Changed test name to something more appropiate.
10037: Merge branch 'master' into 10037-useless-progress-bar
Merge branch '10037-useless-progress-bar'
9701: Merge branch '9463-change-arvput-use-collection-class' into 9701-collection-pack-small-files-alt
9701: Changes on the Python SDK to allow small file packing on Collection class:
9701: Test fixes
9701: Reverted the use of the optional 'flush' argument on ArvadosFileWriter.close()
9701: Several corrections/enhancements:
9701: Fixed a previous test to match new flush() behaviour. Added a new one to check
9701: Set repack_small_blocks() method as @synchronized
9701: Simplifying small bufferblock query
9701: Use a collection.OrderedDict instead of a simple dict to hold bufferblocks so that the packed files order is consistent. Updated related test.
9701: Superfluous variable eliminated
9701: Better bufferblock id generation.
9701: Added clarifying comments to the small block searching list comprehension.
9701: Wrapping actual alloc_bufferblock and delete_bufferblock methods inside synchronized calls, to be able to use them from within already synchronized BlockManager methods.
9701: Merge branch 'master' into 9701-collection-pack-small-files-alt
9701: Fixed arv-put to pass a couple of pending sdk/cli tests. refs #9463
9701: Replaced deprecated use of File.exists?() with File.exist()
Merge branch '9701-collection-pack-small-files-alt'
9463: Commit old arv-put command back, as the new one still have some performance issues when dealing with a lot of files.
Merge branch '9463-revert-arv-put-commit'
10243: Added a flush=True parameter to ArvadosFileWriter.close() so that the repacking feature is disabled by default.
9463: Reverted tests related to old arv-put version.
Merge branch '9463-revert-arv-put-commit'
Merge branch 'master' into 10243-make-packing-optional
10243: Explicitly calling close(flush=False) on tests so that they work with this new feature.
10236: Added --disable-event-listening to arv-mount command so that it doesn't subscribe to any API Server event.
10243: Merge branch 'master' into 10243-make-packing-optional
Merge branch '10243-make-packing-optional'
10236: Merge branch 'master' into 10236-disable-event-listening
Merge branch '10236-disable-event-listening'
10216: Added use_existing field to Container Request. When false and CR is committed, a new container will always be created.
10216: Modified test case, adding it to an already existing test.
10216: Merge branch 'master' into 10216-allow-cr-bypass-reuse
Merge branch '10216-allow-cr-bypass-reuse'
10315: Bring back arv-put command from 9701's branch.
10315: Allow small block packing passing flush=False.
10315: Reduced the time between updates.
10315: Enhanced performance when writing lots of small files without flushing by adding every closed file's size to a counter instead of scanning the pending bufferblock list every time.
10315: Brought back the new arv-put tests from 9701 branch.
10315: Moved the BlockManager's put threads lazy loading to __init__() to avoid locking issues when calling commit_bufferblock() fropm within an already @synchronized method.
10315: Added back the BlockManager's put threads lazy start, but with a specific lock, so it can be called from a @synchronized method.
10315: Update pending write size count just before packing small blocks to see if there really are enough to fill a full one.
10315: Moved conditional on pending write size checking
10315: Merge branch 'master' into 10315-new-arv-put-performance
Merge branch '10315-new-arv-put-performance'
10383: Added --update-collection parameter to arv-put so that it uploads new files into an existing collection.
10383: Fixed Collection.find() so that it returns None also when some partial part of the path does not exist. Updated test to cover more cases.
10383: Enhancements/fixes to the collection update feature:
10383: Tests have been adapted to the new behaviour.
10383: Fixed some tests because the new behaviour on arv-put refuses to run on 'resume' mode when the cache file belongs to a collection. The tests were not testing the resume feature so it was added the --no-resume flag to get them to pass.
10383: Using os.walk() to traverse input directories in an deterministic way.
10383: Re-done of the resume and update logic:
10383: Removing unnecessary locking, as Collection class is thread safe.
10383: Now --update-collection is not mutually exclusive with --resume and --no-resume.
10383: Tidying up internal collection management and naming.
10383: Unused exception removed.
10383: Fixed an error when using Collection.copy() to overwrite an existing ArvadosFile.
10383: Several fixes/enhancements to arv-put:
10383: Merge branch 'master' into 10383-arv-put-incremental-upload
10383: Fixed integration tests that got broken when the resume feature was implemented.
5737: Merge branch 'master' into 5737-ruby231
5737: Removed minitest: it wasn't really running the tests.
5737: Explicitly passing PID file path to passenger so that the tests pass again.
5737: Fixes to remove ruby warnings when running websocket integration tests.
5737: Fixes to remove ruby warnings when running collection api integration tests
5737: Fixed a bug in container reuse test code.
5737: Fixed several warning messages from running tests.
5737: Fixed unused variable declarations.
5737: Merge branch 'master' into 5737-ruby231
10587: Adding --version argument to Python SDK cli commands
10587: Added --version to arv-mount command.
10587: Add --version argument to arvados-node-manager.
5737: Merge branch 'master' into 5737-ruby231
Merge branch '5737-ruby231'
5737: Updated test so it pass on both 2.1 and 2.3 ruby versions.
Merge branch '5737-fix-broken-test-with-231'
10587: Added tests to Python commands to check for the --version argument.
10587: Removed use of multiprocessing module on --version tests. Added a common stderr/stdout redirector context manager for code reuse.
10587: Added setuptools dependency to fpm package building on arvados-python-client, arvados-fuse & arvados-node-manager.
10587: Added setuptools dependency to install_requires lists on arvados-cwl-runner, arvados-python-client, arvados-fuse and arvados-node-manager.
10383: Removed deprecated --max-manifest-depth argument from the help message and ignore its value in make_path.
10383: Changed the time between updates from 1 to 20 secs
10383: Check for expired tokens when getting already uploaded files.
10585: Merge branch 'master' into 10587-python-cli-version
Merge branch '10587-python-cli-version'
10383: Refactored the file upload decision code so that it first skims through
10383: Exception usage correction
10383: Added 2 tests to check that no resuming take place when using either --no-resume or --no-cache.
10383: Merge branch 'master' into 10383-arv-put-incremental-upload
10383: Start cache file from scratch when using --no-resume
10383: Added logger calls for all error/warning/info messages that were printed directly to stderr.
10383: Added --dry-run argument so that arv-put can be used inside a script to check if there is a pending file upload.
10383: Fixed misuse of logger.error()
Merge branch '10383-arv-put-incremental-upload'
10383: args.dry_run wasn't being passed to ArvPutUploadJob class. Also avoid output messages when in dry-run mode.
10383: Remove files from local_collection which don't exist locally, so that the bytes_written count is accurate.
10383: Merge branch 'master' into 10383-arv-put-incremental-upload
10383: Error out when using --name and --update-collection together.
10383: Fix for making arv-keep-get tests pass.
10383: Merge branch 'master' into 10383-arv-put-incremental-upload
Merge branch '10383-arv-put-incremental-upload'
10671: Set up values for pipeline instance started_at and finished_at attributes on state changes.
10671: Merge branch 'master' into 10671-pipeline-instance-finish-time
10671: Changed default assigments to be before validation. Also make sure started_at is not nil when pipeline instance is finished.
10671: Merge branch 'master' into 10671-pipeline-instance-finish-time
Merge branch '10671-pipeline-instance-finish-time'
10223: Added output_name column to container_request. When being finalized, if output_name is set then the output
10223: Added support to CR output_name on cwl-runner. Updated test.
10223: Merge branch 'master' into 10223-cr-set-output-name
10223: Test simplification: better name, assertion error description added, and use of preexisting fixture.
10223: Added relevant explanation on the rollback command
10223: Added test to check for missing full text search indexes
10223: Added CR's output_name to FTS index
Merge branch '10223-cr-set-output-name'
Merge branch '10813-arv-put-six-threads'
10813: arv-put with 6 upload threads, to be tested on production.
10813: Bug fixed on BlockManager's _get_manifest_text() when asked for only include committed blocks on collections with subcollections inside it.
10813: Fixed typo on argument documentation.
10813: Merge branch 'master' into 10813-arv-put-six-threads
10694: Changed the Provenance Graph orientation of Jobs to be the same way as it is rendered on Collections.
10694: Changed Pipeline Instance's Graph orientation to match Collections and Jobs.
10694: Flipped graph orientations so they are all (collections, jobs, pipelines) redered top-down.
10813: Removed unnecessary argument
10813: Added only_committed argument to ArvFile.manifest_text() method to maintain consistency with its counterpart from Collection class.
10813: Merge branch 'master' into 10813-arv-put-six-threads
Merge branch '10813-arv-put-six-threads'
Merge branch '10694-provenance-graph-consistency'
10932: Replaced the use of a list with a set to check if files on local collection are on the local file list, so that the resume start time is greatly reduced.
10932: Changed _file_paths from being a list to a set so we're not going to copy it when checking for missing files on local collection on resume start.
Merge branch '10932-arvput-slow-resuming'
10968: Changed the periodic update thread to run every 1 second while arv-put is checking which files to skip, only notifying the user via the progress indicator.
10968: Added a notification when uploading at least one directory, just to let know the user that the expected bytes count can take some time when trying to upload lots of files.
10968: Deleted space char at the end of a comment
Merge branch '10968-arvput-resume-progress'
3900: When deleting items inside a project, use the delete API. In the special case of trashing collections, first remove them from its parent project.
3900: Reverted changes so that only Collection deletion from a project get deleted via an delete API call.
10956: Get PDH from API server's response when saving a collection so that it doesn't have to be calculated when being asked for later on.
10956: When asked for the recently uploaded collection's pdh, arv-put will compute a pdh from the local collection's manifest and compare it with the API server provided version. If they differ, it will log a warning, always returning the API server's version.
10956: Fixed cwl test mocks to match behaviour change on collection class.
3900: Flipped item.destroy call for semantics reasons
3900: Merge branch 'master' into 3900-trash-collections-delete-api
Merge branch '3900-trash-collections-delete-api'
Merge branch '10956-arvput-pdh-fix'
11121: Add owner_uuid to output collection on crunch-job
11121: Added test.
11121: Merge branch 'master' into 11121-crunch-output-collection-owner
Merge branch '11121-crunch-output-collection-owner'
11002: Added test to make the bug happen.
11002: Improved test mocking a more suitable method and catching the specific exception type.
11002: Added missing assertion to test.
11002: When trying to save the cache's state before quitting, if an exception
11002: Merge branch 'master' into 11002-arvput-crash-fix
11002: Do not try to save internal state when receiving a KeyboardInterrupt exception.
11002: Track this specific error with its own exception class, for future-proofing.
11002: Don't save the state and log the stack trace before quitting upon catching an exception. Also, when receiving SIGINT (KeyboardInterrupt), just quit without any logging.
11002: Added note explaining why we're expecting a SystemExit to catch a SIGINT (KeyboardInterrupt)
11002: Merge branch 'master' into 11002-arvput-crash-fix
Merge branch '11002-arvput-crash-fix'
11139: Added default config parameter 'node_mem_scaling' to be applied to node's ram sticker values. Its default is 95%.
11139: Added new test to check for non-default values. Updated example config files.
11139: Merge branch 'master' into 11139-nodemanager-mem-scale-factor
Merge branch '11139-nodemanager-mem-scale-factor'
10218: Logging node information (cpu, mem, disk) by storing command outputs on the log collection. Added relevant test.
10218: Split multi-line command output so that each line is written to the logs independently.
11014: Check for PipelineIndex#index API to exist for rendering the compute node status pane on the Dashboard.
10218: Use a []string for the entire command instead of splitting it.
11014: When PipelineInstance API is off and also show_recent_collections_on_dashboard configuration is off, now the "Recent processes" panel will take full screen width.
Merge branch '11014-hide-node-status'
10218: Save the container record as a json file and also write it on the logs.
11170: Set up a thread to reap the status of squeue runs so that they don't become zombie processes.
10218: Merge branch 'master' into 10218-record-node-info
11170: Treat the squeue/scancel calls as files instead of treating them as processes. Calling close when finished reading from them.
11170: Updated tests to reflect the use of IO instead of File.
11170: Calling close method from an ensure block.
10218: Get the container record directly from the API server, instead of Marshaling runner.Container.
10218: Merge branch 'master' into 10218-record-node-info
10218: Ask for disk space on 2 specific filesystems (/ & /tmp) to avoid having the df command to return an error on special cases.
10218: Added calls to "df -i" to record free i-nodes. Combined stdout and stderr when running node info commands.
10218: Avoid running df commands more than necessary. Updated test to check for disk inodes report. Added deferred Close() call on CallRaw reader.
Merge branch '10218-record-node-info'
Merge branch '11170-stale-squeue-procs'
7824: Updated arv-ls to use CollectionReader's keys() API instead of the old all_files() / all_streams() methods.
7824: Fix upper/lower case ordering on arv-ls.
7824: Further ordering fixes. Now all arv-ls tests passes.
7824: Moved arv-get code to arvados.commands and replaced bin/arv-get with a stub caller.
7824: Updated code to use newer Collection APIs, leaving all_streams() and all_files() behind.
7824: Use logging facility to show error messages.
7824: Closing file when finished writing
7824: Log errors when trying to read the collection specified.
7824: Several improvements:
7824: Now a subdirectory can be asked to be listed. Also, enhanced error message when asking for a non existant collection.
7824: Added test cases.
6304: Avoid possible dns update race condition by writing the temp file with unique name to avoid collisions.
7824: Merge branch 'master' into 7824-arvls-arvput-collection-api-usage
Merge branch '7824-arvls-arvput-collection-api-usage'
6304: Clean up temp files that could exist in case of writing errors.
6304: Corrected test setup to make it pass.
6304: Merge branch 'master' into 6304-dns-update-fix
6304: Change assertion type for better failure message.
Merge branch '6304-dns-update-fix'
11341: Use NamedTemporaryFile instead of mkstemp for cache saving.
11341: Added additional information log for cases when the user ask to not use any cache.
11341: Fixed log message
Merge branch '11341-arvput-resume-error'
11502: When getting a manifest using a PDH, the manifest is stripped.
Merge branch '11502-arvget-flaky-test'
11502: Fix regression by writing a collection manifest with its access tokens in place when asked by PDH & UUID.
11502: Avoid races on test by getting the unstripped manifest version before the comparison.
11502: Use os.path.join instead of formatting path-like strings on tests.
11502: Instead of asking for a stripped manifest, replace the tokens with some constant string to do the comparison.
11502: Merge branch 'master' into 11502-unstripped-manifest-fix
Merge branch '11502-unstripped-manifest-fix'
10111: Added provenance graph for container requests.
10111: Merge branch 'master' into 10111-cr-provenance-graph
10111: Tests added
10111: Added "child" CRs to the graph, and removed container image and requesting container elements.
11185: Added "Re-run without reuse" button to the completed CR page, so that it
10111: Only include those mounts that are declared on the cwl input definition.
11185: On re-run without reuse, assign 'false' to CR.use_existing. Added related test.
10111: If there is no cwl.input.json object as part of a CR mount list, search for input PDHs on the entire mount list
10111: Replaced graph edges names: cr->child & mounts->input.
10111: Merge branch 'master' into 10111-cr-provenance-graph
Merge branch '10111-cr-provenance-graph'
11185: Now we only have one "Re-run" button that will give the user the option to disable container reuse on a modal dialog.
11185: When re-run, the default behaviour is to copy reuse disabled. Changed modal dialog to allow
11579: Test proving the bug: arv-put does not follow symlinks.
11579: Added symlinked dir traversal as the default behavior.
11579: Tests added.
11579: Explicitly following links on test.
11579: Check if paths exist, raise meaningful exception if not.
11185: Addressed several requests:
11185: Removed newly added completed-acr fixture and modified the already existing completed-older CR
11579: Removed feature about not uploading a symlinked dir twice.
11579: Added assertion on test to check for file symlinks to be copied by default.
11579: Merge branch 'master' into 11579-arvput-follow-symlinks
11185: Merge branch 'master' into 11185-wb-disable-reuse
Merge branch '11579-arvput-follow-symlinks'
Merge branch '11185-wb-disable-reuse'
10111: Skip log outputs from the provenance graph.
10111: Only add cr input collections by PDH.
10111: Flipped arrow between the cr and its childs cr.
Merge branch '10111-cr-prov-regression-fixes'
10111: Better labels on collection on the provenance graph.
10111: Fix integration test
10111: On output collections, name the collection node after the output_uuid reference.
10111: Refactored the graph creation code to minimize the amount of API calls neeeded.
10111: When dealing with output collection nodes, render the specific collection url instead of use its pdh
11647: Test proving that when small blocks get repacked and the resulting big block is
11647: When repacking small blocks into big ones, commit them synchronously.
11647: Merge branch 'master' into 11647-no-perm-tokens
Merge branch '11647-no-perm-tokens'
10111: Moved the api request related code to the controller.
10111: Merge branch 'master' into 10111-collection-labels
Merge branch '10111-collection-labels'
11501: A work unit running time with children was only computed calculating its direct
11501: Improved test name
11501: Fix some tests when trying to use the walltime method on show_runtime.
11684: Reverted easy fix to expose the bug: when there's a delay writing a block that's
11501: Merge branch 'master' into 11501-job-stats-discrepancy
11501: When calculating a work unit's running time, only include 'leaf' children, filtering those that were reused.
11501: Don't filter out reused children when calculating running time.
11501: Simplified helper method call. Changed wording of run time description.
Merge branch '11501-job-stats-discrepancy'
11684: When packing small blocks into one, save references of the files
11684: Instead of fiddling with ArvadosFile object's internals from the BlockManager
11684: Merge branch 'master' into 11684-unsigned-locator-fix
11684: ArvadosFile.flush() now check if it is the only owner of a bufferblock before deleting it, so the extra argument is not required.
Merge branch '11684-unsigned-locator-fix'
11557: Simplifying conditional usage
11557: Copy reused job's output to a collection on specified project (or Home project by default)
11557: Added log collection copying to local project when reusing jobs
11557: Create can_read link on reused jobs
11557: Don't assume log, output and components are always present on a job record.
11557: Merge branch 'master' into 11557-acr-output-col-perms
11557: Moved logic to done.done(). Updated related tests.
11557: Fixed can_read link creation, updated test to check this behavior.
11789: Added '--excluded <pattern>' argument so that files and/or directories can be excluded
11557: Merge branch 'master' into 11557-acr-output-col-perms
11557: Fixed the way that can_read permission link is created. Updated test.
Merge branch '11557-acr-output-col-perms'
11788: Simple fix to upload directory references as subcollections instead of
11778: Added explicit test that checks directory references upload behavior.
11788: Added conditional behaviour when using, or not, a trailing slash on directory references.
Merge branch '11788-arvput-dir-references'
11789: Splitted exclude_* arguments on ArvPutUploadJob class. Simplified
11789: Merge branch 'master' into 11789-arvput-exclude-flag
11789: Unified the exclude logic by removing expected_bytes_for() and moving
11557: Fixeg bug introduced on those cases when reusing a job from a different
Merge branch '11557-acr-output-col-perms'
11789: Path exclude patterns validation and fixes.
11789: Enhanced help message for --exclude
11789: Converting a filter() iterable to list, for python3 compatibility.
11789: Added tests.
2411: Updated check-copyright-notices script to correctly add the comment
11789: Merge branch 'master' into 11789-arvput-exclude-flag
11789: Replaced filter() with list comprehensions.
11789: Added support for the corner case of needing to use a pathname
Merge branch '11789-arvput-exclude-flag'
7475: Cancel jobs that cannot be satisfied instead of endlessly retry to run it.
7475: Catch exceptions when trying to cancel an unsatisfiable job,
7475: Check for job unsatisfiable type (job/container) and cancel it using
7475: Merge branch 'master' into 7475-nodemgr-unsatisfiable-job-comms
7475: Added integration test that checks for scancel to be called and a log entry
5652: Added --silent argument to arv-put to make it produce no
5652: Code style fixes.
5652: Added test that proves that arv-get doesn't write the progress
5652: Merge branch 'master' into 5652-arvputget-silent-mode
5652: Readability improvements.
Merge branch '5652-arvputget-silent-mode'
11167: Removed arv-get calling code from show_file.
11167: Refuse to start when keep-web isn't configured.
11167: Removed outdated comment.
11167: Test adjusted.
11167: Fixed integration test about collection's sharing link.
11167: Merge branch 'master' into 11167-wb-remove-arvget
11167: Removed conditional that check is keep_web is configured
11167: Reverted deletion of integration test. Modified it so it
11167: Re-added another removed integration test, adjusted to work
11167: Re-added jobs integration test, adjusted to work with keep-web.
7475: Merge branch 'master' into 7475-nodemgr-unsatisfiable-job-comms
7475: Fixed integration test
Merge branch '7475-nodemgr-unsatisfiable-job-comms'
11167: Removed conditional code
11167: Added helper function to be used on collection access tests via keep-web
11167: Documented default keep_web_url setting on workbench config file.
11167: Added response header whitelisting needed when using CORS
11167: Refactored tests to use new helper function.
11167: Merge branch 'master' into 11167-wb-remove-arvget
11167: Fixed integration test by replacing the use of PhantomJS with
11167: Removed unnecessary conditional
11167: Added visual studio code control dir to gitignore
8937: Added cache validation method to arv-put. For now it only
11167: Merge branch 'master' into 11167-wb-remove-arvget
11167: Removing unnecessary ActionController::Live from collections controller.
11167: Revert removing ActionController::Live from collections controller
Merge branch '11167-wb-remove-arvget'
11167: Fixed workbench package building scripts.
8937: Merge branch 'master' into 8937-arvput-cached-tokens
8937: Refactored _cache_is_valid() to catch specific exceptions. Fixed
8937: Token expiration is already being checked, removed cache
12055: On EC2 driver, pass the tag list to the node_create call
12055: Updated fake driver to make integration tests work again.
12055: Merge branch 'master' into 12055-nodemanager-ec2-tags
12055: Code simplification.
Merge branch '8937-arvput-cached-tokens'
12055: Changed nodemanager's libcloud dependency to use our v2.2
12055: Updated dev version number & dependency scripts
12055: Merge branch 'master' into 12055-nodemanager-ec2-tags
12055: Avoid RAM exhaustion on bootup by asking AWS only the AMI
12055: Merge branch '12055-nodemanager-ec2-tags'
12212: Adding configuration example for apllying instance profiles to created
12212: Added example on web documentation
Merge branch '12212-aws-instance-profile'
12236: Bumped libcloud to 2.2.1.dev2 that fixes a regression
Merge branch '12236-libcloud-blob-storage-regression'
12236: Bumped libcloud version dependency to 2.2.1.dev2
Merge branch '12236-libcloud-blob-storage-regression'
12195: Catch the special case when a RateLimitReachedError exception is raised,
12195: Log warning message when a retry-after header is not an integer.
12195: Fixed warning message when the Retry-After header value cannot
Merge branch '12195-nodemanager-quota-error'
12341: If a node_destroy call succesfully destroys the VM but takes too long
12341: Ensure shutdown_actor.stop() is called on node_finished_shutdown.
Merge branch '12341-slow-node-destroy'
12073: Prioritize stale node records that have a slot_number when
Merge branch '12073-nodemanager-stale-nodes-recs-bis'
LunarShaddow (1):
Change ssh public key to 700 after key-sync
Manoj (63):
6465: Moved the test "anonymous user accesses collection in shared project" from integration into controller tests.
6466:Added config parameter to disable public projects page and added a test to projects controller.
6465 - Moved test for anonymous user accessing collection in a shared project from integration
6466 - Update body html for enable public page and added tests to project controller test.
Merge branch 'master' into 6466-disable-public-projects
closes #6466
Merge branch 'master' into 6465-optimize-workbench-integration-tests
6465-Removed two description based tests in projects integration test and added four tests in projects controller test.
Merge branch 'master' into 6465-optimize-workbench-integration-tests
6465-Moved test "project viewer can't see project sharing tab" from
Merge branch 'master' into 6465-optimize-workbench-integration-tests
6465- Removed test "edit name and verify that a duplicate is not
6465-Added tests to project_controllers_test and users_controller
Merge branch 'master' into 6465-optimize-workbench-integration-tests
6465-Added tests to collections_controller_test, removed 4 tests
Merge branch 'master' of git.curoverse.com:arvados into 6465-optimize-workbench-integration-tests
6604- Disabled SSH public key notification when shell_in_a_box_url
Merge branch 'master' of git.curoverse.com:arvados into 6465-optimize-workbench-integration-tests
6604-Updated assert statements.
6617-Display https url in repositories panel in manage_account page and checks
closes #6604
6652-Added tests for the system menu and each of the pages in the
refs #6617
6465-Made changes that were recommended.
6652: Moved system menu test into application_layout_test.
6465: Added comments to explain test "find a project and edit description
Merge branch 'master' into 6465-optimize-workbench-integration-tests
6465: Added back bundled with
closes #6465
6465: Removed two tests from projects test and added two to projects
Merge branch 'master' into 6652-test-system-menu
closes #6652
refs #6465
6465: Changed "test dashboard button all #{target}" to use assert_includes
refs #6465
6588: Added new pages in notifications menu for repositories, virtual machines,
6588: Added tests for new pages and modified existing tests to not use
6588: Changed links to use routes and changed tests to match the
Merge branch 'master' into 6588-split-manage-account
6588: Added templates for new routes.
6414: When viewing a public project, the text "Public Projects" now
Merge branch 'master' into 6588-split-manage-account
6588: Changed routes to be under /users, used @object and updated
6414: Added an assertion to test if objects are equal.
closes #6414
6588: Formatted tests and changed others to be more efficient.
6588: Go directly to pages instead of going through home page and search
refs #6588
6588: Removed manage account button, modified tests and deleted views.
6588: Removed last reference to manage account page and reworded tests
6588: Changed settings menu to user settings menu and changed text if account is setup.
Merge branch 'master' into 6588-split-manage-account
6588: Removed unused objects in user_settings_menu_test.
Merge branch 'master' into 6588-split-manage-account
closes #6588
6652: Add a new repository and add a new authorized key buttons no longer
Merge branch 'master' into 6652-test-system-menu
6652: Added assertion and changed assertion to use assert_text instead
closes #6652
6640: Show only My Projects in dropdowns: use search modal for shared projects.
refs #6640
Merge branch 'master' into 6640-all-projects-dup
closes #6640
Misha Zatsman (53):
Initial commit of datamanager.py command line utility. So far all information comes from the python SDK. Future versions will talk to the keep servers as well.
Updated message to be clearer.
Deleted unused code, added code to read blocks on keep server.
Merge branch 'master' of git.curoverse.com:arvados into 1968-monitor-disk-usage
Added manifest size to calculations.
Modified algorithm to take disk replication when counting usage.
Added logging to datamanager, as a step towards writing http server.
Moved main flow into loadAllData method to get ready for multithreading.
Started using global to actually assign to global variables instead of local variables that shadow them. Added webserver and threads!
Added user page. Started using send_error. Added port flag.
Added handler for collections. Also renamed byte_size to byteSize to follow method naming convention.
Added ability to disable server by specifying port 0. Started keeping api reponse just in case. Started tracking replication level for peristers (although still working on using it everywhere). Added maxdict to make replication tracking easier.
Started reporting unweighted persist usage based on requested replication, not actual replication. refs #2572
Added method to computed weighted cost of different replication levels. Added main method to aid in unit testing.
Added unittests for computeWeightedReplicationCosts. Moved arg parsing to main block so that I can use commandline flags for tests when testing. Fixed lots of bugs surfaced by tests.
Added memoization to computeWeightedReplicationCosts.
Added weighted persist disk usage. Removed debugging line I had forgotten. refs #2572.
Added disk usage logging! logs are written to api server.
Added reporting of persisted blocks which are under or over replicated. Also added reporting of blocks without collections. refs #2574
Have a new view showing users at /users/storage. No storage info yet, but it's a start.
Working users/storage page!
Merge branch 'master' of git.curoverse.com:arvados into 2612-workbench-displays-user-storage
Made sizes human readable, added measurement date column.
Shortened date display, put long date view in mouseover text.
Merge branch 'master' of git.curoverse.com:arvados into 2612-workbench-displays-user-storage
Added row to display totals.
Added link to storage page, replaced tabs with spaces.
Merge branch 'master' of git.curoverse.com:arvados into 2612-workbench-displays-user-storage
Fixed indentation.
Merge branch '2612-workbench-displays-user-storage'
Fixing indentation. Closes #2688
Removed hack which was compensating for keep server bug which has been fixed.
Moved datamanger prototype to experimental directory to indicate that
Added most of the garbage collection reporting for #2622. Deleted some whitespace.
Added retrieval of stats from keep servers and computing total and free space on keep disks.
Added computation of resulting free space in garbage collection report.
Added printing of garbage collection report to CSV file. Fixed bug in free disk space computation. Closes #2622
Added hack to deal with the fact that keep servers are reporting each other's disks.
Added code to compute histogram from garbage collection list.
Renamed info to properties to match the name used by arvados.
Added logging of block age vs free space histogram. Fixed an indentation bug. Fixed a bug where I forgot to declare a variable as global. Corrected histogram description in comments.
Fixed bug where block mtimes were stored as strings instead of ints. Closes #2631
Added div for histogram and style for it.
Added printing of histogram json.
Added cache age histogram!
Avoided printing graph if no data. Added date histogram was generated.
Merge branch 'master' of git.curoverse.com:arvados into 2638-add-cache-age-disk-usage-histogram-to-workbench
Added some documentation and renamed variables after Tim's review.
Merge branch 'master' of git.curoverse.com:arvados into 2638-add-cache-age-disk-usage-histogram-to-workbench
Merge branch '2638-add-cache-age-disk-usage-histogram-to-workbench' closes #2638
Reversed histogram format, so timestamps come first.
Merge branch '1968-monitor-disk-usage'
Merge branch 'master' of git.curoverse.com:arvados
Nancy Ouyang (39):
added by_id subdirectory notes and arvados#collection
4926: Added 'getting started' section with three image galleries.
4926: typos & rewording
minor typos
more typos
refs #4926 Merge branch '4926-getting-started'
4926: Fixed images to be consistently 900x500
refs #4926 #5072 Merge branch '4926-fix-imagesizes'
Changed navbars, homepage, to let /doc/index.html function as homepage
Minor fixes, see #5090
4926: renamed share.png to sharing.png due to adblock on firefox
refs #4926 Merge branch '4926-rename-share.png'
refs #5090 Merge branch '5090-doc-to-homepage'
5194: Quickfix, disambiguated getting started and user guide sections, added 'next steps' to getting started guide
5194: minor fixes
closes #5243, #5194 Merge branch '5194-quickfix-disambiguate-gettingstarted-sections'
first page prototype
fix merge conflicts
Merge branch '5493-getting-started-modal' of git.curoverse.com:arvados into 5493-getting-started-modal
text content done; formatting needed
dropped a header
formatted first 2 pages
minor format changes
format pg 3, 4
dashed line
typo
Merge branch 'master' of git.curoverse.com:arvados into 5440-remove-doc-getting-started
1 2 3 ul>ol 3rd page
Merge branch 'master' of git.curoverse.com:arvados into 5440-remove-doc-getting-started
5440: remove all links to /start temporarily to avoid confusing user
linkfix, reword quickstart section on homepage
5440: reword quickstart
5493: fix yellow gif
refs #5493: Merge branch '5493-fix-yellow-gif'
refs 5930: misc. small documentation fixes
5930: fixed as per code review
closes #5930. Merge branch '5930-smalldocfix'
7607: added logic to remove getting started dialogue by default. Still needs tests.
7607: enablee flag should be set to false by default
Nico Cesar (75):
initial version to have a --bridge parameter
fixing some trailing whitespaces
trailing whitespaces
Merge branch 'master' into 5998-arvdock-bridge-option
$start_bridge -> $bridge
Merge branch '5998-arvdock-bridge-option'
adding the creation of plot data that jenkins plot pluging will be able to read
adding 3 more test for the performance suite.
6061: plot data
6061: comment cleanup and final version tested. it works
Merge branch '6061-add-performance-plots' refs #6061
6035: add support for --upload as an option
style and fashion for classy 'if' statements
Merge branch '6035-docker-upload-option'
6035: add support for --upload as an option
Merge branch '6135-docker-git-tag'
bundler 1.10 brakes workbench build because
added shell-image back again to the build. I checked on my local machine and works great
openssl self cert creation failed because missing of "tmp" directory.
qr1hi-automated-performance-suite is failing because the test doesnt give enough time for the page to render
qr1hi-automated-performance-suite is failing because the test doesnt give enough time for the page to render (now 50s makes the test past.). no issue #
login patch because DockerHub is not friendly
we need to manually install python-pyasn1=0.1.7 python-pyasn1-modules=0.0.5 because apt-get wont resolve correctly.
Dockerfile had an error. fixed
5989: create directory structure for temp files
5989: bug_1 added puppet repository patch
Merge branch '5989-bug_1' merge with puppet repo "copy and paste" to sync up duplicated code.
5989: create directory structure for temp files
5989: bug_1 added puppet repository patch
added /etc/default/arvados_pam as a parameter
6384: reverting names
Merge branch '6384-libpam-arvados'
qiuck fix to arvados-pam vulnerability
added exit if git pull fails
scopes in usage() are wrong.
one of the scopes wasnt working
7041: pyvcf was missing from backports in trusty
docker builds are failing for compute nodes.
6034: added scafolding of the first initscript
7330: curl isnt available by default in ubuntu 14.04. added the apt-get
7330: apt-get command had a typo
7330: making sso install guide similar to workbench and api server
7330: 'edit' should be replaced by 'editor' that will go to the default editor of the ditro
7460: c97qk doesnt need mailchimp deployed. this caused deploy to fail in Ubuntu
hkp://pool.sks-keyservers.net:80 is needed to use the transaparent proxy
reverted hkp port 80 explicitly
Merge branch '7591-httplib2-perms-fix-wip'
added more time to ajax timeout
7942: changed capybara timeout from 5 sec to 10 sec so iceweasel in ci.curoverse.com doesnt break as easily
added git_internal_dir init in postinst.sh
I "fonud" a typo
package building is failing because of bad version pinning
Merge branch '8712-fuse-cache-reload-bug'
8959: pinning out the version of python-gflags
I <3 pythong-gflags
Merge branch '8959-python-gflags-dependencies'
Revert "I <3 pythong-gflags"
Merge branch '8959-python-gflags-dependencies'
adding new package for block checks
Merge branch '9014-keep-block-check-package'
bumped the selenium driver version to 2.53.1 to see if we can manage our way out failed builds
9459: added test_in_arvbox.sh and arvbox the docker tag parameter.
Merge branch '9459-test_with_arvbox_tweaks'
Merge branch '9614-cwl-no-websockets'
package has a .service missing
Merge branch '10228-arvados-git-httpd-service'
10232: --dont-create-user flag and sanity check for non-existing users
10232: dont create user logic was wrong
11305: expanded documentation for docker migration
updated documentation about the docker 1.9 migration process
we need the tagged image to do this
11305: added kernel and module dependency for docker migration in doc
Merge branch '11305-migrate-docker19-doc'
11980: debian9 packaging plus llfuse >= 1.2
Merge branch '11980-debian9'
Peter Amstutz (3325):
Merge branch 'master' of git.clinicalfuture.com:arvados into 1637-improve-arv-tutorial
Rewriting documentation work in progress.
Documentation work in progress, making evening check-in.
A little bit of documentation work from home
Merge remote-tracking branch 'origin/master' into 1637-improve-arv-tutorial
Finished first tutorial.
Cleaned up old files and reorganized navigation order.
Cleaned up comments to be hidden from rendered page (still visible in
Missing file
Removed numbering and put navorder for tutorial 2 at 102 to hide it.
Merge branch '1637-improve-arv-tutorial'
Formatting tweaks, mosting changing <pre> to textiles block code (bc.) mark.
* split out sections into their own documents
Merge remote-tracking branch 'origin' into 1495-crunch-documentation
* Mostly done on "Debug a Crunch Script"
Reorganization, added parallel jobs and new pipeline tutorial.
Reorganized user manual into sections
* Fixed links
Working on traits document
* Almost done with trait search tutorial
* Finished trait search tutorial
* Fixed template so subsections only show up in userguide
* mailto:arvados at clinicalfuture.com is now arvados at curoverse.com
Merge branch '1495-crunch-documentation'
* User is now taken to a "welcome" page where they click button to log in
* A couple of tweaks in response to Tom's feedback
Added dax image to repo
Merge remote-tracking branch 'origin' into 1678-new-active-user-orientation
Starting working on wizard for jobs/pipelines
Merge remote-tracking branch 'remotes/origin/1636-inactive-user' into 1678-new-user-orientation-pete-wfh
Wizards that check for uploading collections, running a crunch job,
Merge branch '1636-inactive-user' into 1678-new-active-user-orientation
Floated Dax to the left instead of the right.
Merge branch '1678-new-user-orientation-pete-wfh' of git.clinicalfuture.com:arvados into 1678-new-active-user-orientation
* Added "Docs" menu to top navbar
Bolded the links so they are more obvious.
Tweaked text about ssh keys
Merge remote-tracking branch 'origin' into 1692-redesign-dashboard
Merge branch 'master' of git.clinicalfuture.com:arvados into 1692-redesign-dashboard
Dashboard now consists of recent collections, jobs, and pipeline instances.
Displays contents of manifest instead of manifest locator.
Added pip requirements.txt to automatically install python dependencies for Arvados Python SDK.
Updated Gemfile to include all dependencies of 'arv' command line tool so bundler
Merge branch 'master' of git.clinicalfuture.com:arvados into 1692-redesign-dashboard
New two-column dashboard layout suggested by Adam
Added system status box
Account status is now part of the "notifications" area
Replaces shelling out to 'curl' with httpclient library.
* Updated ssh-access documentation for workbench dashboard changes
Merge branch '1692-redesign-dashboard'
Better CSS fix for menu bar obscuring the target when following intra-document links.
Merge branch 'master' of git.clinicalfuture.com:arvados
Started tinkering with zenweb
Added redirect to config file.
Merge branch 'master' of git.clinicalfuture.com:arvados into 1685-api-server-redirect-to-workbench
Default redirect to workbench in production configuration to workbench.`hostname`
Merge branch '1685-api-server-redirect-to-workbench'
Added config.arvados_docsite, updated links, all tested and working.
Added mailing lists, irc
Added more links
Missing period.
Merge branch '1788-support-and-feedback'
Added theme support to workbench.
Merge branch 'master' of git.clinicalfuture.com:arvados
Tweaked user home template to make theming easier.
Merge branch 'master' of git.clinicalfuture.com:arvados
Added redirect to config file.
Default redirect to workbench in production configuration to workbench.`hostname`
Documentation tweaks based on Nadia's feedback.
Merge branch 'master' of git.clinicalfuture.com:arvados
Object pages now show incoming/outgoing links.
Workbench explicitly requests 1000 results instead of getting the default
Typo fix
Improving link names
Still working on link naming
Zenweb work in progress
Merge remote-tracking branch 'origin' into 1862-show-object-links
Merge commit '030d673e455ac7fa92732eafe95f1db12f6a46b6' into 1862-show-object-links
Finished improving rendering of UUIDs for now.
Merge branch 'master' of git.clinicalfuture.com:arvados
Reworked Files tab of collection view a bit.
Merged 1862-show-object-links into master
Merge branch 'master' of git.clinicalfuture.com:arvados
Moved "size" column to last and added SI order of magnitude suffixes instead of showing raw byte counts.
Merge branch '1748-workbench-view-files'
Story 1879 "log" link on dashboard links directly to log file.
Merge branch 'master' of git.clinicalfuture.com:arvados
Moves "add new item" button from bottom of the page to the top. Refs #1877
Merge branch 'master' of git.clinicalfuture.com:arvados
Logging out now returns you to the login page, rather than an empty "you have logged out" page. Refs #1865
arv-get replaces spaces in get_prefix with \040 to match the manifest. Refs #1849
Fixed Python SDK to quote/unquote spaces when reading/writing manifests. Refs #1849
Merge branch 'master' of git.clinicalfuture.com:arvados
Added validate step, needs testing
Merge branch 'master' into 1880-check-duplicate-public-key
Adds validate :public_key_must_be_unique to AuthorizedKey which checks to see if the key
Fixed spacing on dashboard and generic table page when tables are empty.
Fixed spaces in manifest stream names. Refs #1849.
refs #1912
Merge branch 'master' of git.clinicalfuture.com:arvados
Removed spurious '>' (typo)
Changed upload path in scp command to load to /scratch/you instead of just /scratch, tweaked markup to try and emphasize that "you" that should be replaced by your own username.
Merge branch 'master' of git.clinicalfuture.com:arvados
Moving Collection, Stream and Keep classes into their own files
Renamed Keep/Stream/Collection submodules to keep/stream/collection (lower case)
Added Python SDK section to documentation
Script to build python documentation (requires epydoc)
Moved Python SDK and crunch utility scripts pages to SDK Reference section.
Fixed path to glyphicons
Started tutorial on running external programs
collection.py import stream
Removed "Tutorial:" from titles since that was an artifact from before we had improved the navbar. Also fixed typo in tutorial-trait-search.
New tutorial for running external programs
Fixed import problems due to splitting module into multiple files
Merge branch 'master' of git.clinicalfuture.com:arvados
Merge branch '1911-python-sdk-pydoc'
Merge branch '1868-tutorial-running-binary-progs'
Merge remote-tracking branch 'origin' into 1786-replace-jekyll-with-zenweb
Zenweb work in progress, adding plugin for liquid so we hopefully don't have to rewrite our pages.
Added doc output directories to .gitignore
Added doc/_site to gitignore
Gemfile now just points to gemspec so same dependency information is not listed twice.
Tweaked running external program tutorial to make it more explicit that md5sum is the external program used in the example.
Checkpoint, draws a somewhat useful graph, needs a lot of work though.
Follows both jobs and links to show provenance for a collection
Performance fix to only visit a node once.
Use javascript to set the size of the svg box so that the horizontal scrollbar stays on screen.
Adjusts scrollbar to maintain center of image when zooming
Hooked into bootstrap/jquery to resize svg box when tab is shown.
Left navbar uses bootstrap "nav-header" class for section headers.
Initial work adding provenance query to API server
Provenance graph now uses full width of window
Work in progress
Merge branch 'master' into 1977-provenance-report
Merge commit 'e17169e05c2e69d5bb46792bc33ef3c92f9c5237' into 1977-provenance-report
API server /collections/uuid/provenance endpoint works.
Workbench is able to put together a graph from the provenance document served by the API server. Needs more testing.
Fixed bugs in creating graph from API provenance report.
Working on visualization of pipelines
Merge commit '2728f59' into 1977-provenance-report
Merge branch 'master' into 1977-provenance-report
Started refactoring
Merge remote-tracking branch 'origin/master' into 1977-provenance-report
Very nice provenance diagram for pipelines.
Now renders collections using up to the first three files in the collection
Further improvements to provenance graphs, added ability to select a few jobs and shows the graph for those jobs.
Color-coded graphical pipeline comparison, alternate approach to Tom's
Refactored code for managing divs containing svg (manages scrollbars, zoom buttons).
Working on javascript magic to resize svg div so scrolling works nicely. Seems to work pretty well except when the svg div is the last thing on the page.
Tweak to account for width of scrolbar
Fixed bug in vertical scrolling by setting padding to size of graph or max size instead of dynamic size based on screen position.
Tweaked text a little to make tail/subject head/object correspondence a little clearer.
Merge branch 'master' of git.clinicalfuture.com:arvados
Can now use 'contains' to query individual columns in where() queries, not just any: contains.
Merge branch '1977-provenance-report' of git.clinicalfuture.com:arvados into 1977-provenance-report
* Added used_by query for collections
Merge branch '1977-provenance-report' of git.clinicalfuture.com:arvados into 1977-provenance-report
Added stroke-linecap: round to make lines look nicer.
Reworked top menu. Started upgrading to bootstrap 3.
Menubar and left nav redesign in progress.
Fixed search box for chrome
* Left nav correctly highlights current section
Updating bootstrap-editable to bootstrap3-x-editable
Editable fields works again
Fixed up some glyphicons
Harmonized all sections to use the new content template and layout
Missing from previous commit
Added css to put back badge styles that went away in bootstrap 3
badges.css.scss doesn't actually require bootstrap
Merge branch '1979-workbench-ui-PA'
Fixed/refactored "smart-sizing" to adjust the content area to the size of the window so that the content area scrolls but the page as a whole shouldn't scroll.
Merge branch 'master' into 1786-replace-jekyll-with-zenweb
Jekyll is dead, long live Zenweb!
Merge remote-tracking branch 'origin/master' into 1786-replace-jekyll-with-zenweb
Python documentation step will now report a warning if epydoc isn't in PATH.
Merge branch 'master' of git.clinicalfuture.com:arvados
Removed mkpydoc.sh which is obsoleted by Rakefile task
Started updating docs page to bootstrap 3.0
Added "vendor" to doc generation ignore list for deployment situations
Updated theme to bootstrap 3 to match workbench. Started updating ssh keys tutorial.
Merge remote-tracking branch 'origin/master' into 2061-update-docs
Fixed bug with notification triggering on wrong condition. Also removed old "notifications" partial.
Merge remote-tracking branch 'origin/master' into 2061-update-docs
Updating documentation
Fixed bug with incoming links not showing in metadata tab. Resolves #2080
Can now click through to group pages on groups table
Hack to fix job log links (because crunch still produces identifiers with +K at qr1hi)
Finished updating tutorials for latest sprint
Merge branch '2061-update-docs'
Merge branch 'master' of git.clinicalfuture.com:arvados
Fixed {% include alert-stub %} that shouldn't have been there on a couple of pages.
working on normalization
Normalize method seems to work pretty well now.
Collections are now automatically normalized on both read and write. Tests updated.
Work in progress
Fixing things up
Unit tests for arvados.locators_and_ranges
Added readfrom()
Tests pass
Working on arv-ls
implemented arv-ls to replace whls
added -s option
Added arv-normalize
Added arv keep normalized
Put some code into collection model to do normalization but I don't think that is the right place for it.
Do a partial normalization of file list before sending to client.
Fixed file page to sort by stream name, then by file name
Work in progress arv-mount based on llfuse and supporting directories listings.
Directory browsing works, working on error handling for file reading.
Working on fuse mount
Added comments and some refactoring.
installing llfuse from pip instead of debian
changed fusepy to llfuse
Added basic unit test for fuse mount.
Added test for magic directories.
caching wip
Fixed caching so that reads by other threads for a block which is already
Added --exec, needs testing.
--exec works
Added arv-mount
Changed JOB_WORK to TASK_WORK in mounting keep directory.
Added get_job_param_mount() and get_task_param_mount()
Added symlink arv-mount, arv-normalize, arv-ls to cli/bin
get_job_param_mount() and get_task_param_mount() tested and working.
Added StreamFileReader.as_manifest() back and added test for it.
Added alternate option to one_task_per_input_file to encode specification of input files as collection/stream/file instead of raw manifest text.
Hash now uses get_task_param_mount() to read from fuse mount instead of CollectionReader.
Merge branch '1965-fuse-support-directories'
Merge branch 'master' of git.clinicalfuture.com:arvados
Added localStorage selection to generic index pages.
Added selection box to collections
Frontend feature to add items to locally-stored selection list that is session-persistent.
Fixed url_for() so that links/redirects to workbench records work as expected.
Checkpoint, working on form to set component inputs.
Merge remote-tracking branch 'origin/master' into 2049-run-job-from-workbench
Pipeline parameter editing supports selection box for choosing from workbench persistent selection list. Actually committing changes not tested yet.
Editing input values of pipeline_instances mostly works, but needs polishing.
UI for editing component parameters now correctly highlights the background of required parameters that are not set, and removes the highlight when the parameter is set.
Moved code specific to editing pipelines from editable.js to pipeline_instance.js.
Merge remote-tracking branch 'origin/master' into 2049-run-job-from-workbench
Improved/fixed color coding of job status for pipelines.
Merge remote-tracking branch 'origin/master' into 2049-run-job-from-workbench
Increased refresh rate to every 15 seconds from every 30 seconds.
POST works to actions controller, still need to implement the actual action to build the manifest.
Fixed handling of optional/required parameters with no specified or default value when constructing initial components hash.
Finished code to assemble a new manifest from the supplied files and collections.
Fixed broken links in documentation that were using {{site.basedoc}} instead of {{site.baseurl}}.
Show images in files inline.
Merge branch '2049-run-job-from-workbench' into 1971-show-image-thumbnails
In theory this works but I need a keep server to test it.
Gemfile update using ruby 2.1.1
Added section about installing libwarehouse-perl.
Merge remote-tracking branch 'origin/master' into 2049-run-job-from-workbench
Working on having crunch-dispatch manage pipelines.
Fixed default inactive user page.
Successfully starts first stage of active pipelines.
Merge remote-tracking branch 'origin/master' into 2049-run-job-from-workbench
Running pipeline from workbench works!
Fixed progress bar alignment on Firefox.
Tested, works great.
Made regex case-insensitive and added svg to list of extensions.
Improvements and bugfixes refs #2242.
Fixed data title
Merge remote-tracking branch 'origin/master' into 2042-new-collection-from-selected-files
Fixed bug when selection list was cleared.
Provenance graphing bugfixes
Fixed styling of selection dropdown menu.
Merge remote-tracking branch 'origin/master' into 2049-run-job-from-workbench-correct
Merge remote-tracking branch 'origin/master' into 2042-new-collection-from-selected-files
Merge remote-tracking branch 'origin/master' into 1971-show-image-thumbnails
Merge remote-tracking branch 'refs/remotes/origin/2049-run-job-from-workbench-TC' into 2049-run-job-from-workbench-correct
* Pipeline components are topologically sorted based on :output_of, then sorted lexically.
render_editable_subattribute() uses link_to_if_arvados_object() if the field is not editable.
Merge branch '2049-run-job-from-workbench-correct' of git.clinicalfuture.com:arvados into 2049-run-job-from-workbench-correct
* Added README with information about building/viewing the docs.
Updating and reorganizing tutorials based on new features and feedback.
Removed out of order link to next tutorial.
Merge branch 'master' of git.clinicalfuture.com:arvados
Merge branch '2042-new-collection-from-selected-files'
Reorganizing documentation work in progress, checkpointing so that Brett can start looking through it.
Finished first draft of tutorial running pipeline from workbench.
More documentation updates and reorganization.
Removed h1. titles from pages because it is now part of the page template.
Added liquid template to link to next and previous pages.
Ran linkchecker and fixed links.
Bugfixes to provenance graph:
Merge branch 'master' of git.clinicalfuture.com:arvados
Merge branch '2107-update-documentation'
Merge branch 'master' of git.clinicalfuture.com:arvados
Replaces postgres-specific 'now()' with standard SQL CURRENT_TIMESTAMP
Adds scopes: ["all"] when creating a new api client authorization, because database schema does not permit have a null scopes column.
Added section on setting up SSO server
Working on improving install documentation.
Updated workbench documentation.
Fixed typo in secret_token.rb
Added a README with pointers to the documentation, COPYING with license information, the relevant licenses, as well as a license section to the user guide.
Added more links to information in README.
Formatting tweaking to look good on github.
Added offset to 'list' method on resources.
Filters out "show", "index" and "destroy" methods from discovery document generation because they are
* Added support for offset and limit query parameters (?limit=x&offset=y) to workbench index pages.
Added "Displaying X of Y out of Z"
Merge remote-tracking branch 'origin/master' into 2075-workbench-paging
Integration test for permission links.
* Allow the system user to update job columns, even if the job is "locked"
Added "nondeterministic" column to Jobs table via migration.
Merge branch '2278-crunch-dispatcher-monitor-processes' into 2051-nondeterministic-jobs
* New code to find resolve commit versions and ranges by scanning repositories with "git rev-list".
Added tests for permissions to reuse jobs, tests for excluding commits.
* Added migration to break script_version into separate "repository" and "script_version" fields
Clear rails cache on startup.
Added documentation for jobs and pipelines, and updated docs to reflect that "repository" has been split out from "script_version" and is now required.
Documentation fixes based on running linkchecker.
Enforces that repository, script, script_version, script_parameters are provided.
- Made a note about intentionally not setting "success"
Silly git staging.
Merge branch '2278-crunch-dispatcher-monitor-processes'
Merge remote-tracking branch 'origin/master' into 2051-nondeterministic-jobs
Merge branch '2193-fix-permissions-cache' (no functional changes, only unit tests)
Fixes bug loading API tokens page.
Merge branch 'master' of git.clinicalfuture.com:arvados
Merge branch 'master' of git.clinicalfuture.com:arvados
Working on re-using token per pipeline and expiring token when the pipeline is done.
Expiring pipeline tokens works.
Fixed syntax error typo. We need tests for crunch-dispatcher.rb!
Merge branch 'master' of git.clinicalfuture.com:arvados
Fixed pipeline templates page that was broken by paging merge.
Fixed paging so that it won't fail if the necessary methods are missing on the results object.
Arvados model now validates that all "foreign_key_attributes" exist before create/update.
Fighting with tests
All tests pass!
Added 'is_a' filter and tests, refs #2418 #2228
Removed internal uses of _kind column in API server. Tests pass.
Actually remove _kind columns from schema and tests.
Removed '_kind' from documentation.
Merge branch 'master' into 2051-nondeterministic-jobs
Fixed tests
Removed trailing whitespace.
Removed head_kind and tail_kind from workbench.
Reworked input validation on find_commit_range (for real this time!) and added tests.
Fixes #2047: Catch connection errors in 'arv' and report the error nicely
Merge branch 'master' of git.clinicalfuture.com:arvados
Added more tests and more aggressive input checking.
Merge branch 'master' into 2051-nondeterministic-jobs
Added fix to workbench dashboard so that workbench integration tests pass.
Fixes "sec_error_reused_issuer_and_serial" SSL error in Firefox.
Merge branch '2051-nondeterministic-jobs'
Stylistic changes based on comments from Tom/Brett
Merge branch 'master' into 2333-crunch-dispatch-token-management
Fixed dangling 'end'
Merged branch 1741-internal-git-repo into 2333-crunch-dispatch-token-management
Merged master
API server tests pass, workbench tests are failing
Merge remote-tracking branch 'origin/master' into origin-2228-check-filter-uuid-columns
Updated Job section of API documentation regarding repository and
Reorganized job/pipeline docs into relevant API reference sections
* Refactored to remove load_kind_params filter and instead override load_where_param and load_filters_param in the links and logs controllers to add the _kind functionality.
Improved language about git revisions a bit based on review feeback.
Merge remote-tracking branch 'origin/master' into origin-2228-check-filter-uuid-columns
Reverted accidental change to Gemfile.
Links controller checks that head/tail_kind matches head/tail_uuid.
Adds tags to link text for anything using #link_to_if_arvados_object for a collection
Removes spurious whitespace.
Reverted Gemfile.lock
Javascript to replace <span> tags used for labels to square brackets for
Merge branch 'master' into 1971-show-image-thumbnails
Merge branch '2376-show-collection-tags-everywhere' into 1971-show-image-thumbnails
Added thumbnail option to #link_to_if_arvados_object, enabled on pipeline instance and job pages.
Job reuse bugfix: do not reuse completed jobs that have NULL output.
Parameterize the 'arv' command so that it can be specified with an environment
Fixed typo in application_helper.rb
Merge branch '2488-jobs-pipeline-doc'
Fixed error reporting when head/tail_uuid don't match head/tail_kind for links.
Merge branch '2376-show-collection-tags-everywhere'
Increased wait time on capybara test to reduce chance of failure.
Merge remote-tracking branch 'origin/master' into 1971-show-image-thumbnails
Thumbnail behavior now conforms to original story
Merge branch 'master' of git.clinicalfuture.com:arvados
Websocket event testing page.
Proof of concept echo server based on rack middleware that hijacks websocket connections.
Added code to properly initialize EventMachine to deal with Passenger forking.
Moved application-specific code out from rack_socket to lib/eventbus.rb
Moves API token authentication from ApplicationController into ArvadosApiToken Rack middleware.
Added permission checking to event bus so that it only report changes for
Added websocketUrl to discovery document. Workbench websockets testing page
Refactored load_filters_param and apply_where_limit_order_params into lib/
- Added object_owner_uuid to logs table, which records the owner of the object
Test helper now users ArvadosApiToken rack middleware to set current user.
Merge remote-tracking branch 'origin/master' into origin-2608-websocket-event-bus-alt2
Security bug fix to act_as_system_user to restore correct user if an
Merge branch 'master' into origin-2608-websocket-event-bus-alt2
Added teardown to all tests which resets Thread.current[:user] and friends to
Added teardown to all tests which resets Thread.current[:user] and friends to
Fixed bug whereby readable_by could return duplicate rows if there is more than
Added test runner for websocket tests. Requires passenger standalone to be
Added websocket first tests for connecting and checking API token.
Websocket tests now use database cleaner instead of ActiveRecord's
Wrapped channel callback in exception handler.
Added tests, especially for error handling behavior. Fixed bugs found by tests.
Merge remote-tracking branch 'origin/master' into origin-2608-websocket-event-bus-alt2
Resolving application_controller.rb merge conflicts, tests pass.
Bugfix. Objects returned by owned_items are now sorted by uuid in order to get
Merge branch 'master' into origin-2608-websocket-event-bus-alt2
Added ability to select which fields to return, instead of always returning all fields.
Added support for :distinct to application controller. User can now specify a
Adding more code documentation.
Now supports both websocket integrated (ARVADOS_WEBSOCKETS defined) and
Added code comments.
Unsubscribe message now takes filter definition instead of filter_id, reducing
Short-circuts #readable_by when the user is admin.
Moved message handling out to a separate method for clarity in eventbus.
Merge remote-tracking branch 'origin/master' into origin-2608-websocket-event-bus-alt2
Merge branch 'origin-2608-websocket-event-bus-alt2'
Merge branch 'master' of git.clinicalfuture.com:arvados
Fixed middleware customization so that it doesn't assume the presence or
Changed Log.all.each to Log.find_each because Log.all loads all records (!) and
Another go at migration, uses batched loading and transactions, should have
Merge branch 'master' of git.clinicalfuture.com:arvados
Added 'select' and 'distinct' parameters to discovery document. Improved
Merge remote-tracking branch 'origin/master' into 2035-arv-mount-tags-folders
Fix for interaction between 'select' and 'order' (SQL requires that order
Refactored directory handling a bit in fuse to better accomodate other virtual
Added run_test_server helper module for Python SDK that runs API server in test
Fixed botched 'distinct' parameter, now is a boolean instead of taking a
Restored support for old-style 'order' strings and added test. Clarified
Merge branch 'query-select-distinct'
Merge branch 'master' into 2035-arv-mount-tags-folders
Created a seperate arvados-fuse-driver package.
Finished implementing FuseTagsTest, fixed bugs in arv-mount, test passes.
Added websockets support to Python SDK, with integration test. Also built out
Refactored freshness/updating to be more generic for different virtual
Merge branch 'master' into 2035-arv-mount-tags-folders
Merge branch 'master' into origin-2035-arv-mount-tags-folders
Added PyYAML to requirements.txt
Fix so that a present but empty select parameter in queries no longer yields
Merge branch 'master' into origin-2035-arv-mount-tags-folders
Adding support for groups and viewing arvados objects as files (serialized
Improved fresh/stale handling with base class, added property fuse inode cache
Added test for group mount. Passes.
Added empty sharing popup.
Almost working sharing button.
Fixed incorrectly created scopes. Fixed link to use reader_tokens[] instead of
Eliminated build.sh and sed in favor of a common setup_header.py script which
Merge branch 'master' into origin-2035-arv-mount-tags-folders
Merge branch '2035-arv-mount-tags-folders' into origin-2035-arv-mount-tags-folders
Merge branch 'origin-2035-arv-mount-tags-folders'
Fixed scope to use request.request_method (the effective request method in
Adding missing file.
2798: Initial commit of Go Keep client library.
- Added MANIFEST.in to include setup_header.py in the source package. Now gets
Merge branch 'master' of git.clinicalfuture.com:arvados
Fixed stop_keep(). Updated test_mount because the fixture changed (again).
2798: Initial commit, supports querying API server for list of Keep disks.
2798: Turns list of keep disk objects into sorted slice of http URLs for keep servers
2798: Added comments to Python shuffled_service_roots and ported function to Go
2798: Started adding tests that ShuffledServiceRoots behavior is consistent
2798: Fixed inconsistencies with Python implementation of ShuffledServiceRoots.
2798: Working on Transfer function, which will be the core function for
2798: Work in progress connecting data read from input Reader to POST requests.
2798: Merged branch with code to read environment variables with branch working on Put support.
2798: Checkpoint commit, tests for ReadIntoBuffer() and Transfer() pass.
2798: Added uploadToKeepServer() test
2798: Read requests from Transfer() now return a slice. Added BufferReader
2798: All PUT tests against stub server pass. Added explict setting of
Added block GET, and tests.
2798: Full integration test with real API server and Keep server succeeds.
Merge branch 'master' into 2798-go-keep-client
2751: Added code to recognize keep proxies in the keep_disks() response and set
1885: First commit of most complete keep proxy, no tests yet. Removed build.sh
1885: Renamed DiscoverKeepDisks to DiscoverKeepServers, moved error.New()
Moved non-keep-specific buffering code into a separate package.
2798: Added AuthorizedGet(), Ask() and AuthorizedAsk(). Added BLOCKSIZE
2798: Completed move of Transfer() related code out to 'buffer' package.
2798: Continued refactoring buffer reader into separate buffer package. Made
2798: Finished writing documentation comment for buffer package.
2776: Started adding KeepServices model.
2776: Added keep_services model and migration.
2776: Added controller and route for keep_services, fixed tests.
2776: Added Keep services page to workbench (uses default layouts)
2776: Fixed backwards-compatible inclusion of service_host/port/ssl_flag for keep_disks.
2776: Fixed run_test_server.py to work with new 'keep_services' model
2776: Added stub API documentation for keep_services table.
2776: Added 'accessable' route to 'keep_services'
2776: Added test for keep_services/accessable route.
2798: Renamed 'buffer' package to 'streamer'
2798: Renamed TransferBuffer->AsyncStream, BufferReader->StreamReader,
mend
2798: Tracks opening and closing of readers, will block closing AsyncStream
2798: Renamed internal messaging structs in an attempt to use the word "reader"
2798: Updated keep client with buffer/streamer changes.
2751: Added support for keep_services table, can still use keep_disks as a fallback.
2751: Added support for keep_services table, can still use keep_disks as a fallback.
2751: merge because git derp.
2776: Fixed misspelled accessable->accessible. Thanks Brett! Tweaked
Merge branch 'master' into 2776-keep-services-table
Merge branch 'master' into 2751-python-sdk-keep-proxy-support
Merge branch '2776-keep-services-table' closes #2776
2764: Fixed scopes_allow_request to use request.request_method (the effective
2764: Fixed active_all_collections scoped token to be able to access
2798: Adds client side support for Keep proxy X-Keep-Desired-Replicas and
Merge branch 'master' into 2798-go-keep-client
Merge branch '2798-go-keep-client' into 1885-keep-proxy
1885: Stubbed out integration test for proxy
Changed API server to use X-External-Client instead of
1885: Integration test of proxy work in progress.
Merge remote-tracking branch 'origin/master' into 1885-keep-proxy
1885: Full-stack integration test (api+keep+keepproxy+keepclient) works!
1885: GET and HEAD through the proxy work correctly now. Added invalid API token test.
Merge branch '2798-go-keep-client' closes #2798
Merge branch 'master' into 1885-keep-proxy
1885: Made ServiceRoots atomically updatable, so that KeepProxy can poll for
Merge remote-tracking branch 'origin/master' into 2044-share-button
1885: Fix bug in not returning from the handler on failed authorization.
Merge branch 'master' into 1885-keep-proxy
2044: Sharing link now shown in "Sharing and permissions" panel, no longer uses
1885: Added logging of invalid requests. Added logging when the server list is
Merge branch '1885-keep-proxy' into 2751-python-sdk-keep-proxy-support
2751: Teaching run_test_server how to run the proxy server for testing. Added -pid option to keep server.
Merge branch '2798-go-keep-client' into 1885-keep-proxy refs #1885
1885: Stubbed out integration test for proxy
1885: Integration test of proxy work in progress.
Merge remote-tracking branch 'origin/master' into 1885-keep-proxy refs #1885
1885: Full-stack integration test (api+keep+keepproxy+keepclient) works!
1885: GET and HEAD through the proxy work correctly now. Added invalid API token test.
Merge branch 'master' into 1885-keep-proxy refs #1885
1885: Made ServiceRoots atomically updatable, so that KeepProxy can poll for
1885: Fix bug in not returning from the handler on failed authorization.
Merge branch 'master' into 1885-keep-proxy refs #1885
1885: Added logging of invalid requests. Added logging when the server list is
Merge branch '1885-keep-proxy' closes #1885
Merge branch 'master' into 2751-python-sdk-keep-proxy-support refs #2751
Merge branch 'master' into 2044-share-button refs #2044
1885: Tests and bugfixes for -no-get and -no-put options.
Merge branch 'master' of git.curoverse.com:arvados refs #1885
2751: Changed to "except:" to "except Exception:". Refactored environment setup in keep_client_Test to be a bit more robust.
2751: Test tweak to clear settings after changing environment variables.
Merge branch '2751-python-sdk-keep-proxy-support' closes #2751
Merge branch 'master' of git.curoverse.com:arvados refs #2751
Merge branch 'master' of git.curoverse.com:arvados #2751
Adjusted timeout on websocket_test from 3 seconds to 4 in an attempt to reduce
Merge branch 'master' of git.curoverse.com:arvados
Merge branch 'master' into 2044-share-button
2044: Fixed CollectionsController#search_scopes to use 'filter' correctly
Go Keep client correctly closes response body on client requests, should fix
Merge branch '2044-share-button' refs #2766
Added rescue AccessForbiddenException to collections controller and view to
Added proper sharing scope to permit actually downloading shared files. refs #2766
Another fix to sharing scope to use keep_services/accessible. refs #2766
Pass PATH, PYTHONPATH and PERLLIB environment from crunch-dispatcher to
2939: Added a status page for jobs that is similar to the pipeline instance
2939: added no_reuse, nondeterministic
Merge remote-tracking branch 'refs/remotes/origin/2939-create-params' into origin-2939-re-run-job-button
2939: Added generic "options" parameter to #create to allow workbench pages to
2939: Added repository to re-run with latest button label
Merge remote-tracking branch 'origin/master' into origin-2939-re-run-job-button refs #2939
2939: Use JSON.dump instead of Oj.dump, solves the symbol keys problem.
2919: Reinstated absolute positioning, changed font size so that text is no
Merge branch 'master' into 2882-job-process-stats refs #2882
2882: arv-mount --exec forwards SIGINT and SIGTERM to the underlying subprocess.
2882: Added button to cancel a running job.
2882: crunch-dispatch now sends a clean environment to crunch-job.
2882: Added crunchstat service to collect cgroup stats, and added support to
Merge remote-tracking branch 'origin/master' into origin-2939-re-run-job-button refs #2939
2882: Explicit polling frequency
2955: Periodically compare Job.running to @running and fail any jobs not in
2955: Fixed refresh_running to set finished_at on failed jobs. Increased
2955: Added check that an orphan job hasn't produces any logs for 5 minutes
2955: crunch-dispatch now sends a clean environment to crunch-job. (cherry-picked from #2882)
2955: Fixed query of log table that checks for age of last log by an orphan
2882: arv-mount restores signal handlers after sp.wait(). Fixed spelling ellapsed -> elapsed.
Merge branch 'origin-2939-re-run-job-button' into 2882-job-process-stats
Merge branch 'master' into 2882-job-process-stats refs #2882
2882: Improved re-run job buttons on job status page
Merge remote-tracking branch 'origin/master' into 2882-job-process-stats
2955: Sets jobrecord.canceled_at when canceling/failing an orphan job.
2882: Removed unnecessary no_reuse options
Proposed new git hook policy: only applies to first-parent, enforces master is
Added pattern to permit commits that merge master heads.
Disallow commits that merge local and remote master, in order to maintain
Added mention of git reset to instructions for remerging/rebasing.
2955: Moved logic to clean jobs table from crunch-dispatch into a standalone
2961: Added tab_pane query parameter which returns just the desired tab pane partial.
2955: Renamed script to "cancel_stale_jobs". Fixed spelling canceled_at -> cancelled_at.
Merge branch '2882-job-process-stats' refs #2882
Fixes PipelineInstancesControllerTest. application_controller#create now does
2961: Removed spurious elementId from subscribeToEventLog(). Added arv-log-event-listener[data-object-type="xxxxx"] to
2961: Now loads tab panes (past the first one) in the background.
Reverted change a0aba6d "crunch-dispatch now sends a clean environment to crunch-job" refs #2882.
Merge branch 'master' into 2955-fail-orphan-jobs
Reverted change a0aba6d "crunch-dispatch now sends a clean environment to crunch-job"
Merge branch '2955-fail-orphan-jobs' closes #2955
2961: Added checks if we are rendering a tab partial and skips expensive calls
2961: Save user information as part of the session cookie, eliminates looking
Merge remote-tracking branch 'origin/master' into 2961-load-tab-partials
2961: Collections show controller now delegates to super to handle tabs
2961: Escaped erb used embedded in javascript. Updated comments in
Specify server-side keep-alive ping interval for websockets. no issue #
2961: Changed index methods on some controllers to use render_index instead of
2961: Job index needs to call super sometimes, render_index other times.
Fix for running workbench against server that doesn't have 'Job.supplied_script_version' yet. refs #2939
Merge branch 'master' into 2961-load-tab-partials
2961: Added missing user uuid to session.
2961: Removed dangling close div.
2961: Added use count so ajax loading icon is accurate when there are multiple
2961: Track valid/up to date state for each panel, only reload panels when they
2961: Fixed name of collection on collection show page to fix test.
2961: Do not include meta tag for websockets if there is no websockets in the
2961: Don't call #render_index from #index on folders_controller because #index
2961: user-setup-modal-window div is no longer in footer_html because it is
2961: Removed calls to wait_for_ajax, fixes user tests.
Merge branch '2826-simple-go-sdk' closes #2826
Merge branch '2961-load-tab-partials' closes #2961
Added error-handler for x-editable that extracts Arvados API error field for
2883: Initial work on fetching and parsing job log
2883: Log viewer works, supports sorting and filtering.
2883: Moved hardcoded styles out into a stylesheet.
2883: Styling column widths so they don't jump around when filters are turned
2883: Live updating log wip
2883: Removed commented out code related to live loading (saved it to another branch).
2882: Search subdirectories of cgroup for stats, because compute0 has a
Merge branch 'origin-2882-job-process-stats' refs #2882
Made a note about user how permissions are applied when attempting to create a
Runs crunch script using "stdbuf -o0 -e0" so stdout and stderr are unbuffered,
Crunchstat logs where it reads its stats from. refs #2882
Bugfix searching for cgroup stats files. refs #2882
Bugfix in logging which cgroup stats files it uses. refs #2882
Expanded search path for cgroup stats, changed command line interface a bit.
2883: Moved addToLogViewer function out of the template and into assets and
Merge branch 'origin-2883-job-log-viewer' closes #2883
Fix fuse driver tests broken by changes to test fixtures. no issue #
2883: Added radio buttons to filter tasks by success/failure.
some refactoring and added detection of task success or failure
Merge remote-tracking branch 'origin/origin-2883-job-log-viewer' into 2883-job-log-viewer
2883: Fixes secondary key sort bug. Adds filtering on successful/failed tasks.
2883: Fix pattern for discovering success/failure of tasks.
2883: Added overview, node column and sort by node, style tweaks.
2883: Added node slots as its own column.
2883: Distinguish between complete/incomplete and success/failure.
2883: Added big red "show failed tasks" button.
2883: Added pagination for large logs.
2883: Improved pagination control a bit, added search.
Ajax load of tab panels now includes any query parameters the page was
Fixed bug in Keep proxy when talking to Keep server that doesn't report
2884: AJAX load a preview panel in collection and pipeline template picker.
2884: Can now choose a specific folder to browse in the file selector modal.
2884: Selecting folders works for run pipeline dialog. Increased page size
2884: Fixed horizontal scroll in folder selection dialog
2884: Added spinner to indicate infinite scroll is loading.
2884: Adjusted column size on components page to work better when used in
Fixed smart scrolling as intended for svg divs. refs #2919
Merge branch 'master' into origin-2883-job-log-viewer
2883: Use smart scrolling so keep the sorting and filtering selections on the
Add error handling when log collection is not found. refs #3027.
Merge branch '2883-job-log-viewer' of git.curoverse.com:arvados into origin-2883-job-log-viewer
Merge branch 'origin-2883-job-log-viewer' closes #2883 refs #3027
2986: Cleaned up arv cli help to be more helpful and consistent. refs #1667.
2986: Make arv-run-pipeline-instance be nicer to the user when run without
2986: Implemented "arv edit" command. Refactored 'arv' to be easier to follow.
2884: Added fail callback preview loading fails. Added comments explaining a
Merge remote-tracking branch 'origin/master' into 2884-improved-picker
Merge branch '2884-improved-picker' closes #2884
Fix pipeline instance creation workbench tests. refs #2884
2986: Can now specify which fields you want to edit, because fields like 'uuid'
2986: Writes tempfile with appropriate extension so editor can apply proper editing mode.
2986: Report parsing errors and print out contents instead of just blowing up.
2986: Restored "arv pipeline run" subcommand. Arv edit will now ask you if you
2986: Handles collection ids in 'arv edit' (tells the user collections are not
Merge branch '2986-arv-edit' closes #2986
Suppress "Resource or subcommand '' is not recognized." message when running
Fix symlinks in sdk/cli/bin for arv-mount and arv-keepdocker. no issue #
Adds "--strip" option to arv-normalize to strip off authorization tokens.
Fixed cancel_stale_jobs to authorize as system user before modifying jobs
Fix get_n_objects_of only accepts ArvadosBase classes, not any class. no issue #
Only attempt to preload ArvadosBase objects in render_pipeline_component_attribute. no issue #
Now includes runtime_constraints from the component when creating a new job. no issue #
Add support for ARVADOS_API_HOST_INSECURE environment variable. Use Net::SSL
Remove spurious print from fuse driver. no issue #
MagicDirectory.__contains__ now creates CollectionDirectory (which creates
Make task working directory world writable, to deal with uid mapping problems
Merge remote-tracking branch 'origin/master' into pete-fixes
Don't print blkio stats if no reads or writes have occurred since the last poll.
Improve arv edit failure handling.
Reused variable 'n' for two different things. Oops.
Refactored arv-normalize calls into function.
Refactored "save temp file rescue behavior into function
Use /tmp/crunch-src for docker jobs and $ENV{CRUNCH_SRC} for non-docker. Removed umask.
Add -> indirection to $self->{'noVerifyHostname'}
Fixed regex for extracting just hash and size part of locator
Remove spurious 'puts'
Use logging module instead of "if DEBUG:"
fuse test fixes
"Combine selected" actually works again.
Fixed logging statements to format correctly.
Merge remote-tracking branch 'origin/master' into pete-fixes
:none dispatcher only runs one job at a time.
Remove spurious 'puts'
Remove import from __future__
Fixed get_n_objects_of_class datatype assertion for 'dataclass'
Merge branch 'pete-fixes' of git.curoverse.com:arvados into pete-fixes
Fixed render_pipeline_component_attribute test if dataclass derives from ArvadosBase.
Merge branch 'pete-fixes' closes #3096
Update required arvados gem to depend on version of arv-run-pipeline-instance
Handle zero-length streams that contain zero-length files. refs #3084
Handle empty directories in arv-put. refs #3087
Fix tests broken by adding zero-length block manifests containing a zero length
Add 'run-command' generic crunch command wrapper. refs #2342
Fix 'arv' to handle missing/non-writable $HOME. no issue #
Set $HOME to $TASK_WORK for tasks. no issue #
Set $HOME to /tmp/crunch-job for docker.
Merge branch 'arv-crunch-HOME-envvar' no issue #
Update required arvados-cli package to get fix 55e1aaf (Fix 'arv' to handle
Add dockerfile to build arvados/jobs-bwa-samtools image. no issue #
Always initialize arvados.api() so it doesn't silently fail when
Make sure set return code non-zero.
Change crunch-dispatch to use "git fetch-pack --all" insted of "git fetch" to
Set success/failed flag explicitly based on return code.
Improve error messages
more status messages
More clever find
fix find
Dockerfiles for java-bwa-samtools, bcbio-nextgen
Add explicit version constraint < 1.0.0 on 'jwt' gem for arvados-cli, because
Merge branch 'master' into job-docker-images
Change crunch-dispatch to use "git fetch-pack --all" insted of "git fetch" to
Use 'install' to create /keep, /tmp/crunch-src, /tmp/crunch-job mountpoints in
Use separate $HOME for commands run outside the docker
Should probably wait to install ruby before setting GEM_HOME and friends.
Typo fix ${HOME} -> $ENV{HOME}
Do not mount $TASK_WORK inside the container; scratch space is entirely
Improve output messages a bit
First try at bcbio-nextgen script
Fixed +x
fixed syntax error
fix config copy
open for writing
write template
refer template directly
fix sub R1 R2
tool-data in work
Backed that out, make a symlink to tool-data instead
fix bwa_index
Docker job's $HOME is now inside docker container, we don't need a special
"filters" is now propagated through from pipeline component to the job
Merge remote-tracking branch 'origin/master' into job-docker-images
Update arvados-cli dependency of API server so crunch-dispatch.rb uses
job create method parameters need to be converted to json before being passed
Added note about how to build bcbio-nextgen docker image. Make
Update api server Gemfile dependency on arvados-cli package so crunch-dispatch
Don't set "too long" flag on websocket test timer if the websocket is already
Don't raise exception if filters parameter parses to
Don't send null job creation parameters in arv-run-pipeline-instance. no issue #
Update api server Gemfile dependency on arvados-cli package so crunch-dispatch uses commit:66efe78 no issue #
Merge branch '2342-run-command-wrapper' refs #2342
Merge branch '2991-job-docker-images' refs #2991
Update api server Gemfile dependency on arvados-cli package so crunch-dispatch
Merge branch '3185-job-invalid-without-errors' refs #3185
run-command retry after exception when writing output to keep. no issue #
Add $(node.cores) substitution to run-program, and add CRUNCH_NODE_SLOTS to
Add -n cores to run bcbio in parallel on a multicore node.
Merge branch 'bcbio-nextgen' refs #2991
Bugfix os.environ() -> os.environ[] no issue #
arvados-bcbio-nextgen writes output now. Put write_directory_tree under the
Default arvados-bcbio-nextgen pipeline to use freebayes instead of gatk. no issue #
Consider script_parameters in determining whether to merge script boxes in
Don't render selection check boxes when using show_file partials from chooser
arv-run-pipeline-instance sends error message to logs table when job submission
Update package version dependencies on arvados-cli to use commit:7bd2ee2
Google API client jsonifies the body, so we don't have to.
Merge branch '3123-pipeline-report-job-submit-errors' closes #3123
Update arvados-cli dependency to correct deployed version. no issue #
Don't use "arv" command wrapper, call arv-get/arv-put directly. Avoids
add 'import sys' to arvados-bcbio-nextgen. no issue #
bc-bio script links gemini reference data. refs #2991
missing paren typo in arvados-bcbio-nextgen.py. refs #2991
Keepproxy use client-supplied token when forwarding GET and PUT requests.
Merge branch '3206-keepproxy-use-client-token' closes #3206
Use resumable upload writing output in run-command
Forward signals to process.
More work on signal handling
Record signals a different way
Fix wrong number of parameters
Still messing with signals.
SigHandler object
Wrong num args
Add debugging
Turns out sys.exit runs finally: blocks.
Actually use the resume cache
Merge branch 'resumable-output-upload'. Adds signal handling and error recovery
3232: Reorganized user guide
Set $CRUNCH_SRC correctly when running the Docker container. refs #3370
Described with how to use "arv edit" in "Writing a pipeline". Switched order of paragraphs in "introduction to crunch". Grammar and wording tweaks.
Fixed one character typo "ghttp" -> "http" caught by linkchecker.
Full split-fastq wip
Don't split fastq, just organize into directories.
Added "foreach" to run-command wrapper. Needs testing.
foreach just refers to other parameter to use as iterator variable.
typo fix get_job_param() -> getjobparam()
Input is "reads" not "input"
Same bug, different place
Left debugging in by accident
debug print manifest_text
Debug manifest format to see why api server is rejecting it
Change collection create request
Simpler split-fastq, doesn't try to upload the collection, assume crunch-job
Refactored and generalized list expansion, can use "foreach" to repeat portions
Fix syntax error
import stat
.st_mode
import copy
Set first task to success
Print traceback on exception
More logging
Rearranged exception handling a little more
Pretty print
Better error from glob
extra / after dir
str or unicode
Fix $(node.cores)
Combined contents of "Setting up SSH" into "Virtual Machines" panel. Added
Use :body_object instead of :body
Added special case to ArvadosModel#readable_by so that a name link going from a
Add check to JobsController#create that collection output of a job is readable
Updates in response to Radhika's comments.
Changed button label to "Add to project..." to match button introduced in #3452.
Merge branch 'master' into 3232-user-guide-updates
Fixed record symbolic name, comment based on feedback.
Merge branch 'origin-3442-no-reuse-job-when-output-unreadable' refs #3442
Merge branch 'origin/3443-read-collections-from-name-links' refs #3443
Fix for test_mount.FuseGroupsTest to reflect test fixture change. refs #3443
Added $(task.outdir) and $(job.src)
Fixed typo introduced by merge. refs #3443
Fixed "Tutorial align using bwa mem and SortSam" template. Tweaked
Fix bug in StreamFileReader incorrectly updating self._filepos resulting in
Decompress-all wip
fix parameters
Works now
Simplified collection-merge. Added comments to collection-merge and decompress-all.
Merge branch '3373-improve-gatk3-snv-pipeline' of git.qr1hi.arvadosapi.com:peter into 3373-improve-gatk3-snv-pipeline
Merge branch 'master' into 3373-improve-gatk3-snv-pipeline
$(link ...) now expects complete path.
Added explicit instructions on sample data to use for tutorial pipeline.
Merge branch '3232-user-guide-updates' closes #3232
Fixed link to installing python sdk page. refs #3232
Renamed "job.src" to "job.srcdir", "job.id" to "job.uuid", and "task.id" to "task.uuid"
Use rvm-exec to run gem to install Arvados cli. Also install dtrx.
Working on split-fastq to actually split the fastq file.
Check for None
Fix syntax error
import string
arvados.LOCATOR
Chunking fastq seems to work, but is very slow.
Disable chunking
Merge branch 'master' into 3373-improve-gatk3-snv-pipeline
Add documentation to PipelineTemplate schema for title, description, link_name of pipeline template script_parameters.
Merge branch '3372-update-pipeline-template-schema-docs' refs #3372
"Writing a crunch script" now shows how to run locally, "Running on an Arvados cluster" shows how to commit to git and create a pipeline template.
Merge branch 'master' into 3373-improve-gatk3-snv-pipeline
3373: decompress-all: cleaned up code a bit. Fail properly with error message on bad input.
3373: split-fastq: merge loops to capture both single and paired fastq files.
3373: Switch run-command to use python logging instead of print.
3373: arv-run-pipeline-instance: removed .to_json where objects are supplied to :body_object and will be serialized anyway.
3373: more run-command cleanup, global p -> taskp, tweaked logging levels
3373: Fix syntax error in split-fastq
3373: Fix wrong regex group
3373: split-fastq: Fix regex again
3373: split-fastq: Fix loop
3254: Typo fix "reproducibilty" -> "reproducibility". Changed use of the term "parallel" to "concurrent" refs #3556
Merge branch '3254-run-job-locally-docs' closes #3254 refs #3556
Fix broken links. refs #3254
Fix broken link again. refs #3254
3036: Work on migration code for 'name' and 'can_read' links to distinct collection entries.
Delete obsolete, undesirable HTML page templates.
3036: More work on migration code. Now updates links table. SQL syntax errors fixed.
3373: Test filename extension before attempting to call dtrx. Non-zero dtrx exit code is now treated as failure.
Merge branch '3373-improve-gatk3-snv-pipeline' closes #3373
Fix syntax error in decompress-all.py refs #3373
Fix regular expression in decompress-all.py refs #3373
3036: More work verifying that migration does the right thing.
Added virtual working directory "task.vwd" option to run-command. Added
Fix degenerate case exponential runtime in regular expression use by
3505: Adjust order of initialization to reflect potential side effects (e.g. file
3505: Fixed typo comparison of list against 0 instead of len(list).
3036: More work on migration. Converts collection->collection provenance links into jobs. Handles orphan collections;
3505: Move helper scripts into crunchutil module. In run-command, added
3505: Fix typo in run-command
3505: tinker with logging
Merge branch 'master' into 3505-virtual-work-dir
3036: Manually migrated collections and name links on test fixtures.
3036: Removed "add can_read link after creating collection record" logic.
3036: Removed special case uuid_like_pattern. Removed special case in
Merge branch '3505-virtual-work-dir' refs #3505
3036: Removed special case find_object_by_uuid. Converted some queries that
3036: Removed special case normalize_collection_uuids. Removed special case
3036: Split out logic to determine if a user is actually allowed to set the
3036: Remove obsolete 'name' link tests.
3036: Fixing uuid -> portable_data_hash in collections controller tests.
3036: ensure_owner_uuid_is_permitted checks that owner_uuid is a User or Group.
3036: Adding pre-validation callback to set the portable_data_hash if necessay.
3036: Test fixing work in progress.
3036: Remove include_linked from project contents method because 'name' links
3036: Restored normalize_collection_uuids removed in 22457e1
3441: Change logging level of nonfatal Keep errors to "debug".
3036: API server tests pass
3036: Migration adding (owner_uuid, name) uniqueness constraints to
3036: Migration will rename rows if necessary to ensure (owner_uuid, name) uniqueness
Merge branch '3441-python-keep-log-level' refs #3441
3036: API server unit tests pass again after adding uniqueness constraints.
3036: Ruby style change, use :search_up and :search_down symbols instead of UP and DOWN constants.
3504: Fix collection page rendering to work with collections as regular uuids.
3504: Fix collection chooser. Fix generic link render to use uuid if nothing
Merge branch 'master' into 3504-clients-compatible-with-3036
3036: Updated migration to set modified_at instead of updated_at, added modified_by.
Merge branch 'master' into 3036-collection-uuids
3036: Fixed migration to migrate modified_by_client_uuid and
3036: Fix merge mistakes in collection_controller.
3036: Fix #updating collection objects. Fixed database seed for empty_collection.
3036: Fix arv-put to write name/owner of collections object directly when
Unset ARVADOS_ variable from the environment before running tests. no issue #
3036: Commented out failing FUSE tests. Fixed uuids on fixtures to use zzzzz
3036: Fix arv-put to write name/owner of collections object directly when
3504: CollectionReader accepts arvados object collection uuids. 'arv-put' sets
3504: Use preload_links_for_objects in projects_controller. Use
Permit setting environment variables on command line. Added workbench_test,
3504: Fixed preload_links_for_objects to use @objects.to_a. Fix collections
Merge remote-tracking branch 'origin/master' into 3504-clients-compatible-with-3036
3504: Set the capybara driver explicitly so tests pass again.
Declare empty variables up top to avoid bash parsing errors. no issue #
3504: Set owner_uuid when creating collection (will set owner_uuid of name link
Merge remote-tracking branch 'refs/remotes/origin/3504-clients-compatible-with-3036' into 3036-collection-uuids
3036: Use preload_objects to fetch owner_uuids in the controller instead of in the view.
3036: Use preload_objects to fetch owner_uuids in the controller instead of in the view.
3504: Don't set owner_uuid redundantly.
3504: Remove link to links index page on tags. Add comment to
3504: Remove link to links index page on tags. Add comment to
3504: Support copy and moving collections on new apiserver
3036: Most workbench tests pass.
3504: Back out tags on project rows. Revisit in a later story.
3504: Don't create stand-in "name" links for post #3036 apiserver
3504: Support copy and moving collections on new apiserver
3504: Don't create stand-in "name" links for post #3036 apiserver
3036: Always select "id" column unless distinct = true. Include uuid =
Merge remote-tracking branch 'remotes/origin/3504-clients-compatible-with-3036' into 3036-collection-uuids
Merge branch '3504-clients-compatible-with-3036' refs #3504
3036: Move manifest_text validation into Collection model. Change
Merge branch 'master' into 3036-collection-uuids
Display first file in collection if name field is nil or empty. refs #3504
Change 'expire_time' column to 'expires_at' for consistency with other date
Display first file in collection if name field is nil or empty. refs #3504
Improve error checking a bit on find_commit_range for bad revisions and missing
Reject empty fields in job creation filters. no issue #
Merge branch 'master' into 3036-collection-uuids
Accomodate otherwise valid user-supplied portable data hashes with no size
With new apiserver, after upload, arv-put prints portable data hash instead of
3136: Fetch actual job objects for use in rendering pipeline jobs.
Merge branch 'master' into 3036-collection-uuids
3036: Fix merge conflicts. One python sdk test still fails.
Enhanced run-tests.sh command line supporting choosing specific Python and Ruby
Merge branch 'master' into 3036-collection-uuids
Merge branch '3036-collection-uuids' closes #3036
3136: Use map to correctly get the list of uuids. Check that there is a job
Merge branch '3136-stale-pipeline-instance-display' closes #3136
Fixed missing import in robust_put. no issue #
3342: Migration to remove obsolete output_is_persistent column from jobs.
3342: Removing remaining traces of output_is_persistent. Added code in
3342: Fixing outputs
3342: Fix links to log files to use portable_data_hash. Remove spurious puts.
3342: Tweak pipeline component status rendering. Support "output_name":false
3342: Update documentation. Add "output_name", remove "output_is_persistent".
3342: Handle case in PipelineInstance#friendly_link_name where
3342: Fix syntax error in pipeline_instance.rb. Restore code in crunch-job
Merge branch '3342-pipeline-output-current-project' closes #3342
Set Gemfile version target for arvados-cli to correct version. refs #3342
3719: Raise more specific SubstitutionError instead of generic exception on
Remove assertion testing for obsolete output_is_persistent field. refs #3342
3719: run-command uses its own logger instead of the default Python logger to
3719: Robust-put uses run-command logger.
3586: Set job priority column default to '0', add NOT NULL constraint.
3586: Set priority to "0" if nil. Fix test fixtures.
3586: Migrate "priority" column so it is an integer instead of as string.
3709: arv-put now prints collection uuid by default. Add --portable-data-hash
Merge branch 'master' into 3709-arv-put-print-uuid
Fix nil dereference causing a fiddlesticks on the collections index page. no issue #
3644: Switched around command line parameters a bit. Will default to --home
Merge branch '3709-arv-put-print-uuid' closes #3709
3719: Python style/wording fixes.
Merge branch '3719-improve-subst-error-reporting' closes #3719
3644: More renaming GroupDirectory to ProjectDirectory, removing name links.
Fix $leave_temp in clear_temp() no issue #
Fix arv-put tests refs #3709
3644: Working on adding timestamps
3763: Workbench will not send attributes to API server that are nil and not
Merge branch '3763-move-collections-error' closes #3763
3644: Project and Home directory work. Added support for returning correct
3644: build_project_trees method ported from workbench, needs testing
3453: Add support for --images to get a list of available images. Add support
Delete names and description columns from jobs that shouldn't be there. Delete
Merge branch 'master' into 3586-job-priority
Add name and description columns back in for jobs. refs #3036
3644: Shared project list almost works, still debugging.
3644: HomeDirectory and ProjectDirectories work, added filename sanitization.
3644: Collections work again, added .portable_data_hash and .manifest_text
3644: Re-added support for files with json contents of arvados objects.
3644: Default mount mode now includes home, shared, by_hash, and by_tag.
3644: Tested, fixed various mount modes.
3644: Tweak CollectionReader manifest text regular expression to use multiline
3644: Add threadsafe arvados api access. arv-mount now releases llfuse global
3644: Change metafiles to .arvados#collection and .arvados#project. Change
3644: Added choose-your-own-adventure README files to the --all and --by-id
3644: Add --by-id to invocation of arv-mount in crunch-job to get the same
Fix environment variable setting on command line. no issue #
Remove name and description columns on jobs table introduced accidentally
3644: fuse_arvados test pass.
3710: Refactored code to reduce redundancy. Use select() to get the manifest
Merge branch '3710-combine-selections-bug' refs #3710
3453: Now lists images by default if called with no parameters. Fixed reported
3644: Changed so ctime == mtime because ctime is not creation time (arg!).
3644: minor control block indentation fix.
3822: Added 'ensure_unique_name' option to #create method for API server to
3822: Fixed use of string instead of symbol in hash, added missing list separator.
3822: Updated Gemfile
Merge branch '3822-ensure-unique-names' refs #3822
Updated Gemfile again with published package version. refs #3822
3453: Create links and collection objects in the specified project, even if
Added create#ensure_unique_name to discovery document. "Add a subproject"
Merge branch 'master' into 3453-arv-list-docker-images
3453: Add ensure_unique_name=True to collection create.
3644: Restore 'tag' tests and add 'SharedDirectory' and 'HomeDirectory' tests.
Merge branch 'master' into 3644-arv-mount-projects
3644: Moved some common regex patterns into SDK
3644: Update tests broken by merge
3453: Refactored arv-put to remove support for name links, correctly use
Merge branch '3644-arv-mount-projects' closes #3644
Merge branch 'master' into 3586-job-priority closes #3586
Merge branch '3586-job-priority' closes #3586
3453: Rename check_project_exists to desired_project_uuid. Now raises and
Merge branch '3453-arv-list-docker-images' closes #3453, #3472, refs #3125
3187: Migration to add started_at and finished_at to job_tasks and pipeline_instances.
3187: Record started_at and finished_at for tasks and pipelines.
3187: Added queue_position field to job response.
3187: Lots of work on presentation of job details, calculating time and
3187: Added algorithm to calculate wall clock run time from a set of overlapping ranges (i.e. concurrent jobs).
Merge branch 'master' into 3187-start-finish-timestamps-tasks-pipelines
3187: Tested and fixed and time calculation algorithm.
3187: Generalized reloading to reload other parts of the page in addition to
3187: Update the task start time when the task actually starts.
Merge branch '3187-start-finish-timestamps-tasks-pipelines' into 3187-pipeline-instance-page
3187: Setting dates to locale works.
3187: Text tweaks.
Fixed typo Y% -> %Y
Merge branch '3187-start-finish-timestamps-tasks-pipelines' refs #3187
3605: Provide "info" field to all requesters. Filters out any keys with text
3605: Dashboard compute node and pipeline status work in progress
Merge branch 'master' into 3605-improved-dashboard
Working on recent pipelines/collections tables
Merge branch '3187-pipeline-instance-page' into 3605-improved-dashboard
Remove projects panels from dashboard
Updated Gemfile to pull in changes from commit:ac594f1 refs #3187
3605: Finally settled on a layout. Adding in remaining information and fixing bugs.
Merge remote-tracking branch 'origin/master' into 3605-improved-dashboard
3605: Getting jobs/queue_size from API server works. Fixed server-side
3878: Fix block cache sharing across threads so arv-mount uses a bounded amount
3605: Automatically reload the dashboard using ajax.
3878: Remove raise debugging exception.
Merge branch '3878-arv-mount-block-cache' closes #3878
Fix test
Fix arv-run-pipeline-instance to not crash when template is null. no issue #
Fix test
Merge branch '3605-node-info-field' refs #3605
Merge branch 'master' into 3605-improved-dashboard
3605: Add "queue_position" to jobs. Also add a route on jobs controller to get
3187: Fix tests and some debug spew.
Merge branch '3187-pipeline-instance-page' into 3605-improved-dashboard
3605: Test queue_size method and queue_position field. Adjust "GET queue"
Merge remote-tracking branch 'origin/master' into 3605-improved-dashboard
3605: Add properties column to node. Put public-facing node information (cpu
3605: Do .to_i on total_cpu_cores, total_ram_mb, and total_scratch_mb in Node#ping.
Merge branch '3605-node-info-field' refs #3605
3605: Use #each_with_index instead of #each_index
Merge branch '3605-queue-position-size' refs #3605
Merge branch 'master' into 3605-improved-dashboard
3605: Workbench tests pass!
Merge branch 'master' into 3187-pipeline-instance-page
Fix Canceled -> Cancelled (american vs canadian spelling, canada wins)
Merge branch '3187-pipeline-instance-page' into 3605-improved-dashboard
Merge remote-tracking branch 'origin/master' into 3605-improved-dashboard
3605: Fix tab_pane rendering. Remove spurious console.log.
3605: Fix tab param['tab_pane'] merge conflicts causing tests to fail.
Merge branch '3605-improved-dashboard' refs #3605
3605: Additional layout fixes. Better handling of pipelines where started_at and finished_at is missing or bogus.
3605: Change runtime() to render_runtime(). Improve rendering when started_at
3605: More improvements to overflow rendering. More fixes to rendering when
3605: Render individual job status using the same partial as the pipeline
3605: Make compute node summary table no longer clickable since it was a little
3605: fix test text
3605: Check that a pipeline component entry is a hash before trying to render it.
3605: Fallback to 0 (instead of nil) when tasks is empty
3605: Filter out nodes that have a last ping time of more than 1 hour. Add
3605: Rename "default_show" to "title_and_buttons". Revert capitalization
Merge remote-tracking branch 'origin/master' into 3605-improved-dashboard
3605: Remove total nodes from compute summary table.
Merge branch '3605-improved-dashboard' refs #3605
Bugfix extra space in display in render_runtime and queued time for pipeline.
Bugfix time display, also bugfix for job state "Completed" -> "Complete". refs #3187
3988: Update crunch-job, arv-run-pipeline-instance and workbench to read/write
3988: look at job state instead of cancelled_at to determine if the job should
3899: Fix syntax errors in arv-run-pipeline-instance, and update job record if
Merge remote-tracking branch 'origin/master' into 3988-crunch-use-job-state
3899: crunch-dispatch uses state column to mark crashed jobs as failed.
3168: Check internal git repository if the commit already exists before calling
3988: Job.queue uses "state = Queued" instead of previous multi-column-null test.
3859: Implement Job lock method on api server. This takes a queued job and
3859: Removed validate state change (not the fight with crunch-job)
3859: Check crunch-job return code properly. Restore job state change
Setting running = false is ok for successful jobs, so take out extra exit_code == 0 check.
3899: Shorten count of ended/succeed/failed in a-r-p-i with some clever Ruby.
3988: Add note to Job.state. Replace logic to compute state based on
3899: Set :state in pipeline_instance_helper.rb and remove Job::state.
3168: Fail on git tag collision instead of putting job back into the queue.
3899: jobs_controller uses state = Queued instead of multiple state columns.
Merge remote-tracking branch 'origin/master' into 3899-crunch-use-job-state
3168: Instead of crashing using save! log an error and go to the next job.
Remove useless take() and untake() methods.
Merge branch '3168-crunch-git-fetch' closes #3168
3899: pipeline_instances_helper: Don't set :state if it's already set. Add newline to jobs.yml.
3899: Uncomment line that was commented out for debugging and committed by mistake.
Merge branch '3899-crunch-use-job-state' closes #3899
4004: Put "All jobs/pipelines/collections" buttons back onto dashboard. Also
3871: Encode/decode file and directory names to UTF-8
Merge branch 'master' into 3859-api-job-lock-method
3871: Pass on UnicodeEncodeError. Don't return those entries in the directory
Merge branch '3871-non-ascii-project-names' closes #3871
3052: capture stdout to log in crunch-dispatch
3052: Rename variable "std" to "stream"
3052: Style tweak, iterate over j[:buf] to read streams
4004: Rename "Compute status" to "Compute and job status". Added integration
3859: Cleaned up validate_state_change. Added unit test for job locking.
3959: crunch-job use job locking API
Merge branch 'master' into 3859-crunch-job-use-lock
Merge branch 'master' into 3859-api-job-lock-method
3859: Add functional test for locking and fix tests broken by validate_state_change.
3899: Fix for counting complete/failed components refs #3899
4004: Add newlines for readability
Merge branch '4004-restore-dashboard-buttons' closes #4004
3859: Added AlreadyLockedError for job lock conflict. Added runningbarbaz
3769: Allow up to crunch_limit_log_event_throttle_rate per
3859: unit test for job state changes
Use job state field instead of running/success columns. refs #3898
Merge branch 'master' into 3769-throttle-logs
Merge branch '3052-crunch-log-stdout' into 3769-throttle-logs
3769: Add rate_limit with log throttling logic. Multiple configuration
3769: Tweak throttle/truncate messages and application.default.yml defaults.
Merge branch '3859-api-job-lock-method' refs #3859
Add test fixture to fix "test_cancel_someone_else's_visible_job" refs #3859
3052: Change ruby style ruby style {:stderr => ''} to json style {stderr: ''}
Merge branch 'master' into 3052-crunch-log-stdout
Merge branch '3052-crunch-log-stdout' closes #3052
Merge branch 'master' into 3769-throttle-logs
4042: task.foreach should now accept multiple parameters and generate tasks
4042: Add support for filter and group of lists
4042: Fix syntax error
4042: list handling bugfix
4042: add "extract"
4042: Add list index
4042: fix bad syntax
4042: fix bad keys
4042: add command pattern to list index
4042: expand list fix
4042: another expand fix
4042: Start working on run-command reference page.
3769: Spelling fix and remove debugging puts.
Merge branch '3769-throttle-logs' closes #3769
4042: Getting started on run-command reference
4042: Subst will raise exceptions if files/directories don't exist.
4042: Functions defined in get_items() now moved into expand_items()
4042: First draft document complete, need to verify that examples all work.
4042: Add dry-run mode to run-command. Test and fix examples from documentation.
Merge branch 'master' into origin-3381-job-progress-bar-bug
Merge branch '3859-crunch-job-use-lock' closes #3859
3381: Merge job_status_label and job_progress into a single job_progress
4042: When listing directory, return list of absolute paths
3381: Reorganize _running_components to be clearer. Fixing workbench.
4042: Rename bad reuse of 'p' to 'match' in expand_item. Finish describing
4042: Update documentation samples. Small fix to dry-run to allow supplying custom TASK_KEEPMOUNT.
Merge branch '4042-run-command-MxN' of git.curoverse.com:arvados into 4042-run-command-MxN
Updated examples.
3381: Fix layout comment
Merge branch '3381-job-progress-bar-bug' closes #3381
4042: Typo fixes. Highlight run-command and script_parameters in text. Rename
Merge branch '4042-run-command-MxN' closes #4042
3656: Add arv-create command. Refactor run_editor to be shared by arv_edit and arv_create.
3656: Support additional create parameters on the command line, only open
3656: Documentation updated to use "arv create".
3787: Started page, ran into some problems
3656: Add missing file
3692: Fixed test, and fixed the actual bug
3609: Refactoring arv-ws into a reusable command module. Working on adding
3609: --job and --pipeline logging implemented, needs testing
3609: Logging should work for jobs but running into a bug in eventbus
Websocket server side fix, perform database notify in after_save callback on
3609: Add some options to control polling fallback, improve error logging a bit.
Websocket server side fix, perform database notify in after_save callback on
3692: Explicitly incorporate sequence number test into where clause
Fix filters, subscribing to components of pipelines.
Start working on adding local unix pipe support to run-command.
3609: Piped commands works
3609: Fix trimming redirect parts of the command line.
Catch ECHILD from os.waitpid()
testing
Use branch
sleep so it doesn't go haywire
New wait logic, report all exit codes.
Search up path to see if a file is in a collection. Refactor to work towards supporting pipelines.
3609: Parallelizing on -- works, added tentative support for --group.
3609: Add command line help strings
3692: Bug fix for inadequate grouping when constructing selection.
3656: Alphabetize list of subcommands. Rename tmp -> tmp_file. Small wording change on doc pages.
3656: Delete unused documentation page
3609: Add --script-version and --repository to aid testing
3609: Add support for batch size, improve ability to pass lists of lists
3609: Further improve list handling. Update documentation to new preferred
3609: Use run-command batch function instead implementing it in run.py. Permit
Merge branch '3656-arv-create' closes #3656
Merge branch '3692-event-bus-fix-and' closes #3692
Add blacklist to commit hook. no issue #
3609: Added documentation page. Added to "arv" frontend command. Bug fix to
3609: Fix schema so 'order' parameter of 'list' is an array instead of a string.
3609: Add test case for PollClient. arvados.event.subscribe() starts a new
3609: Roll back schema change, not appropriate for this branch
3609: PollClient waits on event object instead of time.sleep, fix filters. Also fix --poll-interval and --no-poll
3609: Prefetch pipeline components to find out which jobs to subscribe to.
3787: Completed first draft explaining how to pull, configure, commit and upload a custom docker image.
3787: Extra note that once inside the container the user can do anything.
Missing file
Merge branch 'master' into 3609-arv-ws
Merge branch '3609-arv-ws' into 3609-arv-run
3609: Reenabled support for websockets. Now sets name of pipeline to user's command line.
Making arv-run documentation more readable.
3609: Add additional api_ fields to api client object so that subscribe() can
Merge branch '3609-arv-ws' into 3609-arv-run
3609: import signal; add_mutually_exclusive_group doesn't take any arguments.
Merge branch '3609-arv-ws' into 3609-arv-run
Better documentation WIP
3609: Back out setting api_version, api_host, and api_insecure on api client
Merge branch '3609-arv-ws' into 3609-arv-run
3609: Set "name" to abbreviated form, put full command line in the description field. Continuing work on documentation.
3609: Testing fixups.
3609: Fix PollClient for case when there are no filters (get everything).
Merge branch '3609-arv-ws' refs #3609
Merge branch 'master' into 3609-arv-run
3609: Further documentation improvements, --local now runs pipeline runner instead of just arv-crunch-job, added --no-reuse and --no-wait
3609: Look for parent project in current working dir and create pipeline in
3609: Don't match any character for -lfoo arguments, just match letters.
4083: Searched for everything using strptime and added import _strptime where it was missing. closes #4083
3698: Add check that output_of refers to an actual component.
3698: Fix: components keys are symbols, not strings. Tweaked error message to
Merge branch '3698-invalid-output-of' closes #3698
4058: If a portable data hash correpsonds to a single collection, automatically
4058: Tests for portable data hash redirection and multiple matches page.
3787: Update title, grammar, add "Share Docker images", add job-pipeline-ref.html based on review feedback.
3787: Add note about needing to be in docker group. A few other minor text tweaks.
3787: docker group -> @docker@ group for consistency
Merge branch '3787-docker-docs' closes #3787
4084: Should be able to write tests against websocket-based functionality now.
4084: Add live logging test
4084: An actual failing test that detects the "new log lines causes scroll bar to jump to bottom" bug.
4084: Rework refresh event handling:
4084: Job display, log and buttons refresh using same mechanism as pipeline instances.
4084: Missing files.
4084: Remove stray debugging statement
4058: Tweaked presentation per review feedback
4084: Added tests for auto-refresh of pipeline instance, job, and dashboard pages.
4058: Typo fix "collection" -> "collections"
Merge branch '4058-workbench-collection-redirect' closes #4058
Increase limit for collections controller querying same portable_data_hash as
4295: First pass at more efficient database utilization. A few tests fail.
4295: Fixed tests
4295: Add test that large numbers of events are handled efficiently and correctly.
4295: Abort on NoMemoryError
4295: Fix typo in comment
4295: Use EventMachine::schedule when there is potentially more events to send instead of pushing an event to the channel (which wakes up all listeners).
Merge branch '4295-efficient-websockets' closes #4295
Increase the timeout on "lots of events" websocket test. refs #4295
4031: Refresh provenance helper graph generation to fix bugs and make better
4031: Find for collection dependencies even in the middle of script parameter
4084: Split out behavior of updating job subscribe list to
Merge remote-tracking branch 'origin/master' into 4084-log-pane-refresh
4031: Test for collection provenance/used_by
4084: Fix workbench tests.
4084: Change $(this) to this and add comments. Refactor @pidfile in test_helper slightly.
3609: Inherit --retries from _util. Be more specific about error being caught. Add comments.
3609: Correctly recognize and rewrite directory names in collections.
4031: Improve provenance graph tests for readability, use fixtures instead of
Merge remote-tracking branch 'origin/master' into 4031-fix-graph-connections
4031: Add owner_uuid to graph_test_collection fixtures.
4084: Only load tab on switch if it doesn't have "pane-loaded" class. Increase
4031: Fix tests. Move RequestDuck to test_helper.rb so as not to duplicate code.
4084: Remove "no-auto-reload" css class hint, tabs no longer reload on click by
Merge branch '4031-fix-graph-connections' closes #4031
3609: Documentation improvements. More error checking on run-command list function evaluation.
3609: Further improve documentation and code comments.
3609: Simplify "group" example a bit more.
Merge branch 'origin-3609-arv-run' into master closes #3609
3400: ArvadosResourceList returns all items (paging as necessary) unless limit() is specified.
3400: Too much code that assumes it can call .first over and over without
add 'arv-run' to setup.py refs #3609
4310: Fix race condition between testing for tag and setting tag by reversing
4310: Remove stderr redirect from git rev-list, add it to git tag. Prefix log
Merge branch '4310-git-tag-race' closes #4310
Go to next job after fail_job when there is a tag mismatch. refs #4310
4334: Convert "idle*" state to "down"
4314: Don't expire the token if the job is still in queued state.
Merge branch '4314-crunch-token-expire' refs #4314 refs #4310
4310: Lock job before changing its state to failed.
4310: Qualify namespace of AlreadyLockedError
Merge branch '4310-lock-before-fail' closes #4310 (again)
Merge branch '4334-idle-star-is-down' closes #4334
4084: Add 'arv-log-refresh-control' div to job log pane. Log scrolling test tests both pipeline_instance and job log panes.
3400: ArvadosResourceList retains api token that is active when it is created.
Merge branch '4084-log-pane-refresh' closes #4084
Tweak uuid of test fixture so it doesn't match other tests unintentionally. refs #4084
Python run_test_server runs websockets separately non-SSL for compatibility
3400: Stub out obsolete #links_for and remove result_links and related test.
Merge branch 'master' into 3400-workbench-all-items
Merge branch '3400-workbench-all-items' closes #3400
4535: Add default_url_options to application controller that consults
4535: Change default to false because ~ indicates "must be filled in".
Merge branch '4535-configure-api-host-url' closes #4535
4570: Workbench/API server support for multiple authentication providers on the SSO server.
4570: Rename 'auth_method' parameter to 'auth_provider' to get the terminology right.
4570: Update documentation on configuring SSO server and configuring API server
4570: Fix typo.
4570: Wording fix
4570: Added note about running SSO server.
Install PyYAML into VENVDIR for non-Python tests that use run_test_server.py. no issue #
Update version pin of arvados-cli gem. refs #3698
Touch the "crunch_refresh_trigger" file when the state changes. This notifies
Merge branch '4314-trigger-job-update' closes #4314
3198: Start to refactor locators_and_ranges to use for replacing ranges
3198: Initial support appending to streams.
3198: Support for truncating files. A few tests. Next step work on random access writing.
3198: Can use write to append only.
3198: Refactor stream and file classes and functions a little bit for readability.
3198: Random access updates works.
3198: Working on optimizing rewrites
3198: Support repacking buffer blocks when writes are superceded.
3198: New Collection/AravosFile design work in progress
3198: Refactoring. Added Range object instead of fiddling with arrays, should
3198: Fixing tests broken by previous commit's refactoring. Creating new
3198: More refactoring and bug/test fixing
3198: Writing files works
3198: Populate collection with locator or manifest. Also added Python container API.
3198: Async put, read prefetch via BlockManager. Added arvfile tests (forgot
4869: Improve logging
4869: KeepClient now has a default timeout per block request (10 minutes). In
4869: Keepstore now returns Content-Length headers, and logs the error message
4869: Correctly handle zero-length blocks in Keep client/Keep proxy. Remove
4869: Based on Go documentation, don't set a body ReadCloser on the request
4869: Strip all newlines from error responses, not just leading and trailing
3198: Fix syntax errors
Merge branch '4869-keepproxy' refs #4869
3198: Start using BlockManager. Needs tests.
4869: Enable TCP keepalive and adjust connection timeouts to Keep client.
3198: Many tests. Fixed lots of bugs.
3198: Bugfixes. Working on benchmarking upload/download performance.
3198: KeepClient creates a requests session to re-use connections. export_manifest correctly strips hints when portable_manifest=True.
3198: Added Collection.rename (needs test). Fixing tests broken on account of
3198: Figured out how to mock session object, reverted most tests updated in
3198: Fixed up the rest of the tests to mock the session.
3198: Apply StreamFileReader tests to ArvadosFileReader
4869: Client.Timeout and Client.Transport are now correctly set in
Merge branch 'master' into 4823-python-sdk-writable-collection-api
Merge branch '4869-keepalive' refs #4869
4570: Revert to links on log in page instead of form. Fixup documentation to
Merge branch 'master' into 4570-multi-auth-method
Merge branch 'master' into 4823-python-sdk-writable-collection-api
4823: Working on porting more StreamReader tests to ArvadosFile.
4570: Fix tabs, CSS on log in button.
Merge branch '4570-multi-auth-method' refs #4570
4312: Use "install" phase of bootstrap script to report the installed versions
4312: Call virtualenv pip directly instead of using activate.
4312: Fix dpkg search to use dpkg-query.
4924: Refactor arv edit and arv create to improve error handling.
4312: Remove redundant parenthesis.
Merge branch '4312-crunch-report-sdk-version' closes #4312
4924: Distinguish between errors the user can do something about (syntax errors
4924: Rename HTTPResponse to ArvadosAPIError. Use NET::HTTP Response titles
4924: Update prints uuid from results, not the uuid originally specified.
4924: Collapse JSON rescue blocks to reduce duplicate code.
Merge branch '4924-arv-edit-error-handling' closes #4924
4968: Fix polling refresh on project directories
Explicitly specify ruby 2.1 in "rvm alias" no issue #
Merge branch '4968-refresh-project-dir' closes #4968
4904: Set up fuse, set up event bus, run docker
4904: Chooses most recently modified collection and runs web service on it.
4904: Fixed event listening. Terminates properly on signals. Tested and works now.
4904: Renamed "runit.py" to "arv-web-example.py"
Merge branch 'master' into origin-4823-python-sdk-writable-collection-api
4823: Fixed stream reader test cases
4823: Add clone, synchronized writer
Configure dockerfile with passenger. Add sample applications.
4838: Add --set-executable-bit option to make all files from mounted collections be executable.
4904: CGI sample works.
4904: Rename to arv-web.py to reflect a more general purpose tool.
Add 'apps/arv-web/' from commit 'f9732ad8460d013c2f28363655d0d1b91894dca5'
4904: Moved to services/, finished first draft documentation.
4904: Fix markup of links for phusion passenger and apache, and add links to WSGI and Rack.
4904: Added docker_image files to samples. Added discussion about using your
5016: PollClient.run_forever() polls self.stop() in order to have a chance to process KeyboardInterrupt.
4904: Minimal event subscribe example.
4904: Inspect event type and only print collection create events.
4838: Set +x on all StreamReaderFiles
Merge branch '4838-fuse-executable-files' closes #4838
4823: Working on method documentation and comments for arvfile
Merge branch '5016-ctrl-c-arv-ws' closes #5016
4823: More docstrings for public Collection and ArvFile API
Revert "4838: Add --set-executable-bit option to make all files from mounted collections be executable."
Merge branch 'master' into 4904-arv-web
4823: Bufferblocks are owned by a specific ArvadosFile
Merge branch 'master' into 4904-arv-web
4904: Eliminate downtime switching collections.
4904: Finally figured out a (hopefully) reliable way of sending SIGHUP to
4904: Handle failure to delete non-existent cidfile. Fix docker_image in sample-static-page.
4904: Fix documentation issues. Add comments to arv-web.py.
Merge branch 'master' into 4823-python-sdk-writable-collection-api
4823: Adding @_synchronized to protect arvfile and block manager. Updated
4823: Added SYNC mode constants.
4823: Refactored Collection into SynchronizedCollectionBase, Subcollection, and
4823: Add copyinto()
4823: Added merge(), live sync. Removed deferred _populate().
Merge branch 'master' into 4823-python-sdk-writable-collection-api
4823: Fix bugs, fix tests, existing tests pass again. Still need to write new tests.
4823: Add Collection.copy tests
4823: Add tests for Collection.clone and Collection.merge
4823: Implemented diff/apply/update methods for Collection synchronization. Needs tests.
4823: Add tests for Collection diff and apply.
4823: Switched order of diff (now gives a diff *to* the target collection
4823: Files now fall under collection's lock. Add tests for merge conflicts.
4823: Another merge conflict test.
4823: Rename @_synchronized -> @synchronized and @_must_be_writable to @must_be_writable. Fix arvfile tests.
4823: Add root_collection() to make it easier to find the root. Adjusted
4823: Added apiconfig option to arvados.api() to accept a dict with
4823: Fix instance of Collection -> CollectionRoot. Tweaked a couple of comments.
Merge branch 'master' into 4823-python-sdk-writable-collection-api
4823: Remove commented out StreamWriter code from much earlier version of branch.
3021: Bugfix for arvados_fuse.TagsDirectory to correctly merge tag list updates
4904: Refactor arv-web main() into ArvWeb class. Add CollectionDirectory.change_collection to FUSE.
4904: Detach from Docker container to get cid from stdout instead of fiddling
Merge branch 'master' into 4904-arv-web
Merge branch 'master' into 4823-python-sdk-writable-collection-api
4823: Revert settings mistakenly added to application.yml.example
4904: Remove extraneous ciddir and cidfilepath. Explicit non-zero exit code
Merge branch 'master' into 4904-arv-web
Merge branch '4904-arv-web' closes #4904
4823: Style cleanup on docstrings, renamed some single-letter variables to be
5095: Don't check update when getting mtime().
5095: Record mtime() from group#contents
5095: Set _mtime when the collection object is updated.
Merge branch 'master' into 4823-python-sdk-writable-collection-api
5095: _mtime on collection defaults to 0 instead of time.time().
4823: Notes on how the mocking works with requests.Session
Merge branch '5095-fuse-ls-takes-forever' closes #5095
4823: Add new tests for BufferBlock, BlockManager, saving, updating. Stylistic
4823: More tests and fixes for updating and merging from remote api record.
Merge branch 'master' into 4823-python-sdk-writable-collection-api
4823: Record manifest_text response from API server. Give better names to
4520: --project-uuid now used for owner_uuid on all create() calls.
4520: tested copying, fixed bad 'properties' field
4823: Fix fuse tests for SafeApi -> ThreadSafeApiCache changes. Add a couple
Merge branch 'master' into 4823-python-sdk-writable-collection-api
4823: Revert FUSE changes unrelated to ThreadSafeApiCache
5277: Add test for mtime. Use ciso8601 module to parse arvados timestamps.
4823: Refactoring. ReadOnly Collection is now CollectionReader, replacing old
4823: Fix tests broken by prior refactoring. Renamed 'api.py' to 'apisetup.py'
4823: Handle edge cases of files named '.' so that the FUSE test passes. Added
4823: Clean up imports in collection.py
Merge branch 'master' into 4823-python-sdk-writable-collection-api
Merge branch '5277-fuse-mtime-fix' closes #5277
3785: Upload job log to collection with --project-uuid same owner_uuid as job.
4520: Refactor code to create the collection record. Also refactored code
4520: Bonus fix because arv-copy was giving KeepClient.put() unicode strings
4520: Better checking to see if collection already exists at the destination.
4520: manifest_text() is utf-8 encoded by default so it can be safely put() to
3785: Log tab is no longer suppressed for anonymous users.
4520: Coerce unicode strings to ascii in put(). Use result.content (returns
4520: Tweak test that put(u'foo') does the right thing.
Merge branch '4520-arv-copy-project-uuid' closes #4520
Merge branch '3785-job-log-collection-owner' closes #3785
4823: Remove sync_mode() from Collection in favor of writable() flag.
Merge branch 'master' into 4823-python-sdk-writable-collection-api
4823: Add arvapi parameter to one_task_per_input_file() to solve mocking
5310: Use c.get('name') instead of c['name']
5309: Fix keepclient and keepproxy bugs related to error handling:
4823: Add flush() to ArvadosFile. Fix tests to avoid using internal APIs. Fix
5322: Add documentation to "install SSO" section. (Possibly this should go
5305: Handle collection pdh for docker image
5305: Add heuristics to choose name when collection is referenced by PDH instead of uuid
5309: Add comment about testing for content-length error bug.
Merge branch 'origin-5309-keepproxy-panic' closes #5309
4823: More fixes and cleanups.
4823: Docstring and comment fixes.
Merge branch '4823-python-sdk-writable-collection-api' closes #4823
Update arvados-fuse dependency on python sdk refs #4823
5305: Added num_retries to all execute() calls. Refactored
4956: Add 'maxRequestSize' to discovery document
5305: Remove erroneous comment
Fix arv-mount use arvados.config.settings() to initialize ThreadSafeApiCache
Fixed SafeApi -> ThreadSafeApiCache refs #4823
Merge branch '5305-arv-copy-fixes' closes #5305
Merge branch '5322-sso-manual-account-doc' closes #5322
4956: Add maximum request size checking to http_request patch in api.py.
5246: Add urllib3 workaround, along with a note why it is necessary.
5246: Use pkg_resources.parse_version to test urllib3 version string. Identify
Merge branch '5246-urllib3-workaround' closes #5246
4956: Refactor http request patching used in Python SDK.
4956: Mark monkey patching methods as internal
Merge branch '4956-limit-request-size' refs #4956
4426: Adjust styling for search button on search box
Fix arv-normalize regression. refs #5145 refs #4823
4158: Compute node details are in properties, not info
Add note to application.default.yml and installation documentation that
Typo fix and gofmt fix.
Merge branch '4158-fix-compute-node-detail' closes #4158
Merge branch 'install-doc-uuid-prefix' no issue #
5448: Don't create new tasks if $main::success is defined. Add failure count
5448: Add --all to "docker images" because it turns out it doesn't list all the installed images by default!
5448: Fix cid file cleanup.
5448: Clean up old cid files during job start.
Merge branch '5448-crunch-job-task-fixes' refs #5448
5476: Increase connection timeout when retrying keep get and put.
5476: Better implementation of connection timeout scaling. Updated docstring
5476: Describe timeout behavior if KeepClient.__init__ docstring.
Merge branch '5476-connect-timeout-scale' closes #5476
When copying a collection that already exists on the destination, fetch
5500: Add SLURM "Communication connection failure" to pattern of temporary node
5500: Add TASK_TEMPFAIL constant. shell_or_die always exits non-zero on error.
Merge branch '5500-crunch-temporary-failure' refs #5500
5285: Remove cruft made redundant by Ruby SDK
5318: Add note on sharing page for repository object about lag time taking effect.
5425: Set memory limit on container as a fraction of total system memory
Merge branch '5285-arpi-use-settings' closes #5285
Merge branch '5318-git-sharing-note' closes #5318
Merge branch '5425-set-docker-memory-limits' refs #5425
5536: Keepproxy retry keep server discovery on error or empty list.
5524: Match magic string arvados.errors.Keep as likely part of an exception
Merge branch '5524-crunch-temp-keep-errors' closes #5524
5539: Arvados-in-Docker improvements
5539: Retry keepproxy if it quits (this happens when restarting the container,
Merge branch '5536-retry-discover-servers' closes #5536
Merge branch 'master' into 5539-better-docker
5539: Require that the user update resolv.conf, and use FQDN (api.dev.arvados,
Added a comment to volume Delete() about race condition.
Merge branch '5539-better-docker' closes #5539
5612: Wrap munge with startup script that cleans up /var/run/munge of stale
Merge branch '5612-munge-docker-cleanup' closes #5612
4752: Use passenger standalone for API server instead of passenger apache module and set ARVADOS_WEBSOCKETS=1
4752: Update documentation related to docker
4752: Fix failing setup by polling for API server to come up.
5626: Add git_hostname configuration option.
Merge branch '5626-git-server-hostname' into 4752-docker-websockets
4752: Websockets works. Fix compute node containers to restart correctly. Fix
5626: Add test for git_hostname configuration option.
5626: Expand test to check both positive and negative cases a bit per request.
Merge branch '5626-git-server-hostname' closes #5626
Merge branch '4752-docker-websockets' closes #4752
5692: Collection.manifest_text(strip=False) will flush open files and wait for
5692: Backed out commit_all() from Collection.manifest_text().
5692: Add flush flag to manifest_text() which calls commit_all(). Added
5692: Move implementation to internal _get_manifest_text(). Fix typos.
5692: Don't call _get_manifest_text from test if we don't need to.
3198: Start by refactoring.
Merge branch '5692-pysdk-manifest-text-flush' closes #5692
3198: Fixed imports. Tests pass again.
3198: Implement cache management for directory objects.
3198: Fixed arv-mount for refactoring.
3198: Manage inode cache based on (approximate) object size instead of object
3198: Fix logging
3198: Use new Collection/ArvadosFile API.
Merge branch '3198-inode-cache' into 3198-writable-fuse
3198: Store filename/directory name in ArvadosFile/Subcollection object.
3198: Can modify file, collection objects, changes are reflected in FUSE.
3198: Support for mkdir, rmdir, unlink, rename. Improve exception catching.
3198: Fix test deadlock. Track inode lifetimes.
3198: Implement rename() (efficient move within/between collections).
Merge branch 'master' into 3198-writable-fuse
3198: Support for listening for events to trigger collection updates. TODO: add to arv-mount
5766: Make sure it doesn't try to fetch arvados packages from PyPi, make sure
Check if cache slot is not None before trying to test the read flag. refs #5831
Merge branch '5766-pip-sdk-install' closes #5766
ArvadosFileReader object always tries to return the exact amount of data asked for.
Return error if Content-Length header is missing instead of panicing.
5856: Tweak tests. Fix typo.
Merge branch '5856-read-exact' closes #5856
Merge branch '3145-missing-content-length-error' closes #3145
5562: Add support for "Expect: 100-Continue" flow for PUT.
5562: Add comment about UPLOAD
5562: Use cStringIO module which is more efficient than StringIO.
run-command list correct directory to be uploaded when $(task.cwd) != $(task.outdir)
5562: keepstub sends "100 Continue" to client. Fix test_write_three_replicas
5787: run-command uploading uses new collection API and supports symlinks into Keep
5787: Copy to the right destination name, add docstring, fix syntax errors
5787: Write files that are pointed to by symlinks that are outside of Keep.
5787: Error writing output will cause task to fail. Tweak documentation page.
5787: Fix syntax error
Merge branch '5562-pycurl-upload' closes #5562
5787: Change caught_error to last_error in vwd.py to return last exception that was caught. Fix typo.
Merge branch '5787-run-command-checkin' refs #5787
3198: Inodes() and Operations() take InodeCache() object directly. Minimum
3198: Add --file-cache and --directory-cache to arv-mount to specify desired
Merge branch '3198-inode-cache' into 3198-writable-fuse, fix tests.
3198: Fix Operation(inode_cache) init. Break up tests into a few smaller
Merge branch '3198-inode-cache' refs #3198
5992: keepproxy strips +K@{5} hints from locators before trying GET. +K at 5-5-15
Merge branch '5992-keep-proxy-ignore-locator-hints' closes #5992
Merge branch 'master' into 3198-writable-fuse
3198: Support efficient moving preserving inode, writing to unlinked files. Many new tests.
3198: FuseMvDirBetweenCollectionsTest works
3198: Serialize updates to avoid doing redundant API server calls.
3198: Support mkdir() and rmdir() to create collections on projects. Support
3198: Start working on rename() for collections in projects
3198: Renaming collections in projects works. Improved conformance to POSIX semantics as best we can.
3198: Support setattr().
Merge branch 'master' into 3198-writable-fuse
5748: Fix leaks in GetBlock error handling and PutBlock use of GetBlock where
Merge branch '5748-max-buffers-leak' into 3198-writable-fuse
5748: Basic tests to check for bufferpool resource leaks
3198: writeto() notifies with WRITE instead of MOD. WRITE doesn't invalidate the inode.
3198: Add docstring with overview of the architecture. Implement check_update
3198: VWD syncs collection before loading from API server.
3198: Enable support for event bus based updates in arv-mount.
Merge branch 'master' into 3198-writable-fuse
6073: Pycurl depends on latest in Arvados SDK setup.py
6071: Increase default timeout and enable retry support.
6071: Refactor project organization a bit.
6071: Fix URL
Merge branch '6073-pycurl-version' refs #6073
6071: Bump up timeout to 120 seconds.
Merge branch '6071-java-sdk-timeouts' closes #6071
5766: Make sure it doesn't try to fetch arvados packages from PyPi, make sure
6090: Docker install uses local SSO server instead of auth.curoverse.com. Also
6138: Added --domain to set ARVADOS_DOMAIN. Removed useless comments in apache2_vhost that mentioned qr1hi.
6141: Remove hard-coded "https://" from "https://{{site.arvados_workbench_host}}" and require that arvados_workbench_host include the url scheme instead.
5548: Script to test, build and upload common workflow language packages and
5548: Remove git commands from run-cwl-tests. Remove cwltool from build-packages.
Merge branch '6090-docker-use-local-sso' closes #6138
Merge branch '5548-test-cwltool' refs #5548
Merge branch '6141-doc-workbench-links' refs #6141
6194: Chunk large ArvadosFile writes automatically instead of raising an error.
6194: Fix typo in invocation of writeto() and use memoryview to avoid copying slices.
6194: Fix test. Lots of small writes break across blocks differently than one huge
Merge branch 'master' into 6194-python-arvfile-large-write
6194: Make splitting loop simpler since [n:n+KEEP_BLOCK_SIZE] returns a short
6194: Simplify test_large_write a little bit.
Merge branch '6194-python-arvfile-large-write' closes #6194
6235: Add method to get parameters from API discovery document.
Fix "docker-node-engine.sh" renamed to "build-node-docker.sh", add additional Docker image to upload. no issue #
6235: Discovery() returns the requested value directly instead of a single-entry map.
Merge branch '6235-go-sdk-discovery' closes #6235
Automatically build and upload documentation site. no issue #
Merge branch 'master' into 3198-writable-fuse
3198: Adjust language of errors to say what the error is instead of saying what the
Since jenkins reuses working directories, handle updating existing git repo of
Set default identity for cwl repo pushes. no issue #
CWL script won't fail when there are no documentation changes to commit. no issue #
3198: arvfile.py and collection.py updates based on feedback.
3198: Additional fixes based on review feedback.
3198: Fix tests for change in ~conflict~ format. Tweak comments.
3198: Fix frontrunning (subsequent updates after a commit causing the
3198: Performance tuning based on benchmarking. Limit number of segments to
3198: Add enable_write flag to FUSE and --enable-write and --read-only to
WIP
3198: concurrency test and associated fixes
3198: Fix test
3198: Rename --enable-write to --read-write. Tweak docs.
Merge branch '3198-writable-fuse' closes #3198
Update arvados_fuse version dependency on arvados-python-client. refs #3198
Additionally build and upload cwl-runner package. no issue #
6264: Arvados CWL runner prototype work in progress.
6264: First pass complete, ready for testing.
6264: Uploads files and Docker images, can almost run jobs.
6264: Passes 29/35 conformance tests. Some debugging and feature addition remains.
6264: CollectionFsAccess falls back reading local files.
Support job reuse, CreateFileRequirement, and EnvVarRequirement
6264: Support setting environment variables.
Merge branch '6264-run-command-task-env' into 6264-cwl-runner
6264: Support EnvVarRequirement and CreateFileRequirement.
6264: Add task.env to documentation.
Merge branch '6264-run-command-task-env' refs #6264
6264: Move cwlrunner into its own module.
6264: Fix bin/cwl-runner to import correct module.
6484: Ignore filehandle flush if object is not writable.
Merge branch 'master' into github-3408-production-datamanager
Merge branch 'github-3408-production-datamanager' refs #3408
Merge branch '6484-fuse-fsync-erofs' closes #6484
6221: Make never_delete true by default, make trash_worker respect never_delete, add
6221: Check never_delete must be true.
6221: Generate trash list, not sent to keep servers yet.
6221: Successfully writes trash lists.
Merge branch '6221-datamanager-delete' refs #6221
Add datamanager tests. refs #3408
6221: Fix error reporting. Fix keepstore not to delete blocks with mismatch
6221: Add trash_list_test.go
6221: Add SendTrashLists() test. Propagate errors from BuildTrashList. Style fixes.
6221: SendTrashLists() returns errors. Tests updated.
6221: Pass server to sendTrashListError(), use NewUnstartedServer()
6221: SendTrashLists only return non-nil array on error.
6264: Use mutually exclusive group for enable/disable-reuse.
6264: Code cleanup: rename "i" in _match() to "filename" and "t" and "r" in
6264: Typo fix.
Merge branch '6264-cwl-runner' closes #6264
6705: PollClient checks if there are more items and sends another query
6705: Use EventMachine::next_tick (which always pushes task into work queue)
6451: Don't crash on ArgumentError. Add test to verify that it doesn't crash.
5990: If --docker-image not specified, don't provide
6681: When input file is "-", get data directly from sys.stdin and use
6593: Use sys.stdout as special case instead of trying treat /dev/stdout as a regular file.
6706: Clear inode/inode cache dicts on destroy() instead of setting inodes to None.
6706: Fixup websocket tests of PollClient
6706: Bonus websockets bugfix, close connection immediately instead of waiting
6706: Add comments about what is going on with EventClient.close()
Merge branch '6705-fix-websockets-replay' closes #6705
6643: Add test case that reproduces the bug
6643: Support associating uuid with multiple inodes to fix double-remove bug.
6706: Add docstrings to close()
Merge branch '6706-fuse-forget-crash' closes #6706
Fix parameter ordering for minor change in cwltool command line semantics. no issue #
6593: Change docstring references from /dev/stdout to just "stdout". Fix logic
6451: Validate that 500 error is returned to client. Tweak code comments.
6451: Refactor tests slightly to call subscribe_test() in two places.
5990: Update arv-run subcommand cli doc.
6593: Typo fix in KeyboardInterrupt handling.
Merge branch '6593-arv-get-stdout' closes #6593
Merge branch '5990-arv-run-default-runtime' closes #5990
Merge branch '6451-websockets-invalid-input' closes #6451
6676: Working through polishing SSO install documentation.
6676: Add note about git and curl. Describe how to run passenger standalone.
Merge branch 'master' into 6676-document-sso
Merge branch '6643-fuse-del-crash' closes #6643
Merge branch '6681-arv-put-pipe' closes #6681
Hotfix, inode cache find() returns list instead of single item now. refs #6643
Hotfix for test, transform "/dev/stdin" to "-" in arv-put refs #6681
6590: Reorganize application.default.yml. Improve documentation for essential
6676: Reorganize instructions a bit more. Fix createdb instructions. Fix
6590: Rename database.yml.sample to database.yml.example so be consistent
6676: Grammar fixes.
Merge branch 'master' into 6676-document-sso
Merge branch '6676-document-sso' refs #6676
6507: Initial commit
Create, destroy, tag nodes.
6507: Add basic azure driver tests.
6507: Added notes to sample azure configuration file
6507: Paste in command line for creating service principals instead of just
6507: Set hostname tag in arvados_create_kwargs instead of sync_node.
6507: Fix tests
6507: Use get_image() instead of list_images() for image lookup.
6507: Fix test
7012: Rewrite introduction page to remove links to Docker install and clarify
7012: Remove Docker install from _config.yml as well.
6918: Update documentation to remove CORS headers from sample nginx
Merge branch '7012-no-docker-install-doc' closes #7012
6941: Add note about needing to fill in correct web server user in places where
Merge branch '6941-docs-fix-www-data' closes #6941
6918: Fix server_name in example nginx config. Remove client_body_buffer_size.
6918: Further clean up recommended nginx proxy configuration.
Merge branch '6918-remove-nginx-cors-headers' closes #6918
6507: Remove unused imports, move _init_ssh_key() into base, set hostname tag on sync_node(), tweak tests.
Fix up additional places in keepproxy install docs to use example
6507: Use dependency_links to specify custom libcloud package needed for azure support.
6507: Filter out "tag_" from create_kwargs.
6507: Change version number for libcloud fork to 0.18.1.dev1 because local tags create too much trouble.
Preinstall forked version of libcloud, because nodemanager "pip install" won't
Merge branch 'master' into 6507-node-manager-azure
Merge branch '6507-node-manager-azure' closes #6507
Script to automatically test python package installation on clean Docker
Sets up local repositories for both centos (yum) and debian (apt), installs
Log package dependencies for each shared object. Add -q so make the logs a bit
Tests compatible libcloud package is installed.
Fix import error for NodeAuthSSHKey refs #6507
7193: Add ex_run_command
7193: Run provision script after creating node, not in sync_node, because
7193: Set timestamp on ex_run_command to ensure that the provisioning script runs.
7193: Bump dependency to dev2
7031: Check out our forked libcloud from git and make packages from that.
7031: Fix python prefix
Bump version to dev2
7193: Move ex_run_command to post_create_node()
7193: Use tags instead of _make_ping_url
Merge branch 'master' into 7031-build-forked-libcloud
Merge branch '7031-build-forked-libcloud' refs #7031
7193: Tweak example config.
Merge branch '7193-provision-azure-compute' closes #7193
Add libcloud-pin and update to dev3 refs #7031
Bump node manager libcloud pin to dev3 refs #7031
Catch errors thrown by sync_node() in Azure driver refs #7031.
Brown bag typo bugfix. refs #7031
7225: Fix typo in _BufferBlock.set_state() that resulted in "wait for pending
7225: Add test to validate fix that 2nd commit call blocks until block is committed
7225: Better exception handling for unexpected state changes.
7228: Add chmod -R 2775 $RELEASE_PATH/tmp/cache/ so that cache is group writable.
Merge branch '7225-collection-hang' closes #7225
7228: Set umask to 002 so cache files are created writable other processes with same GID.
Merge branch '7228-group-writable-tmp-cache' refs #7228
7181: Check for [Errno 2] in the output and assume that means git is missing.
Merge branch '7228-crunch-dispatch-umask' closes #7228
Merge branch '7181-crunch-missing-git' closes #7181
7286: Move logic to shut down newly booted nodes nodes that haven't pinged to
7286: Tests for new "missing and broken" shutdown policy.
7286: Add test that "missing" nodes are not counted towards "busy" (but are
7286: Compute "missing" based on "last_ping_at" instead of using API server's
7286: Missing nodes are considered in "excess" count (reverts previous change). Added test. Also remove debug log statement.
7286: Fix double count of missing nodes in shutdown
7286: Add drain* and fail* to SLURM_END_STATES, because the '*' means the node
7286: Add BaseHTTPError to list of "cloud errors"
7286: Use ex_customdata to put the ping URL on the node instead of
Update pin of libcloud fork to dev4 refs #7286
7286: Add comments clarifying arvados_node_missing() and broken(). Also bump
Merge branch '7286-nodeman-destroy-broken-nodes' closes #7286
6142: If self._set_node_state('RESUME') in cancel_shutdown() returns non-zero,
Update cwl builder process for repository reorganization. No issue #
Run CWL tests for both draft-2 and draft-3. No issue #
6142: Only resume from 'drng' or 'drain'. Add/fix tests.
Merge branch '6142-cancel-slurm' closes #6142
7491: Go keepclient retries GET on either network error, or 500 error from
7491: Add test that default replication is read from discovery document.
7491: Manual merge with 7491-keepclient-bugs
7491: Go through entire server list instead of immediately retrying on error.
7491: Remove KeepServerError, all errors are BlockNotFound errors for now.
Merge branch 'master' into 7491-go-keepclient-retry
7491: Fix response body close, error logging.
6321: Add OSError to slurm shutdown actor retry conditions, to catch and retry
7491: Remove unnecessary writable service roots from GET tests.
Merge branch '7491-go-keepclient-retry' closes #7491
Merge branch 'master' into 7370-package-install-testing
7370: Refactoring way the package test is set up and executed.
Merge branch 'master' into 7370-package-install-testing
7370: Fix centos6 package builder container.
6321: Add test that OSError is caught from slurm subprocess invocations.
6321: Add note about rationale for retrying on OSError.
Merge branch '6321-slurm-oserror' closes #6321
7370: run-build-packages-all-targets hands off to run-build-packages-one-target, suppports --test-packages option.
7582: (1) Probe for non-root Docker user to use instead of assuming "crunch".
7582: Crunchrunner work in progress.
7582: Working on tests.
7582: More tests, add vwd support
7582: Uploader mostly done, writing tests
7582: Uploader passes tests
7582: Runner uploads results. Feature complete.
7582: Better reporting in the log about user probe behavior.
7582: Better reporting in the log about user probe behavior.
7582: Add parameter substitution. Improve validity checking for filenames.
7582: Make fields in Job, Task, TaskDefs public so that json loading reflection works.
7582: Don't call stdbuf in minimal run mode.
7582: fix typo --user=$try_user to $try_user_arg
Merge branch '7370-package-install-testing' closes #7370
7582: Adjust signal catching to eliminate races. Tighten up code based on comments.
7582: Add test for stdbuf in /bin/sh bootstrap script.
Merge branch '7582-run-any-docker-container' refs #7582
Merge branch 'master' into 7582-crunch-runner
7582: Fixup to work with latest cwltool. Runs jobs with Go crunchrunner.
7582: Passes draft-2 conformance tests.
7593: Don't upload the same files more than once. Fix handling "./" in glob paths.
7582: Fixup concurrency around signal catching and forwarding.
Merge branch '7582-crunch-runner' refs #7582
7593: References to files in keep must have keep: URI scheme. Improve error
7593: Fixup to use keep: URI scheme globbing for outputs.
7593: Generate files replaces $(task.keep)/ with keep: notation to reference
7593: Make peter/crunchrunner the default repository (so that it works on
7593: Add sdk/cwl to package building.
Update Gemfile pin on arvados-cli to ensure latest crunch-job refs #7582
Fix typo in arvados-cli version number refs #7582
7593: Add arvados-cwl-runner to disambiguate if there is a conflict over what
Obsoleted by run-build-packages-all-targets.sh --test-packages refs #7370
Remove unnecessary yum update. touch RPM database to work around overlayfs bug.
7593: Add version hint to arvados-python-client. Add get_uploaded() and
Merge branch '7593-cwl-crunchrunner' closes #7593
5353: Give NodeManagerDaemonActor access to ServerCalculator object.
5353: Parameterize the following methods on node size: _nodes_up, _nodes_busy,
5353: Existing tests pass now. (Still need to add a few tests that explicitly
5353: Add a couple of tests to explicitly create nodes of different sizes
Merge branch '3585-arpi-project-uuid-wip' closes #3585
Merge branch '7593-arvados-cwl-runner' refs #7593
5353: Added max_total_price. Added more tests for multiple node sizes.
5353: Add note that min_nodes boots cheapest nodes.
5353: Fixes from testing with Dummy driver.
3137: Add stat counters for bytes uploaded/downloaded (keep) and read/written (fuse).
3137: Refactor stats to record keep & fuse operations as well as bytes.
3137: Add counter & logging for cache hits
3137: Add --stats to arv-mount in crunch-job
5353: Fix typo in _nodes_wanted(). Calculate number of nodes that can boot
5353: Add a couple comments to tests.
5353: Update comment about min_nodes and node size.
5353: Remove extra assertion because busywait does it for us.
Merge branch '5353-node-sizes' closes #5353
3137: Change --stats to --crunchstat-interval as specified on the ticket.
7654: Fix hang in close().
3137: Bump Python SDK version requirement. Tweak Stats() class, don't keep two
Merge branch '3137-arv-mount-stats' closes #3137
Merge branch '7654-ws4py-hang' closes #7654
Hotfix: use a recursive lock for closed_lock so that EventClient.close() can be
5353: Explicitly set size field on node objects returned by list_nodes on AWS and Azure.
5353: Remove checks that cloud_node.size is None (because it should never be None or
Merge branch '5353-set-node-size' refs #5353
Fix for _size_shutdowns and node prices in node manager refs #5353
5352: Set node size on records returned from libcloud create_node
Merge branch '5353-booted-size' refs #5353
Fix Azure node listing in node manager. refs #5353
6309: Add runtime constraint keep_cache_mb_per_task
7832: Add __slots__ and remove __del__ which interfers with garbage collection. Memory usage comes down by ~40%
Merge branch '7832-pysdk-use-slots' refs #7832
6309: Use arv-mount default cache size. Fix documentation typo.
Merge branch 'master' into 6309-file-cache-runtime-constraint
Merge branch '6309-file-cache-runtime-constraint' closes #6309
6429: Initial commit of models creating by rails generate.
Stub out controllers and routes. Add basic workbench support.
Declare serialize / accept json fields, start working on states
Working on state change assertions and testing for container record.
Finish basic state change & record protection unit tests.
6429: Put whitelist_update in module.
6429: Committing container request creates container
6429: Tests for priority update propagation to process tree, max priority from
6429: Use text instead of string for longer API fields.
6429: Complete/Cancelled containers finalize associated container requests;
6429: Don't update priority if the container is already in Complete or Cancelled state.
6429: Improve variable names in tests a bit. Default priority of container
6429: Rename request_finalize to handle_completed
6429: Callback calls container_completed! instead of just setting state to final.
Merge branch 'master' into 7454-azure-custom-data
7454: use small CustomData script to set /var/tmp/arv-node-data/arv-ping-url
7454: Add 'size' to arvados_create_kwargs so that it can be included in CustomData.
7454: Fix tests
7939: Move daemonize to _run_standalone() after llfuse.init(), preserving open
7939: Check exit code in tests.
7939: Additional check for bad token.
6429: Add for test child container requests are set priority=0 when parent
7939: Use 100:: for bogus host check and add comment about API check.
Merge branch '7939-arv-mount-errors' closes #7939
Merge branch '6429-crunch2-api' closes #6429
Small test cleanup. refs #6429
Cleanup of test cleanup. refs #6429
Add required indexes to containers and container_requests tables. refs #6429
Merge branch '6429-fix-container-indexes' refs #6429
7454: switch order of arvados_create_kwargs() to be consistent with
Merge branch '7454-azure-custom-data' closes #7454
Fix documentation typo git_repo_https_base not git_repo_http_base, no issue #
7816: Add exit_code field to Container model. Can only be set when container
8080: Pull c.reload etc out of the assert_raises. Remove spurious blank lines.
Merge branch '7816-exit-code' refs #7816
Add pycurl version constraint to Python SDK because of breaking API change, refs #8104
7816: Update Go SDK to use correct block locator pattern. Change FileToken
7816: Crunch2 executor prototype work in progress.
7816: Converted testing to use stubbed out ThinDockerClient instead of real Docker.
8008: Test each Arvados package individually.
8080: Successfully tests all packages. Produces before/after/diff lists for each package.
8008: Run bundle install and bundle list for api server and workbench.
8008: Add git dependency to api server and workbench.
Fixing our fpm rails package building to handle fpm-info.sh.
8008: Add dependencies on libcurl and postgresql dev packages in order to build
8008: Incorporate SSO package testing into general package testing framework.
8008: SSO package tests pass for all targets.
8008: Remove workbench git version constraint.
7816: Change ThrottledLogger Stop() to Close(). Choose finalState once so that
7816: Fixes recommended by golint.
8008: Move test files into their own directories. Fix rails package tests to
8080: fpm_depends comes before run-build-package commands; fpm_args comes after
Merge branch '8080-arvbox' closes #8080
Merge branch '8080-arvbox' closes #8080
7816: Rename to crunch-run
7816: CopyReaderToLog renamed to ReadWriteLines. Use Writer instead of Logger
Fix gitolite-shell path refs #8080
Fix gitolite-shell path refs #8080
6605: arv-copy preferentially uses HTTP git urls.
Merge branch '7816-crunch2-exec' closes #7816
Add crunch-run to run-tests refs #7816
Enable arvbox user to sudo to crunch user. refs #8080
Enable arvbox user to sudo to crunch user. refs #8080
Document some arvbox hw/sw requirements refs #8080
Document some arvbox hw/sw requirements refs #8080
7846: Better directory entry invalidation, fixes MagicDirApiError test. Also
Merge branch '8008-package-testing' refs #8008
8008: Tweak error code reporting. Remove debugging line.
Merge branch '8008-test-all-packages' closes #8008
6833: Collection update file block list (to get most recent tokens) even when actual
6833: Test to confirm that enabling polling on CollectionDirectory causes
6833: Test setting small blobSignatureTtl. Fix earlier fix.
Merge branch '7846-magic-invalidate-entry' closes #7846
Merge branch '6833-test-token-expiry' closes #6833
Fix python sdk tests refs #6833
Bump libcloud package version pin to 0.20.2dev1 refs #8235
Just create arvbox superuser instead of creating a whole useless database.
Just create arvbox superuser instead of creating a whole useless database.
Add "status" command, refs #8080
Add "status" command, refs #8080
Fix markdown for arvbox README.md refs #8080
Fix markdown for arvbox README.md refs #8080
8206: Refactor _retry into common function wrapper usable by both dispatch and
8206: Add test to support retry on create_driver.
Fix selecting most recent package for package testing, refs #8008.
Bump iteration of rails packages to -4 refs #8008
chown /etc/ssl/private to arvbox refs #8080
chown /etc/ssl/private to arvbox refs #8080
Arvbox fixes: use 'postgres' database when testing if a user exists. Copy
Arvbox fixes: use 'postgres' database when testing if a user exists. Copy
Use volumes for /var/log/nginx and /etc/ssl/private so that changing ownership
Use volumes for /var/log/nginx and /etc/ssl/private so that changing ownership
8319: Fix arv-cwl-runner file references when referencing files in Keep.
8319: Fixes to run bcbio-nextgen CWL workflow. Now creates pipeline instance
Add distinct fpm_exclude so it shows up in the right place on the fpm command
Pin llfuse to 0.41.1 because 0.42 came out and broke things. no issue #
Pin llfuse to 0.41.1 because 0.42 came out and broke things. no issue #
6702: Catch GCE create_node() errors and check if the node was actually
8206: Refactor _retry to RetryMixin. Make retry timing consistent.
8206: Mock time.sleep() to avoid unnecessary delay in test.
Merge branch '8206-gce-retry-init' closes #8206
Merge branch 'master' into 6702-gce-node-create-fix
Merge branch 'fix/build-python-llfuse-version' of https://github.com/wtsi-hgi/arvados
Merge branch '6702-gce-node-create-fix' closes #6702
6702: Refactor create_node to BaseComputeNodeDriver so logic also applies to
Merge branch '6702-gce-node-create-fix' closes #6702
Try to make logging identify the actor consistently
7667: Node manager bug fixes and logging improvements.
7667: Combine polling logs into fewer lines for less noise. Adjust message
8285: Add want_event_subscribe flag to subclasses of fusedir.Directory,
8285: Add test for listen_for_events
7667: Fix log message
7667: Store node size in a table so to avoid blocking on booting and shutdown
Merge branch '7667-node-manager-logging' refs #7667
8404: catch and continue from interrupted system call from os.wait()
8404: Adjust try block to just surround os.wait().
Merge branch '8404-catch-interrupted-syscall' closes #8404
8406: Treat EXIT_TEMPFAIL as EXIT_RETRY_UNLOCKED if we have previously gotten
8406: @job_retry_counts.include? jobrecord.uuid because @job_retry_counts has a default value.
8409: Report Keep cache miss rate & Keep cache utilization
8409: Add recommendation if cache miss rate is above 0.5%. Fix tests.
8409: Adjust recommended miss rate to below 0.2%
Merge branch '8409-report-keep-cache' closes #8409
8015: Crunch run mounting and output upload WIP
8015: Working on WriteTree() and setting up mounts.
Add arvbox support for overriding settings in application.yml
Add arvbox support for overriding settings in application.yml
8015: Finished initial implemention of input and output mounts. Needs tests.
8015: Fix compilation
8015: Fix existing tests, still need tests for new functionality
8015: Test WriteTree
8015: Fix create and update API calls to use correct nested dict.
8015: Fix compile, need to fix tests.
8015: Fix tests
8015: Improve handling for arv-mount start, arv-mount exit.
8015: Add arv-mount command line test
8015: Expand arv-mount command line tests
8015: Report container UUID in error message. Uploader fix for empty & zero-length streams.
8015: Handle Docker logging correctly, interpret frame headers and
Report when container runner finishes.
Add crunch-dispatch-local to arvbox. no issue #
Add crunch-dispatch-local to arvbox. no issue #
Explicitly install Docker 1.9.1 instead of using get.dockerproject.org (which gets the latest). no issue #
Explicitly install Docker 1.9.1 instead of using get.dockerproject.org (which gets the latest). no issue #
8015: check error when deleting temporary directory.
Move services to /var/lib/arvbox/services because they include logs and logs
Move services to /var/lib/arvbox/services because they include logs and logs
8015: Don't use Docker logging driver. Attach stdout/stderr before starting container.
8015: Fix tests again for endianness of Docker logs, added --read-write to arv-mount command line.
8319: Add environment variable to disable websockets.
8319: Add log message about uploading docker image
Merge branch '8319-bcbio-cwl' closes #8319
6605: Check to see if each git URL works by using git ls-remote.
Run slurmctld and slurmd inside arvbox. refs #6518
Run slurmctld and slurmd inside arvbox. refs #6518
6518: Dispatch to slurm using sbatch
Merge branch '8015-crunch2-mount' into 6518-crunch2-dispatch-slurm
6518: Log when container is cancelled.
6518: Add --share, quoting, shut down priority checking goroutine when
Pin google-api-python-client and oauth2client versions to deal with
8488: Use crunchrunner binary instead of git repo
8488: Set certificate path.
8488: crunchrunner brings certificates with it to ensure that it can
8488: Update default arvados branch to master
6605: Use git -c instead of "git config" to set credentials. Refuse to use
8416: Remove find_node and use existing search_for feature.
8437: Add FullStopActor which uses os.killpg() to terminate node manager on_failure. Added test.
Generalize arvbox 'svrestart' to 'sv' no issue #
Generalize arvbox 'svrestart' to 'sv' no issue #
Fix arvados-login-sync for vm account creation
Fix arvados-login-sync for vm account creation
Tweak slurm configuration to run as non-root user.
Tweak slurm configuration to run as non-root user.
Merge branch 'arvbox-fixes' no issue #
8488: Fix output collection to accomodate reverse mapping fixes in cwltool.
6518: Working on using strigger to update job records when crunch-run cannot.
6518: strigger works
8437: Add missing test.
8488: Update cwltool pin
8488: Use OrderedJsonModel so that pipeline components show up in the order they were queued.
8416: Don't catch exception from search_for, potentially hides exceptions we
Merge branch '8416-node-manager-find-node' closes #8416
8437: Test all exceptions that should be caught.
Merge branch '8437-nodemanager-on-failure' closes #8437
6605: Add --allow-git-http-src and --allow-git-http-dst instead of looking at ARVADOS_API_HOST_INSECURE.
Merge branch '6605-arv-copy-http' closes #6605
8015: Fix typos in error reporting. Fix tests.
Add 'tools/arvbox/' from commit 'd3d368758db1f4a9fa5b89f77b5ee61d68ef5b72'
Merge branch '8015-crunch2-mount' closes #8015
8231: Move arvbox README into install guide.
8488: Determine actual output dir from reading logs. Need to set
8488: Set docker_outdir to get correct outdir in command line. Bump cwltool
8231: Fix nginx logging issue
8231: Convert arvbox documentation to textile. Update root Arvados README to
Merge branch '8231-publish-arvbox' closes #8231
Remove arvbox from arvados-dev (now in arvados/tools) refs #8231
8488: Match line beginnings
8488: Fix reverse mapping for references to input files from keep.
Add git clone to arvbox quickstart doc. refs #8230
8620: Better onramp in README.md so people visiting Github page for Arvados get
8620: Add link to cloud.curoverse.com, and mention using arvbox for development
8620: Crunch is a cluster and HPC workflow engine.
8620: Link to Hacking Arvados page under Development.
8620: Remove unnecessary "Arvados" header.
8620: Add note about arvbox publicdemo mode and link to Hacking page.
Merge branch '8488-cwl-crunchrunner-collection' closes #8488
Merge branch '8620-better-README' closes #8620
Arvbox improvements: add "pipe", "cat", "ls". Report both container IP address
Add arvados-cwl-runner to arvbox sdk install script. no issue #
arvbox "ready" script correctly waits for all sdk packages to be ready. no issue #
Rename arvbox command 'reboot' to 'rebuild' no issue #
Self test script for running CWL conformance tests on arvados-cwl-runner
Bugfix for CWL conformance test script, refs #8558
CWL test select config, refs #8558
Fix crunch-run tests to pass reliably. refs #8015
8654: arvados/jobs image install arvados-cwl-runner package
8558: Add test case for running ArvadosCommandTool
8543: Implement TellActorProxy which uses Actor.tell() instead of Actor.ask().
Merge branch '8543-nodemanager-fewer-futures' closes #8543
8636: Fix ArvPathMapper.reversemap to return a tuple as expected by caller.
8654: cwl-runner crunch stub.
8654: cwl-runner crunch script
Stuff
Fixes
Add Dockerfile & update sources.list
8673: Adding --project-uuid so resources are created in the specified project.
Fix node manager to send ActorProxy instead of TellActorProxy to
8673: Typo fix for --project-uuid. Refactor main loop locking to address
8673: Report pipeline instance uuid. Reuse jobs by default. Bump cwltool
8673: Fix --project-uuid for arv-keepdocker upload.
Fix proxy/tell_proxy usage in _begin_node_shutdown, refs #8678
8654: Passes in local testing.
Merge branch '8673-cwl-runner-project-uuid' into 8654-arv-jobs-cwl-runner
8654: Collect output files into output collection, use as output for task.
Merge branch '8673-cwl-runner-project-uuid' closes #8673
Fix cwl-runner tests. refs #8673
Really fix cwl-runner tests this time. refs #8673
Proxy objects held in node manager dict of booting nodes should be regular proxy(), not
6518: Merge in tests. Code cleanup around variable naming and comments.
Merge branch 'master' into 6518-crunch2-dispatch-slurm
6518: Add crunch-dispatch-slurm to tests.
6518: Add check for sbatch/strigger command line in test.
6518: Use ARVADOS_API_HOST when checking strigger command line in test.
Merge branch '6518-crunch2-dispatch-slurm' closes #6518
8561: Fix test_arvados_node_match
Node manager bugfix: late subscribers should get proxy() not _later (which is a tell_proxy())
8561: Set ec2_instance_id and cloud_factory().node_id to mock cloud/arvados id
Improve exception reporting for arvados-cwl-runner, no issue #
8654: Resource upload WIP
Fix log scanning for "crunchrunner: $(task.outdir)" performed by arvados cwl
Enable BaseComputeNodeDriver.list_nodes to pass through keyword arguments.
Merge branch 'master' into 8654-arv-jobs-cwl-runner
8654: Scanning for and uploading dependencies for submit works. Add test for
8654: Fix keeppath and add ensure_unique_name to upload.
8654: Update cwl:tool entry
8654: Add runner job to pipeline instance.
8654: Set args.submit = False
8654: Add args.debug = True to get backtraces on errors
8654: Job has uuid and state fields
8654: Bump arvados-cwl-runner version dependency on arvados python client.
8654: When uploading, reuse existing collections with same owner/name/PDH
8654: Bump arvados-cwl-runner version dependency on arvados python client.
8654: Upload Docker images
8654: Support waiting for submitted runner job to complete
8654: Support --quiet
8654: Handle error on capturing runner output
8654: Remove print statements.
Crunchrunner fix connect os.Stderr to cmd.Stderr & update crunchrunner
Merge branch 'master' into 8654-arv-jobs-cwl-runner
8654: Use iterator instead of readlines() when going through job log file.
8654: Bump cwltool version
8766: Port logic from arv-run-pipeline-instance for copy collection to parent
8766: Refactor logic to copy output collection and add tests.
Merge branch '8766-cwl-collection-project' into 8654-arv-jobs-cwl-runner
Merge branch 'master' into 8766-cwl-collection-project
Merge branch '8766-cwl-collection-project'
8654: Add value for quiet to args.
Fix merge conflict. refs #8766
Don't retry slurm subprocess calls that raise OSError, no issue #
8654: --version reports versions for arvados-cwl-runner, arvados-python-client,
8654: Passes 100% CWL conformance tests using running cwl-runner in a crunch job!
Merge branch 'master' into 8654-arv-jobs-cwl-runner
8654: Bump cwltool version dependency and print cwl version string in cwl-runner crunch script.
8815: Crunch-job bind mounts crunchrunner binary and certificates from host.
8815: Rely on system-provided crunchrunner. Also use arvados/jobs by default if no docker provided.
8815: Now expect /usr/local/bin/crunchrunner. Bind mount host certificates to
8815: Fix syntax errors.
8654: Make --submit --wait the default mode.
Merge branch '8815-crunchrunner-everywhere' into 8654-arv-jobs-cwl-runner
Merge branch '8815-crunchrunner-everywhere' closes #8815
Fix sdk/cwl test refs #8815
8654: Fix version string produced by arvados-cwl-runner.
8654: Pin pyasn1_modules to version that is compatible with pyasn1==0.1.7.
crunchrunner crunch script selects between $JOB_PARAMETER_CRUNCHRUNNER
8654: Update test_with_arvbox.sh
8654: Print uuid of uploaded docker image on stderr instead of stdout.
8654: Add comments
8654: Add missing test_submit
8840: Use 'with_lock' instead of 'transaction' in Job.lock method.
8857: Add --ignore-docker-for-reuse option to assist workflow development.
8654: Rename tests/inp/ to test/input/
Merge branch 'master' into 8654-arv-jobs-cwl-runner
8654: Fix versionstring(). Improve help text / comments / style tweaks.
8654: Update test because input cwl files changed.
Merge branch '8654-arv-jobs-cwl-runner' closes #8654
Merge branch '8840-lock-job-record' closes #8840
8828: Move logic for checking $(which crunchrunner) into script that runs before invoking Docker on the compute node.
8828: Fix bind mount point for certificates.
Merge branch '8828-which-crunchrunner' closes #8828
Merge branch 'master' into 8857-cwl-job-reuse
8816: ComputeNodeUpdateActor._throttle_errors logs errors instead of re-throwing them.
8816: Handle cloud errors slightly differently from unrecognized errors.
Merge branch '8857-cwl-job-reuse' closes #8857
8816: Use is_cloud_exception to determine if exception is a "cloud error". Add
Merge branch '8816-compute-node-update-exception' close #8816
8712: Test case that reproduces cache-spill bug.
8712: Set self.collection = None when clearing the contents of a
8712: Propagate return value of clear() from super method. Test cache clearing
8799: Nodes with slurm_state are "down" are checked with sinfo and either reenabled or are valid for shutdown.
8799: Nodes in "drain" state are not automatically eligible for shutdown to
Remove over-quoting from crunchrunner and certificate volume mounts. refs #8893
7658: Clean up & handle subscription filters consistently across EventClient,
8799: shutdown_eligible() returns "node is draining" when in drain state. Add comments about iterating over cloud_nodes to check for "down" nodes. Fix tests.
Merge branch '8799-make-drained-nodes-idle' closes #8799
Separate out Go and Python dependencies into separate directories that don't
Add --help to test_with_arvbox.sh, no issue #
8931: Add exception handling around callbacks and PollClient api call.
Add args.ignore_docker_for_reuse=False to cwl-runner crunch script. refs #8857
Rename "rebuild" back to "reboot" and change "rebuild" to mean "build
8931: Add retry and exception handling around PollClient initial log query a well.
Propagate designated stdout stream from keepdocker.main() to put.main().
Add /var/lib/gopath and /var/lib/pip permissions fixup to Arvbox createusers.sh
8953: Node manager shutdown policy change WIP. Still fixing tests.
8953: Tests pass, with some removed due to removal of the corresponding behavior.
Fixup test_node_undrained_when_shutdown_cancelled and test_alloc_node_undrained_when_shutdown_cancelled.
8953: Add missing transitions.py
8953: Fix indentation of shutdown_eligible().
8953: Clarify how to use return value from consulting transitions table and shutdown_eligible().
8953: shutdown_eligible() returns a tuple. Report reason for shutdown decision.
8953: Assign to tuple (eligible, reason)
8953: Don't start shutdown on 'drng*' or 'alloc*'.
Merge branch '8953-node-manager-FSM' closes #8953
Don't try to drain node if no nodeename associated. refs #8953
Don't issue drain when shutdown has been cancelled. refs #8953
Don't double-count nodes that are shutting down. refs #8953
Merge branch '8953-no-double-count' refs #8953
Don't shut down if state is ('down', 'closed', 'boot wait', *) refs #8953
Arvbox run websockets in separate puma server instead of in API server process.
Pin bump cwltool dependency and pin version so it doesn't break again due to
Fix race conditions in test_node_undrained_when_shutdown_cancelled
8931: Use RetryLoop to retry api calls. Add max_wait to RetryLoop. Add test
Bump cwltool fpm package version to fix package builds. no issue #
8931: Use RetryLoop around websocket reconnect. Create a new _EventClient
8931: Fix indentation mistakes. Fix tests.
Merge branch '8931-event-thread-catch-exceptions' closes #8931
Fix arvbox only run pipelines with crunch-dispatch0 to avoid submitting duplicated jobs. no issue #
8998: Monkey patch URI.decode_www_form_component to validate efficiently.
Merge branch '8998-optimize-decode-www-form-component' closes #8998
Update version pin for cwltool fpm packages. refs #8653
9119: Use Oj strict mode for decoding JSON.
8886: Experimental asynchronous permissions update.
8886: Add timestamp checking to permission updates.
8886: Restore behavior in group_permissions to call
Merge branch '8886-async-permission-update' refs #8886
9161: There's a window between when a node pings for the first time and the
9161: Eliminate 'booted' list and put nodes directly into cloud_nodes list.
9161: Adjusting behavior to accomodate down/broken/missing nodes.
9161: Add _nodemanager_recently_booted as new way of remembering nodes which are in intermediate state between being created and showing up in the cloud node list.
9161: Put nodes tagged _nodemanager_recently_booted nodes back into the node list.
9161: Decisions to start and stop compute nodes are now based on an explicit
9161: Don't automatically consider nodes with job_uuid set to be 'busy'.
9161: Remove spurious prints
Merge branch 'master' into 9161-node-state-fixes
9161: Remove unused "paired()" function
8236: Add watchdog actor. This calls ping() on every other actor to check that
Merge branch '9161-node-state-fixes' closes #9161
Merge branch 'master' into 8236-nodemanager-watchdog
8236: Add comment to BogusActor.ping()
8236: Restore os.killpg(). Create a new process group so that it won't kill
Merge branch '8236-nodemanager-watchdog' closes #8236
9018: Change os.killpg() -> os.kill, don't create new process group.
Merge branch '9018-nodemanager-kill-instead-of-killpg' closes #9018
8653: cwl-runner crunch script rewrites keep file paths into CWL File objects.
8653: Update cwl-runner to match changes in sdk/arvados-cwl-runner
8653: Check that parameters are basestring before matching regex.
8653: Update load_tool in cwl-runner crunch script
8653: Set basedir for CollectionFsAccess
8653: Fix pathmapper API
8653: add cwlVersion so file validate correctly.
8653: Use load_tool.fetch_document() instead of Loader() to read raw document.
8653: Fix tests.
Merge branch '8653-cwl-runner-handle-files' closes #8653
9303: Watchdog kill node manager on any error
Merge branch '9303-kill-nodemanager-on-dead-actor' refs #9303
Log watchdog exception refs #9303
9303: Fetch arv_node before trying to shut down node, because monitor actor may
Merge branch '9303-actor-dead-dead' refs #9303
9187: Refactor dispatcher support into common library and update to use Locking API.
Arvbox installs binaries for go 1.6 instead of golang Debian package
9187: Check LockedByUUID on container updates and terminate status updates if
9187: Slurm dispatcher improvements around squeue
9187: Improve squeue synchronization
9187: Add documentation comments to Squeue functions.
9187: Fix comment typo
9187: Fix refactoring messup
9187: Remove "squeueError" because checkSqueue for a successful squeue run. Refactor tests a bit and add a test for canceling containers.
Merge branch 'master' into 9187-crunchv2-dispatching
Merge branch '9187-crunchv2-dispatching' closes #9187
9275: Initial pipeline/job component update from response
Merge branch '9275-cwl-runner-creates-jobs' closes #9275
Bugfix submitting cwl jobs with arvados-cwl-runner refs #9275
9187: If a container is reported Queued, but we are monitoring it, stop monitoring it.
9187: Don't try to take lock on containers with priority 0.
9187: Add comments.
Merge branch '9187-requeued-containers' closes #9187
9388: Process each notify individually instead attempting to batch them up.
9388: Record every log id sent and don't send duplicates.
9353: Add libcloud.common.BaseHTTPError to CLOUD_ERRORS.
8442: Enable net when API is enabled.
9353: Don't retry all BaseHTTPError, only retry InvalidInstanceID.NotFound.
9388: Only record log ids sent during "catch up" phase, to avoid duplicates.
9388: Add ws.notify_queue to ensure that notifies occurring during "catch up"
9388: Reset sent_ids for consistency
Merge branch '8442-crunch-run-enable-net' refs #8442
Merge branch '9388-websocket-every-notify' closes #9388
Add missing package 'less'. Also reduce crunch-dispatch-local poll interval to 3 seconds. no issue #
Use "less -R" in "arvbox log" so that color codes from api server logs display
Add missing arvbox package "sudo" refs #9408
8442: CWL create crunch2 containers WIP
8442: Add --crunch1/--crunch2 switch
8442: Submit containers Work in progess.
8442: Return PDH for Docker container. Working on setting up mount points.
8442: Setting up mount points works. Capturing output works.
8442: raise UnsupportedRequirement for unsupported features in the conformance
8442: Fix message
8442: Adding --submit support with --crunch2. General refactoring into more/smaller files.
8442: Debugging container --submit with crunch2
8442: import fixups
8442: more import fixups, import workflow file correctly.
8442: Tweak internal handling of keep: paths, examine exit codes to determine
8442: Submit CommandLineTool containers directly without an intermediate
8442: Generatefiles encode utf-8
9427: Websockets now uses a thread per connection for database query and
Bugfix "arv-ws --id" option, refs #9388.
9427: Add connection limit.
Add missing arvbox package "virtualenv" refs #9408
9427: Add limits for connections, subscriptions, queued notifications, and
Merge branch '9353-retry-http-error' closes #9353
8442: Rename --crunch1 and --crunch2 options to --api=jobs and --api=containers
8442: Fix tests. Update comments. Rename jobs -> processes since it doesn't hold just "jobs" any more.
9427: Back out redundant connection pool configuration option and add comment
8442: Bugfixes from end-to-end testing.
8442: cwl-runner crunch script must provide work_api parameter to ArvCwlRunner
8442: Make work_api parameter to ArvCwlRunner constructor optional.
Merge branch '9427-threaded-websockets' closes #9427
Increase timeout on failing websocket test. refs #9427
8442: Fix error reporting on missing Docker image
8442: Code cleanup. Remove dead code. Update comments.
Merge branch 'master' into origin-8442-cwl-crunch2
Merge branch 'origin-8442-cwl-crunch2' closes #8442
Update run-build-packages versions for schema_salad and cwltool, refs #8442
9488: Add log message to notify users on sucessful websocket reconnect. refs #9488
9442: Override kwargs["outdir"] so that $(runtime.outdir) is set correctly.
9442: Fix runtime.outdir for containers.
9442: Make cwl-runner crunch script backwards compatible (note-6)
9442: Incorporate check for correct $(runtime.outdir) into tests.
Merge branch '9442-cwl-runtime-outdir' closes #9442
9511: Bump cwltool dependency to get bugfix (81ff56f239de3ff0d17c8dec98d1a40f1e41cb7d) affecting bcbio run.
Merge branch '9511-cwl-type-checking' closes #9511
Update arvados-cwl-runner cwltool pin to get ruamel.yaml fix (c8f584840591f3a4a0fad3bce6b39f8866cd9dad) refs #9458
9369: arvados-cwl-runner documentation and examples, add deprecation notices to pipeline documentation
9369: tweaks
9486: Add InstanceLimitExceeded to is_cloud_exception
9570: Add support to crunchrunner for redirecting stderr to support CWL 1.0.
9570: CWL v1.0 support wip
9570: Pass tests except related to directory features
9570: Support Directory and file literal features. Support
9570: Fix for Pathmapper API change. Most tests pass or exit unsupported
9570: Fix secondaryFile staging to ensure that all files are in the same
9570: Update version pin and add reminder to setup.py to do it in the future.
9570: Set docker_outdir consistently.
Merge branch '9570-cwl-v1.0' closes #9570
Merge branch '9486-retry-instance-limit-exceeded' closes #9486
Fix tests. refs #9570
9369: Update tutorial to CWL v1.0
9369: Update CLI install documentation to include arvados-cwl-runner (& arvados-python-client)
9369: Add section about Docker. Change cluster references to qr1hi. Fix path to use /home/example.
Remove spurious print introduced in 6884eed, refs #9570
9570: Update cwl-runner for latest cwltool. Ensure use_container,
9570: Bump cwltool dependency
9570: Remove spurious print statement.
9570: Adjust Directory objects, too.
9570: Just add keep: to input keep locators in cwl-runner.
9570: Call normalizeFilesDirs to make sure basename is there.
9570: Use special path mapper for staging files to output dir.
9369: bwa-mem-input-local hashbang uses cwltool.
9570: Raise exception on workflow failure.
9369: Add note about Javascript eval timeout.
Merge branch '9570-cwl-runner-fixes' closes #9570 (again)
Merge branch 'master' into 9369-arv-cwl-docs
9369: Remove cwl install from Python SDK page. Remove "sudo -i" from
Merge branch '9369-arv-cwl-docs' closes #9369
Bump version of setuptools used by run_tests.sh for cwl-runner dependency refs #9570
9614: Don't use websockets for event notification until it is more reliable.
9674: InitialWorkDirPathMapper bug fix to set correct source path for files in
Merge branch '9674-cwl-initialworkdir' closes #9674
9552: If compute_checksum is true, check if checksum needs to be computed on final output, and add it if missing.
Merge branch '9552-compute-checksum-final-output' refs #9552
9751: Provide make_fs_access=CollectionFsAccess and bump cwltool pin.
9751: Fix direct to keep directory references.
9751: Override realpath in CollectionFsAccess
9751: Fix unit tests. Rename dir6.cwl to keep-dir-test-input.cwl
9751: Bump cwltool dependency.
9751: Fix handling local Directories
9552: import compute_checksums from cwltool.
Merge branch '9751-cwl-keep-dir' closes #9751
9615: Add check_writable to check for "writable" field and raise UnsupportedRequirement.
Merge branch '9615-cwl-writable-unsupported' refs #9615
9776: Ensure that $HOME and $TMPDIR are set correctly.
Merge branch '9776-cwl-HOME-TMPDIR' closes #9776
Fix schema salad package build version. no issue #
Fix failing tests. refs #9776
9783: Report useful error if subdirectory of a collection doesn't exist or isn't a
9820: Directly poll job or container records that we are interested in. Benefit: puts less load on database than log table polling, and doesn't miss events.
Merge branch '9820-cwl-poll-jobs' closes #9820
8232: Remove outdated/misleading Docker install
9827: Use "arvbox rebuild" to ensure up to date arvados/arvbox-demo image.
Add --compute-checksum to sdk/cwl/test_with_arvbox.sh refs #9552
Merge branch '9827-no-docker-caching' closes #9827
Test fixture for uncommitted container request.
9043: Expand "uncommitted" test fixture to include more optional parameter types. Add uncommitted_ready_to_run to test the "Run" button.
9043: Initial work on rendering input forms for CWL workflows. Works for
9043: Fixup incoming values from x-editable to have correct types. Work in progress.
9043: Complete support for editing basic parameter types.
9043: Initial commit testing for container request editing
9043: Remove non-existent "Graph" tab. Fix "false" selection dropdown. Use
9043: Use match_uuid_with_optional_filepath instead of hardcoded uuid regex.
9043: Handle errors on when trying to commit container.
9043: Add check that Run button is disabled before filling in final require
9043: Remove redundant ex_opt_string and unsupported ex_opt_file_array from test fixture.
Merge branch 'origin-9043-test-edit-container-request' closes #9043
9766: Supports packing workflow, setting defaults, uploading dependent files/docker images, creating/updating workflow record.
9766: Full workflow: create workflow using command line, select run from
9766: Add test for workflow create.
9766: Fix make_workflow to not modify input object. upload_dependencies
Merge branch '9783-cwl-error-invalid-dir' closes #9783
Merge branch 'master' into 9766-register-workflow
9766: Restore 'replication_desired': None
Merge branch '8232-docker-remove-obsolete' closes #8232
9850: Update cwltool to get javascript evaluation performance improvements.
Merge branch '9850-cwl-js' closes #9850
Fix tests refs #9850
Add README describing scripts in build/ no issue #
Add cachecontrol and cwltest to PYTHON_BACKPORTS for latest cwltool refs #9850
9923: Fix instanceof() to check correct base class
Merge branch '9923-cwl-collection-dir' refs #9923
9857: Allow any character in filenames.
9932: Add CWL best practices guide
Merge branch 'master' into 9766-register-workflow
9932: Copyedits
Merge branch '9932-cwl-style' closes #9932
9944: Add 'lockfile' to backports and update cwltool dependency
9944: Fix schema-salad 'lockfile' dependency to account for different package
9944: Tweak centos6 Dockerfiles to fall back to mirror if repoforge main is
Merge branch 'master' into 9944-cwl-lockfile-packaging
Merge branch '9944-cwl-lockfile-packaging' closes #9944
Don't pass work_api twice in ArvadosCommandTool constructor. Bump arvados-python-client minimum dependency because tests depend on it. refs #9944
Add -f argument to curl to exit nonzero and try mirror if primary
9001: Catch exceptions during log parsing and skip the line. Print log info
Don't use cached layers and always delete intermediate containers when building
Merge branch 'master' into 9766-register-workflow
Restore use of --build-arg COMMIT in arvados/jobs Dockerfile to select "cut"
10004: Add check_sinfo() to end srun_sync() if any of the allocated nodes are
10004: Add comment documenting reason why check_sinfo is needed.
Merge branch '10004-check-sinfo' closes #10004
10001: Update cwltool dependency to get parallel scatter fixes.
Merge branch '10001-cwl-scatter' closes #10001
9924: Mark all slots as "failed" on "Unable to confirm allocation" error.
9924: Tweak comment and remove superfluous variable assignment.
Merge branch '9924-revoked-working-slots' closes #9924
Update cwltool version, refs #10013
Fix typo, updated version on wrong package, refs #10013
Merge branch 'master' into 9766-register-workflow
9766: Rename metavar. Post packed yaml text instead of json. Adjust fixture
9766: Fix test to expect yaml text.
Merge branch '9766-register-workflow' closes #9766
100032: Add metrics to job submission in --debug mode.
Merge branch '100032-cwl-metrics' refs #100032
Fix misake in arvados-cwl-runner upload_dependencies refactor. refs #9766
Fix arvados-cwl-runner tests, refs #9766
10081: Submitting whole workflows as jobs WIP
10081: Quote expression interpolations. Set hints and requirements.
10081: Update cwltool version for bugfix.
10081: Add test for recursively calling cwltool when arv:RunInSingleContainer
10081: Custom extension of v1.0 schema to recognize RunInSingleContainer.
10081: Fix RunInSingleContainer test
10088: upload_dependencies reads raw files to find actual dependencies
10088: Bump cwltool dependency for bugfix to scandeps.
10088: Bump cwltool dependency for another bugfix to scandeps.
Merge branch '10088-raw-files' into 10081-cwl-run-same-job
10081: Mock Collection class in TestWorkflow
10081: Dump as yaml instead of json. Add test for extracted subworkflow.
10081: Fix quoting issue with workflow & input object documents. Improve test
10081: Upload dependencies. Fix keep paths. Ensure correct quoting.
Merge branch '10088-raw-files' refs #10088
Update cwltool version pin, refs #10088
10117: Consider both use_count and ref_count and check subdirectories to determine if it is safe to evict directory contents from cache.
10117: Additional refinement: a directory not in use but still referenced needs
10117: Restore original semantics of invalidate() and add code comment, add
10117: OrderedDict preserves insertion order, so use inode as key instead of cache_priority.
10117: Check that inode is in entries before trying to remove it, along with
10117: The check needs to be between clear() and the deletion from entries.
Update cwltool again for bugfix, refs #10088
Merge branch 'master' into 10081-cwl-run-same-job
10117: Tweak behavior of has_ref(). Fix naming of CacheTest.
10081: Rename cwl.input.json to cwl.input.yml, fix test.
10117: Revert mistaken change to Gemfile.lock.
Merge branch '10117-fuse-refcount' refs #10117
9988: Add arvados-specfic hints RuntimeConstraints and APIRequirement.
10088: Updated cwltool deduplicates dependencies to avoid creating separate
10081: Change logical identifier for Arvados schema to http://arvados.org/cwl.
Merge branch '10088-cwl-dedup-deps' refs #10088
Merge branch 'master' into 10081-cwl-run-same-job
Merge branch '10081-cwl-run-same-job' into 9988-cwl-arv-hints
Bump cwltool version in fpm_build refs #10088
Merge branch '10081-cwl-run-same-job' closes #10081
Fix typo.
Merge branch '9988-cwl-arv-hints' closes #9988
10081: Fix renamed methods.
Merge branch '10081-update-cwl-runner' refs #10081
8018: Add container_count field.
10129: Trim listing from Directory objects that are keep references, because
Fix #10133, use "resolved" (keep reference) instead of target (filesystem
10129: Ensure that directory literals get preserved when trimming keep directory listings.
10129: Rename del_listing to trim_listing and add docstring about what it is
Merge branch '10129-cwl-remove-listing' closes #10129
8018: Create new container if there are retriable container requests.
10151: Ensure that job_order_object is normalized and Directory listings are enumerated.
10032: Add more metrics
Merge branch '10151-cwl-dir-listing' closes #10151
Drill down on metrics on job submission.
10032: Update schema salad and cwltool packages to get document validation optimizations.
10032: Fix test to include --preserve-entire-environment
Merge branch '10032-cwl-spinup' refs #10032
10165: Always make an output collection when a workflow completes.
Merge branch '9857-cwl-acceptlist-re' refs #9857
10165: Add FinalOutputPathMapper. Delete basename/size/listing from output object because that information is captured by the enclosing Collection. Sort keys.
10165: DRY key cleanup.
10165: Handle copying complete collection contents into subdirectory.
10165: Pass around keep client object instead of creating new client per
10165: Add test for make_output_collection
Merge branch '9001-summary-skip-malformed-lines' closes #9001
8018: Fix state transition checks. Add test of retry and fix other tests to
10165: Add --output-name option to set name of output collection.
9308: Add 'keepTmpOutput' option to crunchrunner
Merge branch '10165-cwl-outputs' closes #10165
8018: Add migration script for "container_count" field.
9308: Add test for KeepTmpOutput flag.
9308: Set task.keepTmpOutput: true when outputDirType is keep_output_dir.
9307: Bump tests to use v1.0 and make sure that arvados_cwl.add_arv_hints() is called.
Bump schema-salad & cwltool version for cwl document validation bugfix. no issue #
9307: Improve crunchrunner error handling. Fix typo.
Merge branch 'master' into 9307-cwl-use-tmp-output
Merge branch '9307-cwl-use-tmp-output' closes #9307 closes #9308
8018: Additional assertions checking when container_uuid changes/does not change.
10186: Add support for "partition" in runtime_constraints. Add --all to squeue to ensure that it sees all paritions.
10186: Add PartitionRequirement
Enumerate Directory contents in output when --compute-checksum is
Bump cwltool version for bugfix https://github.com/common-workflow-language/cwltool/pull/206
Merge branch '8018-container-retry' closes #8018
10186: "partition" container runtime constraint takes array of strings instead
Merge branch '10186-crunch2-slurm-partition' closes #10186
10221: Always upload all dependencies up front. Ensures consistent reuse
10221: Add path mapper tests direct keep references, uploaded files, and keep mounted files.
10221: Don't depend on st.keepref from arvados.commands.run.statfile and uploadfile, use file_pattern to construct predictable keep reference.
10259: When using --submit with jobs API, create RunningOnServer pipeline
10259: Workaround for spurious "job_order" on command line. --no-wait returns
Merge branch '10259-cwl-pipeline' closes #10259
Merge branch '10221-cwl-pathmapping' closes #10221
10172: Implement /containers/current endpoint to get current container for API token.
10172: Allow auth_uuid to set container output. Work in progress.
10259: Add one level of indirection on dict parameters so that
10172: More work giving auth_uuid limited permission to set progress and output
Merge branch '10259-cwl-pipeline-value' refs #10259
10025: Rearrange arvbox layers for better caching. Embed build version to
10025: Use docker tag -f to change "latest".
10025: Add ARG arvados_version to .dev and .demo
10025: If $TAG starts with '-' it is not a tag.
10025: run-build-docker-images.sh can use "arvbox build" instead of "rebuild"
10200: Copy cwl-runner crunch script into arvados_cwl package to avoid version skew. Keep old cwl-runner code as a fallback to avoid breaking installations that haven't been updated.
10194: Record git version in package
10259: When creating pipeline to be managed by a-r-p-i, wrap value in dict for
10259: Update arvados-cli version pin
Merge branch '10200-cwl-crunch-script' closes #10200
Merge branch 'master' into 10259-cwl-pipeline-null
10259: Move trim_listing out of upload_instance
Merge branch '10259-cwl-pipeline-null' refs #10259
Fix API server version pin on arvados-cli. refs #10259
10194: arvados/jobs a-c-r pinning WIP
10194: Create job with pinned cwl version.
10194: Update test
10194: Add package test for arvados-cwl-runner. Symlink package tests so
10194: Package testing passes for debian7
10287: When creating a new account, if provided use 'username' from SSO server.
10172: Tests and related fixes for auth_uuid setting output on container.
10172: Detect when running as crunch container or job and set output.
Merge branch 'master' into 10172-crunch2-container-output
10172: crunch_script only updates task record on error, will normally be
10172: Bugfix getting container output when container is reused. Tweak output
10172: If RuntimeConstraints.API is set, refresh container record to check if
Merge branch '10025-arvbox-layers' closes #10025
10172: Add test for API: true and container setting output.
Check docker version to decide whether to use -f with docker tag, refs #10025
10172: Fix container request test. Replace
10194: Use SETUP_DIR for checking git version and writing _version.py. Add
Merge branch 'master' into 10194-cwl-version-skew
Merge branch '10194-cwl-version-skew' closes #10194
Add requests to ubuntu1204 backport lists, refs #10194
10194: Improve versioning of arvados/jobs image.
Merge branch '10194-jobs-image-versioning' refs #10194
10172: Add skip_before_filter :find_object_by_uuid for current. Typo fixes.
10172: Add container.output check to TestFullRunWithAPI.
Merge branch '10172-crunch2-container-output' closes #10172
10346: Checkpoint. Merge schema and method docs. Write doc describing syntax of HTTP requests.
10346: Work on resource fields.
10346: Document token scopes.
tag arvados/arvbox-base with latest so that FROM arvados/arvbox-base:latest works. refs #10025
10346: Checkpoint
Compute correct arvados-cwl-runner version, refs #10194
10346: Discuss execution. Start writing code examples.
10346: Document SDK examples uniformly
10346: Added UUID type to each resource. Some formatting cleanups.
10346: Start documenting how to access Keep.
10346: Document api_clients and trusted client.
10346: Add in a bunch of technical detail about Keep
Fix git log to get last log for sdk/cwl or sdk/python only, not repository head refs #10438
10448: Pass through enable_reuse flag to runner.
10448: Tests include enable_reuse flag and that --disable-reuse is passed to
Merge branch '10448-cwl-runner-disable-reuse' closes #10448
9849: Cache docker lookup.
9849: Use one regex instead of 3 when scanning log.
Use --net=host when running jobs inside arvbox to work around Docker-in-Docker
9849: Provide stub log to be parsed.
10346: Add a brief description about each resource.
10346: Add storage to sidebar. Tweak headings.
Check exit code before rethrowing error from catching SystemExit from
10461: Add test for fetching arvados_jobs image.
10461: Remove arv-keepdocker pull from test_with_arvbox.sh. Only install
Merge branch '9849-cwl-docker-lookup' refs #9849
Merge branch '9849-cwl-efficient-log-scan' refs #9849
Merge branch '10461-cwl-keepdocker' refs #10461
10460: Support for file and directory literals when constructing output
10460: Update cwltool dependency.
10460: Style tweak make_output_collection
Merge branch '10460-cwl-directory-literal' closes #10460
10460: Fix initialWorkDir file staging broken by change in visit() behavior for directories.
10466: Report missing 'location' in File object as workflow error.
10460: Add implied secondaryFiles based on input parameter spec.
10460: Don't use adjustFileObjs to set secondaryFiles because it traverses secondaryFiles.
10460: Want to propagate size fields to output.
10460: Add check for 'dockerOutputDirectory' and raise UnsupportedError for now.
10346: Edits from review WIP.
Merge branch '10460-cwl-staging-fix' refs #10460
10346: Document manifest format in more detail. Document token signatures. Add Arvados_permissions.svg
10346: Updates from review comments.
10346: Fix cross references
10410: Build Ruby from source instead using Debian package in order to install a more recent Ruby.
10346: Add slides about Keep and Crunch.
10346: Add session establishment diagram. Clarify role of block signatures.
10346: Fix formatting on container_request page.
Merge branch '10346-rearrange-api-docs' closes #10346
10410: Add -o Acquire::Retries and combine package installation into single layer.
Merge branch '10410-arvbox-ruby' refs #10410
10529: Improve handling errors raised when collecting outputs. If state
Permit use_existing in create-and-commit container requests. refs #10216
Merge branch '10529-cwl-exception-stuck' closes #10529
Arvbox sets PATH, GEM_HOME and GEM_PATH in common.sh. refs #10410
10498: Handle project_uuid of None in arv_docker_get_image.
Merge branch '10498-arvdocker-project-uuid' refs #10498
10583: Bump versions.
Merge branch '10583-cwl-salad-ver' refs #10583
Update cwltool version with compatible dependencies, refs #10583
Update arvados-cwl-runner with compatible dependencies, refs #10583
Install upgraded setuptools in arvbox, refs #10583
10630: Bump cwltool version to completely skip debug statements that make
Merge branch '10630-no-json-debug-dump' refs #10630
10599: Add run-build-packages-one-target.sh --only-build, run-build-packages looks for $ONLY_BUILD in its environment.
Merge branch '10599-run-build-packages' closes #10599
10647: Script + Dockerfile to build arvados/jobs image from local git checkout, for development.
10647: Image tag is most recent git hash of python sdk or arvados-cwl-runner.
10647: Get necessary version of cwltool from sdk/cwl/setup.py instead of
10293: Bug fixes
10293: Override workflow runner done() to get container from container request.
10293: Don't try to json decode empty output file to reduce spurious errors.
10293: Improve error messages for --api and unsupported InitialWorkDirRequirement.
10293: Avoid noisy and unhelpful error when cwl.output.json is missing.
10651: Add --submit-runner-ram to specify amount of RAM that should be
10293: Fix error message when wrong --api provided.
10651: Check --submit-runner-ram has a valid value. Add tests.
10293: Update test_done to test more stuff and remove redundant test_done_use_existing_collection.
Merge branch 'origin-10293-cwl-cr-output' closes #10293
Merge branch '10651-cwl-submit-runner-ram' refs #10651
10655: Add arvbash to tools/
10649: Make errors emitted by squeue and scancel show up in logs.
10658: Flush logs even when maintaining container in queued state.
10658: Put container into cancelled state on error during docker load or mount
Fix bug done.done() not returning output value. refs #10293
10519: Don't fiddlesticks on uncomitted container request page if workflow or
Update arvbox to work with passenger 5.0, refs #5737
Merge branch '10647-dev-jobs-image' closes #10647
Merge branch '10655-arvbash' closes #10655
Use regex to determine whether to print help text for arvbash.sh. refs #10655
Pin websocket-client backport dependency to 0.37.0 to avoid tertiary dependency
Bump package iteration on arvados-docker-cleaner for dependency fix. no issue #
10576: Working on fetch & url join for keep references.
10576: Tests & fixes for keep ref url join.
10576: Add resolver to execute from keep references and arvados workflow
10576: Running jobs from keep: and arv: prefixes WIP. Tests passing, needs some integration testing.
Merge branch '10649-squeue-stderr' closes #10649
Merge branch '10658-crunch-run-missing-files' closes #10658
Merge branch '10519-cr-fiddlesticks' closes #10519
10576: Bump cwltool and schema-salad versions.
10576: Bump cwltool, schema-salad and rdflib-jsonld package versions.
10674: Initialize defaults when creating container request from workflow.
Merge branch 'master' into 10576-cwl-keep-fetcher
10576: Fill in cwl.input.json as a "json" mount instead of creating a new collection.
10681: Add --name flag to set name on pipeline instances and container
10576: Fix tests for cwl.input.json part of container request record and not
Merge branch '10681-cwl-name-flag' closes #10681
10686: Specify explicit project when listing images to fix problem with default
Merge branch '10686-gce-list-image-project' refs #10686
10681: Apply --name flag to creating and updating workflows and pipeline templates.
10681: Add tests for workflow and template update. Refactor slightly to reduce repetition.
Merge branch '10681-cwl-wf-name-flag' closes #10681
10674: Extend pipeline/workflow creation test to check that parameter with default value is filled in.
10674: Add test checking that "Run" button is disabled because required inputs
Merge branch '10674-wb-workflow-defaults' closes #10674
10576: Set template_uuid when creating container request from workflow. Set
10576: Bugfix, set self.uuid on ArvadosContainer.
Merge branch 'master' into 10576-cwl-keep-fetcher
10576: Set name default to name of workflow when running single command line
10576: CWL tutorial documentation refactor WIP.
10587: Print arvados-node-manager package version on startup.
Merge branch '10587-node-manager-report-version' refs #10587
Nodemanager also logs libcloud package version, refs #10587
10576: Add CWL extensions page.
10576: Integrate CWL extensions page, reorganize running and developing docs.
10576: Add retries to arvcontainer. Add command line options to documentation.
10576: Add ability to run pipeline templates containing CWL from command line.
Merge branch 'master' into 10576-cwl-keep-fetcher
10576: Fix formatting in cwl-run-options. Fix dangling refs tutorial-pipeline-workbench.html
10576: Tweak Docker docs and add pipeline deprecation notice.
Merge branch '10576-cwl-keep-fetcher' closes #10576
Bump libcloud for ex_get_license() bugfix. refs #10686
Bump schema-salad version to fix rdflib-jsonld dependency. refs #10576
10684: Add Arvados-specific search path to Python SDK arvados.util.ca_certs_path.
10684: Add Arvados-specific search path to Go SDK TLSClientConfig.
10684: crunch-run adds /etc/arvados/ca-certificates.crt to binds, updated tests.
10684: Only bind mount certificate when API access is indicated.
10684: Refactor into MakeTLSConfig(). Only look for
10684: Add -ca-certs option
10684: Don't overwrite default certs search path unless -ca-certs provided.
Fix CWL tests refs #10576
Merge branch '10684-crunch-run-ca-certs' closes #10684
Fix crunch_script.py to pass name arg, refs #10681
Fix dev build of arvados/jobs image to avoid installing versions of
Adjust arvados repo minimum script versions for job dispatch. no issue #
Fix CWL tests affected by minimum_script_version tweaks. no issue #
10497: crunch-run records logs even when container is cancelled.
Merge branch '10497-crunchrun-logs' refs #10497
Arvbox use --local to install bundler if gem is available, also recognize .tar.bz2 in
10497: Update cwltool & schema salad versions.
10497: Add source line reporting to errors and fix tests to work with CommentedMap/Seq behavior.
10497: Improve up front feature checking and error reporting.
10497: Use SourceLine in context manager in PathMapper. Remove unnecessary import.
10497: Update ruamel.yaml, schema salad, and cwltool
10497: Pop cwl:tool parameter from job order
10497: Add pathlib2 to backports
Arvbox common.sh script won't fail if bundler gem not cached on disk.
Merge branch '10497-cwl-better-errors' refs #10497
Fix crunch script to construct toolpath before it is turned into a File object. refs #10497
10782: Bugfix, on retry only update container_uuid on container requests
Merge branch '10782-container-retry' closes #10782
10497: Print last 25 lines of log when job fails.
10497: Add --no-log-timestamps to eliminate redundant timestamps when running in Arvados job or container.
10497: Don't prefix every line of log from job.
10497: Increase number of log lines shown by runner. Turn off 'debug' mode on crunch script.
10497: Filter crunchstat lines to make more space for useful logs.
10497: Add logname (crunch-run, stdout, stderr) to container log tail
10722: Issue better error when 'id' is missing.
10497: Replace \d\d\d\d with \d{4}
Merge branch '10722-cwl-subworkflow' closes #10722
Merge branch '10497-cwl-job-logs' closes #10497
10790: Increase timeout reading from arv-put.
Merge branch '10790-crunch-job-log-timeout' refs #10790
10812: Fix check for collection reuse, same PDH and similar name.
10869: Pass CollectionFetcher to load_tool in crunch_script so that it will recognize keep: URIs.
Merge branch '10812-similar-collections-check' refs #10812
Merge branch '10869-cwl-keep-ref' refs #10869
10812: Update arvados-cwl-runner dependencies. Remove pathlib2 from backports.
Merge branch '10812-cwl-package-update' refs #10812
10793: Bump cwltool pin for workflow engine improvements. Propagate on-error
10793: Return output, status tuple from executor.
10793: Move output_callback to finally block to ensure it is always called.
10793: Fix to return tuple for alternate exit points --create/update-workflow and --no-wait
10793: Ensure on_error is set if not present in job_order_object.
Merge branch '10793-cwl-continue-on-error' closes #10793
Create tmp/cache dir after cleaning old one.
Merge branch 'arvbox-tmp-cache' no issue #
10881: Add python-futures as explicit cwltest dependency and bump iteration.
Merge branch '10881-build-python-futures-package' refs #10881
Bump libcloud pin to 0.20.2.dev3 for Azure China support. refs #10833
10292: Update cwltool for bugfix: intermittant workflow deadlock on subworkflow completion.
Merge branch '10292-cwl-deadlock' refs #10292
10935: Make Rails method deep_munge a noop, so that empty arrays can round trip
10812: Add --submit-runner-image to override default Docker image when submitting workflow runner.
10935: Add test that round tripping works after disabling deep_munge.
10812: Perform merge of logs based on timestamp instead of sorting log lines.
10812: Add tests
Merge branch '10935-noop-munge' closes #10935
Merge branch '10812-submit-runner-image' refs #10812
Merge branch '10812-log-merge' refs #10812
10812: Use packed workflows for all run modes.
10812: Handle workflow keep references.
10812: Don't try to upload embedded tools separately unless they have an
10812: Improve check that already packed workflow collection exists with same
10812: Fix imports, reference #main in cwl:tool
10812: Check for dockerOutputDirectory and raise UnsupportedRequirement up front.
10812: Don't use cStringIO which doesn't like unicode strings.
10812: Handle $schema references.
10812: Update cwltool and schema-salad dependencies.
10812: Fix indentation typo.
10812: Add num_retries to CollectionFetcher and collectionResolver.
10812: Handle expected NotFoundError error when checking if file exists in keep.
Merge branch '10812-cwl-separate-dependencies' refs #10812
10895: Improve reuse for RunInSingleContainer jobs
10895: Don't add uploaded files to pathmap if they are not referenced.
10895: Don't report "reused" for Failed/Cancelled containers. Tweak log
Merge branch '10895-run-in-single-container-reuse' refs #10895
10812: Fix bug and add test that docker dependencies are found and uploaded.
Merge branch '10812-docker-upload-fix' refs #10812
arv_docker_get_image should not modify dockerRequirement parameter. refs #10812
Bump schema-salad version for schema loading performance fix. no issue #
10846: Remove duplicate log message suppression by ComputeNodeMonitorActor
10846: Revise shutdown behavior.
9397: Add manifest normalization and sub-manifest extraction by path.
9397: Move most functions into private API.
9397: Use idomatic loop form.
10846: Specify whether to try to resume the node when cancelling shutdown
9397: Remove conflicting code for easier merge
Merge branch '9397-go-manifest' into origin-9397-prepopulate-output-directory
9397: Use ManifestTextForPath
9397: Fix collection.go to use blockdigest.LocatorPattern
9397: Fix collection.go to use blockdigest.LocatorPattern
9397: Fix major bug in firstBlock(). Refactor ManifestTextForPath() to
9397: Update method documentation for Extract()
9397: Tweak method documentation for Extract()
9397: Update comment & tests for CollectionFileReader to reflect it is more
Merge branch '9397-go-manifest' refs #9397
Merge branch 'master' into origin-9397-prepopulate-output-directory
9397: Adjust stream normalization to use just digest (no hints) to de-duplicate
9397: Fix bind behavior to include Path. Update tests to check container binds
Merge branch '10846-nodemanager-errors' closes #10846
Merge branch '9397-prepopulate-output-directory' refs #9397
Update cwltool pin. closes #10993
Update schema-salad pin. refs #10993
9397: arvados-cwl-runner implementation of InitialWorkDir on container API.
9397: Fix keep URI splitting. Fix error when container has error log
9397: Improve container InitialWorkDir test.
Merge branch '9397-cwl-initialworkdir-crunchv2' closes #9397
Fix arvbox build.
Merge branch 'fix-arvbox' no issue #
Set docker_image_formats to v1.
Update test_with_arvbox script.
Merge branch 'arvbox-docker-imgfmt' no issue #
6520: Call squeue to include slurm jobs in wishlist. Call sinfo to update
6520: Add testcase getting wishlist from squeue.
6520: Add testcase using sinfo to set crunch_worker_state
6520: Assign weight to node based on price to preferentially schedule on
6520: Update cloud_node.size to match server calculator configuration.
6520: Apply node size fixup in CloudNodeListMonitorActor
6520: Check that hostname is set before setting weight.
6520: Don't ever assign compute0, start at slot 1 when assigning hostname.
9277: crunch-run Log and output collections set is_trashed to true instead of
9277: Include trashed collections when creating copy collection for container
6520: Explicitly mark node as 'down' before shutting down.
Merge branch '9277-trash-container-outputs' closes #9277
6520: Expand recognized 'busy' states.
Merge branch '6520-skip-compute0' refs #6520
6520: Crunch v1 recognizes 'mix' and 'drng' as busy states for
Merge branch '6520-nodemanager-crunchv2' refs #6520
6520: Add ReqNodeNotAvail to list of reasons (along with "Resources") to boot a new node.
Merge branch '6520-pending-reason' refs #6520
10629: improve debug logging
10629: Make tracking and dirtying of _committed flag efficient.
10629: Don't flush dirhandles.
9277: Container output check must be unscoped to include trashed collections.
9277: arvados-cwl-runner sets "is_trashed" when directly setting output of container.
9277: Add test that setting trashed, unreable collection is disallowed.
Merge branch '9277-container-output' closes #9277
Merge branch '10629-fuse-listing-perf' closes #10629
Arvbox now uses Go websockets server instead of Puma. no issue #
Add ability to use "arvbox start test" to re-run tests without tearing down &
11226: Disable google api client discovery doc caching, use only httplib2 caching.
Merge branch '11226-discovery-doc-cache' closes #11226
Bugfix: python Collection class sets _portable_data_hash in _populate_from_api_server
8567: find_all_for_docker_image() returns only Docker images compatible with Rails.configuration.docker_image_formats. Follows migration links.
8567: Add & tweak tests for selecting compatible Docker image format.
8567: Refactor code that queries migration links into get_compatible_images.
8567: Adjust job container resolve test now that images are filtered based on
8567: Tests default to image format v1 to avoid breaking all the tests that
8567: Use Docker image repo+tag name instead of PDH so that API server can select correct image format.
6520: Node manager docs WIP
6520: Add information about setting up SLURM to crunchv2 documentation.
6520: Add node_mem_scaling to documentation.
6520: Add page with sample ping script. Improve instructions on creating
8567: If a search_term looks like a PDH, always treat it as one. Also use
8567: Fix tests now that container_image provides docker repo+tag and not PDH.
8567: Docker image migration WIP.
8567: Creates migration links.
8567: Add status reporting to migrate script.
8567: Move out of tools/ into sdk/python and docker/docker19-migrate.
8567: Rename docker19-migrate to migrate-docker19 for consistency with
8567: Add check for ARVADOS_API_HOST_INSECURE
8567: Fix migrate links to use PDH instead of UUID.
8567: Add docker19 migration instructions to install guide.
8567: Add note about updating API server configuration.
Merge branch '6520-nodemanager-docs' closes #11123
Merge branch '8567-cwl-docker-img' refs #8567
Merge branch '8567-api-select-docker-fmt' refs #8567
Add missing documentation file. refs #6520
8567: Add check that admin token is used and ensure that migration links are
11254: Cloud nodes where "actor is None" are considered to be in shutdown. The
11254: Refactor _node_states
Merge branch '11254-nodemanager-no-actor' closes #11254
11288: Slurm requires reason to put node in DOWN state.
Merge branch '11288-slurm-down-reason' closes #11288
8567: Better command line UI
8567: Add migrate19
8567: arv-keepdocker catches EPIPE so it doesn't print backtrace when piped to
8567: Bugfix check for existing repo+tag links.
Merge branch '8567-docker-migrator' refs #8567
If API server doesn't publish supported Docker image versions, make that an
Fix test broken by change of default behavior in arv-keepdocker. refs #8567
8567: Use vfs Docker graph driver for maximum compatability. --dry-run reports
8567: Make sure /root/.cache/arvados/docker is tempdir volume mount as well.
8567: Add start_docker and kill_docker functions with wait loops.
11255: Add -container-enable-networking and -container-network-mode options.
11255: Add documentation about configuration to enable host networking always.
11255: Fix whitespace
Merge branch '11255-docker-host-networking' closes #11255
11325: Remove "broken node" check. Assume if the node really isn't
11324: Fix crash in NodeManagerDaemonActor when receiving a node_can_shutdown
11323: Don't try to offer_arvados_pair on unpaired nodes which are being shut down.
Merge branch '11325-no-broken-nodes' refs #11323, refs #11324, refs #11325
Merge branch '8567-docker-migrator' refs #8567
10401: Support in arv-run for uploading Directories.
Merge branch '10401-run-upload-dir' refs #10401
Update cwl test_submit refs #10401
11332: Override relaxed path checks to be even more relaxed, and bump cwltool &
Merge branch '11332-cwl-filenames' closes #11332
11332: Suppress phone-home behavior in check_exists by recognizing
11333: Add explicit version pin on 'typing'
Merge branch '11333-cwl-phone-home' closes #11333
Fix packaging for python-arvados-cwl-runner refs #11333
Fix invalid records in API collections test fixture and improve error message
10401: Use use_custom_schema feature for extensions.
10401: Allow Arvados extensions to be used in "requirements" not just "hints".
10401: Rework support for uploading Directories (don't assume files are
10401: Use custom_schema_callback. Bump schema-salad and cwltool versions.
10401: Bump arvados-python-client version for updates to
10401: Only upload file:// identifiers.
10401: Fixup file/directory literal handling
10401: Update crunch script, no longer expands directories by default.
10401: Bump schema salad for self-colliding id fix.
10401: Bump cwltool for packing fix.
11332-fix-crunchscript: Set ACCEPTLIST_RE because crunch_script alternate entry
8567: Improve disk space accounting. Default to use overlay storage driver (but user can override).
Merge branch '11332-fix-crunchscript' refs #11332
9132: WIP switching to using official Docker Go client.
9132: few more updates to use the new API
9132: Fix after rebase
8567: Use arvados.config instead of os.environ. Tweak disk space accounting output.
8567: Log total bytes to migrate
9132: Fix ContainerCreate to include HostConfig, container UUID as name.
9132: Fix test errors due to rebase.
Merge branch '9132-dockerclient' closes #9132
8567: Add workaround for "trap EXIT" in pid 1 bash process. Add comments.
11407: keepclient checks for ARVADOS_DEBUG and enables debugging.
Merge branch '11407-go-sdk-debug' closes #11407
11413: Fix issues with node manager on GCE:
11413: Wrap destroy_node with similar logic to create_node: on exception check
11413: Use getattr() in exception handler.
Merge branch '11413-nodemanager-gce-fixes' refs #11413
11423: Fix bug checking existence of Directories which refer to root of keep collections.
11423: Restore check to use pdh_path because it is only supposed to match Files
Merge branch '11423-cwl-collection-dir-ref' closes #11423
Add missing test file refs #11423
Add check that gnutls.h is available so pycurl can compile. no issue #
Increase ping delay in WatchdogActorTest to try and reduce spurious test failures. no issue #
11237: Bump google-api-python-client to 1.6.2 for better compatability across
11237: Don't build oauth2client on ubuntu1604
11237: Tweak version dependencies.
Merge branch '11237-google-api-python-client' refs #11237
8465: Set proper container flags to provide data on stdin.
11332: Use urllib quote/unquote to handle special chars (e.g. '#')
11332: Add @ to quote safelist.
Merge branch '11332-cwl-url-quoting' refs #11332
8465: arvados-cwl-runner support for stdin and stderr redirection with containers.
Merge branch 'master' into 10401-limit-dir-expansion
10401: Get structure.sql from master.
Build SSO packages for Ubuntu 1604. refs #11457
cwl/test_with_arvbox.sh pulls arvados/jobs:latest refs #11457
Merge branch 'master' into 10401-limit-dir-expansion
8465: Add unit test for generating container request with std* redirection.
10401: Add integration tests for directory listing behavior.
10401: Document LoadListingRequirement in Arvados extensions.
Merge branch '8465-cwl-containers-stdin-stderr' closes #8465
8567: Migrate unnamed images
Merge branch '10401-limit-dir-expansion' closes #10401
Merge branch '8567-moar-docker' refs #8567
11493: Bump cwltool version for relaxed dependency on 'six'
Merge branch '11493-cwl-six' refs #11493
11462: Store CollectionReader objects in a central cache to avoid redundant
11462: Bump cwltool version for mergedirs() fix
11462: Add missing test file.
11462: Runner asks for 3000 MB of RAM by default instead of 1024.
Add missing test file. refs #10401
11462: Fix crunch script for fetcher_constructor constructor change.
Fix regression, restore a line that shouldn't have been removed. refs #10401
11462: Log ArgumentError in make_output_collection. Add test for CollectionCache.
Merge branch '11462-cwl-cache-collectionreader' refs #11462
11510: Add support for sparse writes to ArvadosFile.
11510: Fix longstanding bug in replace_range() when appending data from a buffer that was written out of order.
11510: Repack writes any time there's more than one segment referencing the same bufferblock.
11510: Fix truncate() to not change file pointer. Additional tests.
11510: Tweak interface to better match Pythons "io.IOBase" class.
Merge branch '11510-sdk-extend-files' refs #11510
11507: Test cases that reproduce bug
11507: Put bufferblocks into DELETED state when deleted so they don't get
11507: Cleanup
11507: More small block packing fixes:
11507: Put self._pending_write_size update back under the lock.
11507: Move repack_writes() method from ArvadosFile to BufferBlock.
Merge branch '11507-deleted-bufferblocks' refs #11507
11549: Test that containers don't mount each file individually, instead mount
11549: Fix container requests so it doesn't mount each file individually,
Merge branch '11549-cwl-smarter-mounts' closes #11549
8019: Rework partial line throttling. Fix sending flush when buffer is ready
11469: Docker-managed volumes go in "Volumes" not "Binds".
Crunch script doesn't fail in file/directory literals that don't have a 'location'. refs #11549
Merge branch '11469-volume-tmp' refs #11469
Merge branch 'master' into origin-8019-crunchrun-log-throttle
11369: RAM request includes keep cache RAM. Add request for temp disk.
11369: Containers API: Add capacity to tmp and output mounts. Also support {outputDirType: keep_output_dir} of RuntimeConstraints.
11369: Log node sizes at startup. Fix setting of scratch disk size.
11369: Fix tests. Tweak test fixture to include a mounts entry with capacity.
11469: All {kind: tmp} mounts set up under host tempdir, don't try to use
11469: Add test for changed tmp mounting behavior.
Merge branch '11369-cwl-crunch2-capacity' refs #11369
Merge branch '11369-dispatch-mem' refs #11369
Merge branch '11469-volume-tmp' refs #11469
Merge branch '11369-log-sizes' refs #11369
8019: Make flush channel buffered again (because we want to be able to do back-to-back flushes).
10645: Tweak so that left column is col-md-6 (instead of col-md-5). Also tweak
Add libgnutls28-dev and python3-dev to install package list to satisfy
11626: Log sbatch errors where the user can see them. Fix tests so there is no lingering goroutines.
10827: Add --validate, --print-dot and --enable-dev options (pass through to
10827: Bump cwltool & schema salad dependency.
Merge branch '10827-checker-cwltool-update' refs #10827
11369: Add keep_cache to base RAM request.
11626: Fix log.Printf -> log.Print
Merge branch '11626-crunch2-log-slurm-errors' closes #11626
Merge branch '11369-cwl-keepcache-ram' refs #11369
11681: Return native str type from Collection.portable_data_hash() (previous commit was actually for CollectionWriter)
Merge branch '11681-str-newstr' refs #11681
11693: Mark container as cancelled if there is an error during finalization.
11693: Dereference symlinks to input files and add incorporate them into output collection manifest.
11693: Add tests.
11693: Add test cases for symlinks to directories and symlinks inside subdirectories.
11693: Add test that error capturing output results in cancelled container.
11693: Detect and error on symlinks pointing to locations outside the output directory.
11693: Only check collection mounts when determining whether to call
Merge branch '11693-crunch2-output-symlinks' closes #11693
11718: Update crunch-run for docker client API change in ContainerWait().
Merge branch '11718-crunch-run-docker-wait' closes #11718
11714: Set CgroupParent under Resources because setting Cgroup in HostConfig
Merge branch '11714-crunch-run-cgroup-parent' closes #11714
11369: Add migration notes about crunchv1-to-crunchv2
Merge branch '11369-crunchv2-notes' refs #11369
11100: a-c-r sets output_ttl and deletes intermediate collections on success.
11100: Propagate through to runner. Use intermediate_output_ttl consistently.
11100: Implement & document arv:IntermediateOutput hint.
11100: Update/add tests for --intermediate-output-ttl
11543: arvados.command.run.uploadfiles takes optional Collection to upload to.
Merge branch '11543-uploadfile-collection' refs #11543
11543: Upload tool dependencies into single collection. Add test for collection per tool. Fix other tests.
11543: Bump version dependency on arvados-python-client.
11100: Separate "trash intermediate on success" behavior from "output intermediate TTL" option. Update documentation.
Merge branch '11543-collection-per-tool' closes #11543
11100: Add test for --trash-intermediate. Add log message when intermediate
11100: Add ciso8601 dependency
11100: Fix test
Merge branch 'master' into 11100-cwl-set-output-ttl
Merge branch '11100-cwl-set-output-ttl' closes #11100
11767: Correctly parse values with unit suffixes printed by squeue.
11767: Recognize lowercase suffixes, just in case future versions of slurm
11767: Slurm apparently will print out half values (like 2.5G).
11767: Test cases that it will convert fractional values from squeue.
Merge branch '11767-slurm-units' refs #11767
11769: HasUuid::UUID_REGEX matches anything that looks like an Arvados uuid. As a result, if crunchv1 and crunchv2 dispatchers are on the same cluster, crunch-dispatch.rb will try to scancel containers thinking they are "orphan jobs". Tighten the regex to only match job uuids.
Fix crunch script to set trash_intermediate and intermediate_output_ttl refs #11100
11766: Bump cwltool version for deadlock fix.
Merge branch '11769-scancel-jobs-only' closes #11769
11767: Make squeue format output pipe (|) delimited so that it doesn't get
Merge branch '11767-squeue-reasons' refs #11767
Merge branch '11766-workflow-deadlock' closes #11766
10312: Identify error message that look like we are hitting a quota or account limit. Set soft node quota in order to stop trying to boot new nodes until the total node count goes down. Probe node quota upward when at the soft limit and able to boot nodes successfully.
10312: Adding ability to substitute fake libcloud driver but run full node manager for integration testing.
10312: Integration test framework for node manager, runs full node manager with
10312: Tests pass for booting single node, multiple nodes, hitting quota, quota
10312: Fix unit tests.
10312: Add services/nodemanager-integration to test list
10312: Add some comments to node manager integration test.
10312: Add example jobs_queue and slurm_queue options to example node manager configurations.
10847: Daemon shutdown now stops most actors, only waits for setup actors.
10847: Don't log everything from node manager unless test fails.
11345: Simplify and consolodate retry for API throttling errors.
11345: Fix unit tests after refactoring error types.
11345: Fix race-prone test test_issue_slurm_drain_retry
11642: Increase arv-put retries. Job considered failure if unable to save job log.
11642: Don't reuse completed jobs where output or log is null or not readable by current user.
11345: Clamp retry-after to (0, max_retry_wait). Deindent retry_wrapper a bit for readability.
11642: Jobs where log is unreadable (but not null) are not offered for reuse
Merge branch '11642-null-job-log' refs #11642
Merge branch '10312-nodemanager-quotas' refs #10312
Merge branch '10847-nodemanager-shutdown' refs #10847
Merge branch '11345-nodemanager-retry-after' refs #11345
Fix integration test, refs #10312
Fix integration test again, refs #10312
Remove node manager integration test wait check for quota set (event ordering), refs #10312
Ensure that /var/cache/apt/archives is present so arvbox apt-get will work. no issue #
Remove apt-get clean when starting frorm bare debian image. no issue #
Add --compute-checksum to test_with_arvbox.sh command line extras refs #11458
test_with_arvbox.sh tags selected arvados/jobs version as latest refs #11458
test_with_arvbox determines the correct arvados/jobs image to pull by querying
8186: Add EBS storage if necessary so there is sufficient scratch space.
8186: Set virtual device path. Require libcloud 0.20 because 0.18 has mistakes in the ec2 instance size table.
8186: Round up ec2 ebs volume sizes. Ensure size is in valid range. Add test for creating ec2 instance with ebs storage.
8186: Add comment/warning about EBS volume size limit.
Merge branch '8186-nodemanager-ebs' closes #8186
11896: Add integration test coverage for gce and aws drivers. Handle disk=None from gce driver.
Merge branch '11896-gce-no-disk' closes #11896
arvbox does pip_install wheel so ruamel.yaml won't complain. no issue #
arvbox docker-in-docker crunch-run uses "host" networking mode to reduce
11095: Add arv:ReuseRequirement hint. Update tests & documentation.
Merge branch '11095-cwl-control-reuse' closes #11095
Add missing test file for noreuse.cwl. refs #11095
8624: New behavior for by_tag/ dir. Allow 'cd' into any tag directory that
Quote subshell expression in test_with_arvbox and make container reset more robust.
8624: Move check that tag exists from __contains__ to __getitem__ method.
Merge branch '8624-fuse-tags' closes #8624
10584: When evicting a Collection from the cache, ensure that get/put worker
Merge branch '10584-fuse-stop-threads' refs #10584
arvados-cwl-runner always submits runner job when invoked with --no-wait refs #9086
Update cwltool & schema-salad and fix for changes.
11948: Strip out default values which reference unavailable files.
11948: Fix unit tests for additional file checking and nameroot/nameext fields.
11948: if -> elif style fix
12037: Support CWL v1.0.1 spec.
12037: Fix tests.
11158: New behavior for project directories. Allow 'cd' into any collection or
11158: Fix order="modified_at".
11158: Restore paging to get full list of collections and projects (but still
11158: Improve arvados.util.list_all() implementation to avoid relying on
11158: Fixes & test updates for ProjectDirectory.
Revert python sdk changes.
11158: Support incremental update of project contents based on websocket events.
11158: Handle trash/untrash events for collections in projects.
12037: Pull default arvados/jobs:latest image as needed.
Merge branch '11158-fuse-projects' refs #11158
Merge branch '12037-cwl-v1.0.1' closes #12037
11545: CWL stress test for crunch / node manager
12057: Add "Priority" to list of reasons to boot a new node.
11925: Make watchdog test more reliable.
11925: Fix flaky tests in test_computenode_dispatch_slurm.
11925: Update dependency on arvados-python-client to fix
Merge branch '12057-nodemanager-priority' closes #12057
11925: Make unit tests more reliable.
11925: Tweak test_node_pairing
11925: Explicitly join actor thread after stopping.
11925: Add check for paired nodes, not just monitored nodes.
11925: More tests wait for expected number of paired nodes.
11925: Two more tests wait for expected number of paired nodes.
12093: Propagate --project-uuid to runner job inside container.
Merge branch '12093-cwl-project-uuid' closes #12093
Merge branch '11925-nodemanager-watchdog-test' refs #11925
12100: Bump cwltool for v1.0.2 secondaryFile handling.
Merge branch '12100-cwltool-update' closes #12100
Update test to reflect slight change in cwltool packing behavior. refs #12100
Merge branch '11545-stress-test' closes #11545
Bump cwltool for python-requests dependency fix refs #12100
arvbox installs nodejs and sets up workbench js assets. refs #12033
Update cwltest package to latest so conformance tests pass no issue #
Bump cwltool to eliminate nameext/nameroot in output. refs #12100
12234: Update arvbox to Debian stretch, Docker 17, Postgres 9.6
12186: Filter nameroot/nameext from RunInSingleContainer and update test.
12186: Filter nameroot from runner-submitted input object.
10349: Working on Python code snippits for manipulating container requests.
10349: Add "Create a collection sharing link"
10349: Add snippts for listing inputs/outputs of containers, failed child
12234: Need --system so pip installs SDK packages globally.
10349: Fix formatting. Add link from user guide to improve discoverability.
Merge branch '12234-arvbox-stretch' refs #12234
Fix arvbox-demo refs #12234
10349: Add code snippet for combining collections.
10349: Add support for syntax highlighting inline code blocks.
Merge branch '10349-cookbook' closes #10349
Merge branch '12186-cwl-no-nameroot' closes #12186
Fix arvbox-demo (2nd try) refs #12234
12248: Bugfix reversemap.
Merge branch '12248-crunch2-reversemap' closes #12248
12032: Benchmark for group permissions.
12032: Use permission_view in subquery to filter objects readable by user.
12032: Additional refactoring of readable_by. Refactor "trashable" into module.
12032: Tests for group delete behavior.
12032: Fix hiding trashed groups directly owned by user.
12032: Add test fixtures
12032: Adding controller tests
12032: Controller support for group trash.
12032: Log.readable_by uses permission_view
12032: Refactor readable_by to minimize subqueries.
12032: Test for ensure_unique_name when untrashing groups.
12032: Bug fix so include_trash still respects permissions.
12032: Add copyright notice
12032: Make queries to groups table a special case, streamlines permission logic.
12032: Fix owner_check to include user_uuids and target_owner_uuid
12032: Ensure that permission_view is invalidated & refreshed
12032: Added migration for materialized view
12032: Update structure.sql
12032: Add comments to migration. Also special case api_client_authorizations
12032: Controller & integration test teardown invalidates permission view.
12032: Rename permission_view and remove Rails permission cache.
12032: Add missing files.
12032: Update install guide to specify minimum Postgres version of 9.3
12032: Restore create_permission_view.sql with note that it is only
12032: Fix down migration.
12032: Database reset used by integration tests refreshes permission view.
12032: Container readable check uses subquery instead join to avoid returning
12032: Update structure.sql from master schema + migration only.
12032: Remove startnode from materialized_permission_view (again), tests pass
12032: Refresh materialized_permission_view immediately after migration.
12032: Remove hopefully-unnecessary view refreshes between tests.
12032: Remove unnecessary checks on user_uuids.
12032: Remove user_uuid special case from all_perms.
12032: Don't need to refresh permission view in database seeds, should be done already.
12032: Add permission_refresh_lock
Arvbox support for caching and offline install of npm packages. no issue #
Remove -f from 'docker tag' due to Docker update refs #12234
Fix quoting in test_with_arvbox no issue #
Arvbox explicitly initializes postgres with en_US.UTF-8 locale. no issue #
Bump PhantomJS version no issue #
12292: Set defaults for tmpdir, outdir, keep
12292: Find crunchrunner vars even when mixed into a partial line
Install upstream pip to work around Debian bug, no issue #
Merge branch '12292-cwl-tmpdir-default' refs #12292
11068: Catch OSError and give better error message when unable to execute Docker.
12276: Reduce number of spurious invalidations sent to kernel.
12276: Remove debug logging message.
Merge branch '12276-fuse-deadlock' refs #12276
Merge branch '11068-cwl-missing-docker' closes #11068
Merge branch '12032-project-trash' refs #12032
12276: Reduce number of spurious invalidations sent to kernel.
12276: Remove debug logging message.
12287: Add number round-trip test.
Merge branch '12287-preserve-json-numbers' closes #12287
12246: Make "possible causes" message easier to find and read. refs #12246
12278: Propagate --debug flag to jobs.
12278: Can provide custom schema salad to dev jobs build.
12278: SourceLine yields stack trace if debugging enabled
12278: Filter nameext and nameroot from final cwl.output.json
12278: Bump schema-salad and cwltool for SourceLine fix.
12278: Need to set debug logging in crunch_script.py
Merge branch '12278-cwl-debug-flag' closes #12278
12213: Add test case.
12213: Remapping from source to destination collection needs to apply to files as well.
12213: Check for correct test output.
Merge branch '12213-expression-keepref' closes #12213
Add missing test files refs #12213
pin ciso8601 >=1.0.0, <=1.0.4 closes #12422
12418: Explicitly check if collection is not None
12418: Add regression test for glob output on empty collection.
Merge branch '12418-glob-empty-collection' closes #12418
12431: Use libcloud fork 2.2.2.dev1
12431: Fix test
Merge branch '12431-nm-delete-nic' refs #12431
12447: Reduce memory footprint of crunch-run.
12447: Add ClearBlockCache method to KeepClient
12447: Use ReadFull(). Use goimport for import formatting.
12447: Remove .Bytes()
12447: Fix typo shadowing 'data' in BlockCache.Get
12447: Require Content-Length in KeepClient.Get
12447: Simplify BlockCache locking.
Merge branch '12447-crunch-run-leak' closes #12447
12452: Consider unpaired node older than boot grace period 'down'.
12452: Fix test
Merge branch '12452-nm-unpaired-down' refs #12452
Comment out sdk/go/health from run-tests.sh until it is fixed.
12446: Avoid listing every container uuid in status query.
Merge branch '12446-dispatcher-query' closes #12446
12467: Wait for response that image loading is complete
12467: Fix log messages. Defer response.Body.Close().
Merge branch '12467-read-imgload-response' closes #12467
Update libcloud pin for VHD delete bugfix. refs #12324
Phil Hodgson (141):
Correct parameterized input collection reference
Addition of button that copies the current collection to a specified project (refs #3114, #3476)
Can now specify in an arvados model that an element should be rendered as textile (refs #2917, #3354)
Merge branch 'master' into 3354-render-textile
Merge branch 'master' into 3354-render-textile
Bug fix since recent commits to master where Textile was not being rendered any more after an update (refs #2917, refs #3354)
Now recognizes Arvados links using Textile (refs #3522, refs #3354)
Merge branch 'master' into 3114-copy-to-project
Make Collection page's Copy into project use the standard chooser (refs #3114, refs #3476)
3354: Put edit (pencil) icon _below_ textile rather than to the right of it.
3354: Also render specified attributes as textile when content is read-only
3114: bug fix: 'Copy' button is now disabled until a project has been selected for copying to
3114: When current user, show root/home project rather than user attributes when landing on show user page.
3114: bug fix: when rendering JSON response to a generic #show action, correctly set an 'href' attribute in the response to be the #show url for the object
3354: try again with edit (pencil) icon to the _right_ of the editable textile-rendered text
Merge branch 'master' into 3354-render-textile
3354: Rename render_content_from_database helper to render_markup, as a more directly descriptive method name.
3354: ordinary Textile link markup was no longer working - fixed by checking to see if the link was interpreted as an Arvados object.
3661: Added buttons for Copy and Move to the general #show page, showing them if the model class #goes_in_projects?
3661: make existing tests pass
3661: Add a new test that ensures that the new "Copy to project" button works for collections
3661: tentative implementation of Capybara "best practice" of using that class's reset_sessions! in the teardown
3338: System in place that allows for item counts to be displayed next to tab titles via AJAX.
3338: vital action missing from previous commit
3338: Tests that the fixture collection count for a project matches what is displayed in the tab
3634: add gem that handles Bootstrap tab state automagically
3661: removed useless method to wrong project
3661: add centralized method for defining whether an object can be copied to a project
3338: fix the integration test so that it checks the on-screen collection count
3338: move the user_is_manager and shared items to the already-overridden #find_object_by_uuid method so that these are available to all the same actions
3661: proposed change of the selection javascript to use the new copies_to_projects? class method on Group
3338: add a 'count-change' event that the tab panes listen for, and trigger it from destroy actions
3338: move code that sets the share_links to its own before_filter
Merge branch 'master' into 3338-tab-counts
Merge branch '3338-tab-counts' refs #3338
Merge branch 'master' into 3661-copy-move-from-show
3661: visual bug fix: button should be a link
Merge branch 'master' into 3661-copy-move-from-show
Merge branch '3661-copy-move-from-show' refs #3661
Merge branch 'master' into 3634-tab-state
Merge branch 'master' into 3634-tab-state
Merge branch 'master' into 3634-tab-state
3634: set the response to ask the browser nicely to never cache
3634: If a link has the class "force-cache-reload", then on browser-back the page will be reloaded
3634: add anchor to URL when switching project tabs
Merge branch 'master' into 3634-tab-state
Merge branch '3634-tab-state'
Merge branch '3634-tab-state' refs #3634
Merge branch 'master' into 3618-column-ordering
3618: allow for orders to be parameterized for showing records; proof of concept here for project tabs, sorting Data Collections by name
3618: first steps at making it possible to sort in infinite scroll by clicking on a column name
3618: now can use a list of sort columns if there are multiple models involved
Merge branch 'master' into 3618-column-ordering
3618: replace js use of == and != with the more predictable === and !==
3618: Add some basic API server tests for the sort feature
3618: correctly propagate the ordering parameters when scrolling
3618: Add Workbench integration test for sorting data collections
3618: When changing to sorting on a different column, make sure the directional indicator of the previous is removed.
3618: best guesses about which database columns could be used for sorting (not certain about the descriptions, though, in many cases)
3618: remove unnecessary "raw" directive for the sort columns table header attribute
3618: store in the browser history state the current sort settings for each tab
Merge branch 'master' into 3618-column-ordering
3618: Make tests pass: bugs were revealed in "Other objects" tab related to there not being any sorting going on there, etc.; add checks for HTML5 history and for nulls and undefineds
Merge branch 'master' into 3618-column-ordering
3618: bug fix: was confusing the "desc" meaning a "descending sort" with the "desc" that is the beginning of the column name "description"
3618: make sure something happens if the order param is actually detected as an array
3618: improve api security for sorting to make sure each column we sort by is for a real table and a real column in the db
3618: refactor scroll tests to share code
3618: accommodate new fixture indexing in pipeline_instance tests
3618: accommodate new fixture indexing in pipeline_instance tests (including run-time=0 test)
Merge branch 'master' into 3618-column-ordering
Merge branch '3618-column-ordering' closes #3618
3382: always show input parameters, whether value is present or not
4233: "helper" method for replaying a job log from a specified file, with utility rake task
4233: correction: the 'text' in the properties hash when simulating a logged event should be the entire log file line verbatim
4233: (naive) first iteration at graphing job log stats
4233: redraw graph when processing log events
4233: why pass around global variables?
4233: now the y-axis series will scale to always be between 0 and 1
4233: keep graph data array sorted by timestamp
4233: remove explicit JS includes for showing Keep histogram now that they are in the general assets
4233: will rescale y-axis for a series that scrolls off leaving less than fully scaled data points displaying on the chart
4233: only redraw graph a maximum of once every 5 seconds
4233: do not scale the 'cpu' series stats, and divide them by cpu count
4233: bug fix - rounding errors were causing us to miss maxima scrolling off the chart, and rescaling was not occurring
4233: chart stats for parallel tasks separately
4233: show original data in hover info instead of proportions
4233: better labelling includes raw interval data and number formatting
4233: catch and ignore log processing errors
4233: simple but reasonably effective tests
Merge branch 'master' into 3382-always-show-inputs
3382: test that an input without a description is still displayed after it has been specified
3382: test that after editing that an input is editable a second time
3382: test that after editing that an input is editable a second time
4233: add deletion of existing log entries option to log replay helper and rake task
4233: show empty log stats graph when no data, and also show a full width x-axis (currently 10 minutes) even when only a few data points have been plotted
4233: unused variable removed
4233: sort hover display by task then stat name
Merge branch 'master' into 3382-always-show-inputs
Merge branch '3382-always-show-inputs' (closes #3382)
Merge branch 'master' into 4233-graph-job-stats
4233: initial chart points added via AJAX
4233: better hover presentation (left aligned, clearer task numbers)
4233: try to match older log date format
4233: refactor to separate addDatum function
4233: missing include from last commit
4233: remove "delete log entries" option
4233: bug fixes for previous refactor and also now display series for rss and swap mem stats
4233: bug fix: match log line regex from beginning of line
4233: in hover popup only show datapoints that are defined
4233: make the log replay helper deal with the old date format
4233: tab character corrected
4233: do not show "raw" data for datapoint if not defined
4233: refactor to only query the log lines once and use the JS triggers to create both the text log div and the stat chart at the same time
4233: ignore negative deltas when charting log stats
4233: consistent use of ISO time-zone in timestamps corrects differences between Firefox and Chrome's charting results
4233: select from Logs table only columns we will need
Merge branch 'master' into 4233-graph-job-stats
4233: convert accidental tabs to spaces
4233: use "success" JS function after AJAX call to get logs rather than a JS view to "push" logs to the log chart
4233: prepend historical log entries
4233: split up received log entries by newline characters too
4233: only divide CPU usage by number of CPUs when total CPU usage is less than the measured interval time
4233: adjustment to previous commit, round numbers before comparing - stay loose
4233: websocket test of log charting should look at the right datapoint
Merge branch '4233-graph-job-stats' closes #4233
4358: fixed: the provenance graph was being generated twice, the second time for only one pipeline
4232: fixes error displaying queue time
4232: attempts to optimize what appears to be the largest Workbench slowdown when showing a lot of pipelines with a lot of jobs
4232: theoretical fix of slow API response when getting pipeline and job contents of a project
4232: bonus fix of missing word in template
Merge branch 'master' into 4358-graph-not-comparing
Merge branch '4858-graph-not-comparing' refs #4358
Merge branch 'master' into 4232-slow-pipes-n-jobs
4232: remove uselessness (commented out code and other wastes of spacetime)
4232: remove "dependencies" entries from examples in the tutorials if the API is no longer going to be returning them
Merge branch 'master' into 4232-slow-pipes-n-jobs
4232: revert experimental change to using find? for each of the jobs in a pipeline, rather than simply a where clause: there is no evidence that this switch to find? was helping to speed up anything overall
4232: remove traces of no-longer-needed "dependency" code for pipeline_instances
Merge branch 'master' into 4232-slow-pipes-n-jobs
Merge branch '4232-slow-pipes-n-jobs' closes #4232
Radhika Chippada (586):
4525: keep link_to in application_helper, which will be used when attr value is a String or Fixnum etc
4525: link_to to be used when dataclass is a String or Fixnum
4525: delete assets/stylesheets/selection.css
4525: when dataclass is Class, return attrvalue similar to the Array case.
closes #4525
4477: don't crash on no job log
4719: change "There are currently no job in the queue" to "your" queue.
Merge branch 'master' into 4638-ssh-notification
4638: update link text
closes #4638
4638: update test to reflect link text change.
Merge branch 'master' into 4638-ssh-notification
closes #4638
Merge branch 'master' into 4559-long-component-names
4559: missing quote
closes #4559
4523: add owner_uuid index on all tables that support this column.
4523: add search index
4523: search index migration script
4523: migration script update
closes #4477
4523: schema update
4523: trailing white space
4523: update migration scripts to hard code tables and columns.
4523: omit text type columns from searchable columns.
4754: add rails-perftest and ruby-prof gems to enable performance benchmarking and add search_test.rb
4754: search_test assertions
4754: move search test into browsing_test.rb instead of having its own file.
4754: add test_workbench_performance
Merge branch 'master' into 4754-performance-benchmarks
4754: support RAILS_ENV=performance
4754: update assertion to look for a data-object-uuid
4754: add command to be used to run diagnostics testing as a comment.
Merge branch 'master' into 4523-search-index
4799: do not offer "move selected" option when current user cannot write to the project.
4476: include filters in search next_page_href url.
4804: search dialog retains project_uuid param in next_page_href.
4523: update down method in search migration script to drop index only when it exists.
4523: update db migration script for owner_uuid index to drop index only when it exists.
Merge branch 'master' into 4523-search-index
Merge branch 'master' into 4754-performance-benchmarks
refs #4754
refs #4754
closes #4799
closes #4476
4414: add "Add a new project" link to project dropdown.
4414: use ensure_unique_name option to instruct api server to create unique name for new project.
closes #4414
Merge branch 'master' into 4523-search-index
4523: Change description column from text to string on jobs, groups, pipeline_instances and pipeline_templates. Also, update search indexes to include the description column for those tables.
4523: updated render_editable_attribute to use textarea for descriptions.
4523: cleaner update of render_editable_attribute method to use textarea for descriptions.
Merge branch 'master' into 4523-search-index
4523: set description size to 2**17, which seems to be the upper limit for indexing.
4523: set description size back to 2**19
4523: add file_names searchable column to collections data model.
4523: update the collection file_names migration script to set the file_names during migration.
4523: extract unique file names from manifest text and set as file_names attribute during save.
4523: Add test to create collection with large manifest text. This test showed that the max size
4523: add tests to search for file name
Merge branch 'master' into 4523-search-index
4523: exclude text type columns from searchable_columns for ilike operator
4523: some more test assertions
4523: separate filename based tests and description based tests.
4523: add a few functional tests for collections with file_names testing.
Merge branch 'master' into 4523-search-index
4523: slightly improved large description
4523: update migration script to use sql to set file_names
4926: add "upload using workbench"
Merge branch 'master' into 4523-search-index
4523: exclude manifest_text from collection -> searchable_columns and leave the other text columns for other objects.
4523: include stream names in file_names
Merge branch 'master' into 4523-search-index
4523: revert test update now that text attributes in non-collections objects are not omitted from search index.
3756: do not show 'Show' button in admin user's own row.
Merge branch 'master' into 4523-search-index
4523: update migrate script to use batch size and instance method on collection.
3686: Support sharing pane in repository show page.
Merge branch 'master' into 3686-sharing-repositories
Merge branch 'master' into 3756-show-user-button
closes #3756
Merge branch 'master' into 4523-search-index
4523: update migration script to use execute instead of update_sql so that output is not printed to console.
4523: check for a specific file name and stream names in file_names.
4523: use filters instead of where for search
4523: try to resolve column order by down and up migrations through VERSION=20140817035914
Merge branch 'master' into 4523-search-index
Merge branch 'master' into 4926-browser-based-upload
4523: update signed_locator in test
Merge branch '4523-search-index-TC' into 4523-search-index
4926: expand downloading data section
4926: download images
closes #4523
Merge branch 'master' into 4926-browser-based-upload
4926: improved image quality
4926: minor text update
Merge branch 'master' into 4926-browser-based-upload
4926: trailing white space
Merge branch 'master' into 3686-sharing-repositories
Merge branch 'master' into 3686-sharing-repositories
3686: ensure Advanced is always the last pane even after adding the Sharing pane.
closes #3686
4523: add full text search for collections and pipeline_instances.
4523: some more test cases
Merge branch 'master' into 4926-browser-based-upload
4926: add description with section links
4926: move tutorial_expectations into sections that are describing command line tools.
refs #4926
Merge branch 'master' into 4523-full-text-search
4523: use file_names column instead of manifest_text column for full text search of collections.
4523: add full text search index to collections, groups, jobs, pipeline_instances and pipeline_templates.
Merge branch 'master' into 4523-full-text-search
4523: a few more test cases
4523: do not include descriptions in search index and update tests accordingly.
refs #4523
4523: add a new migration script that invokes search_index migration down and up methods when a search_index with description is found (due to an earlier migration of search_index with descriptions).
Merge branch 'master' into 4523-search-index
4878: rerun job with options popup
Merge branch 'master' into 4878-rerun-job
4878: add integration test for Re-run job with options.
4878: now the test works!
3686: when a repository is manageable by a user, make repository name a link in manage_account page. In addition, do not offer Attributes tab in repository page if the user is not an admin.
3686: repository.editable_attributes returns an empty array when the user is not an admin.
3686: enhance manage account test to also verify a writable repository.
3686: improve manage account page test to click on the repository link.
Merge branch 'master' into 4878-rerun-job
4878: show only the re-run popup option
Merge branch 'master' into 4523-search-index
4523: a little bit simpler script
Merge branch 'master' into 3686-sharing-repositories
refs #4523
Merge branch 'master' into 4878-rerun-job
4878: update the popup display to not display radio button when only "use same" option is available.
4902: re-run pipeline instance tests use fixtures instead of creating a new pipeline to be re-run.
4878: add back the tab buttons partial.
refs #4878
Merge branch 'master' into 3686-sharing-repositories
Merge branch 'master' into 4902-use-fixture-to-rerun-pipelines
closes #4902
Merge branch 'master' into 3686-sharing-repositories
3686: verify repositories testing uses hard coded repo info.
3686: update the repository row to offer an exclusive Share link than make the repo name clickable.
3686: add unit and controller tests for repository updates
3686: correct one test assertion and refine some other assertions.
Merge branch 'master' into 3686-sharing-repositories
closes #3686
4523: prepare to merge master into branch
Merge branch 'master' into 4523-full-text-search
4523: full-text search migration script renamed
4523: full-text search is not supported for individual columns and only 'any' is supported.
2659: add permit_anonymous_browsing_for_public_data
2659: add text message "You are viewing public data" to top nav when anonymous user is accessing a public project. also add anonymous access tests.
2659: when an anonymous user is accessing public data, offer "Log in" button instead of "Log out" button.
2659: show tab buttons, selection actions etc only when the current_user is_active to allow clean anonymous user experience.
2659: hide job re-run button when user is inactive and add fixtures and tests for anonymous user to access tabs in publicly accessible project.
2659: Collection show_files shows selection dropdown (which has only create new collection from selected files option) when user is active.
Merge branch 'master' into 2659-anonymous-share-projects
2659: trailing white space
2659: more asserts in dashboard testing
2659: show job cancel button only when user is_active.
2659: It appears that a job's log file is stored in a user's Home project, instead of the project in which the job originated from.
2659: (1) Update link_to_if_arvados_object method to display an object name as link for User type only when the user is active.
2659: add completed job fixture in publicly accessible project and test assertions for re-run job button for it.
2659: slightly improved test flows
2659: remove most of the user-based tests since they are covered in other test files to make the test cases as minimal as possible.
2659: add some more text assertions; several of the same tests are failing quite consistently when run with the entire test suite.
2659: instead of setting up a session as anonymous user, just use anonymous token to serve publicly accessible resources.
2659: do not include "You are viewing public data" in topnav when anonymous user is accessing a shared project.
2659: is_anonymous helper method is not needed anymore.
2659: no need for permit_anonymous_browsing_for_public_data filter; instead, directly use the anonymous token in arvados_api_client.
2659: anonymous token handling in api client updated
2659: prevent nil exception when a content hash link is clicked by anonymous user
2659: load @prov_svg outside of the 'if current_user' block so that an anonymous user can see the provenance graph when accessing publicly accessible collection.
2659: Do not require login for project tab_counts method as well.
2659: skip require_thread_api_token filter only when anonymous config enabled.
Merge branch 'master' into 2659-anonymous-share-projects
2659: add a "you are not logged in" warning in 404 page.
2659: enable anonymous user config in test environment.
2659: fix test assertion
2659: Expect redirect to login page when show method is executed in jobs controller etc when no anonymous configuration is set.
2659: Finally understood the mystery behind the "anonymous tests are failing when run with all other tests".
2659: add test cases where anonymous config is turned off for collections, jobs, pipeline instances, templates, and projects controller show method.
2659: tests do not "assume" that anonymous configuration is enabled.
Merge branch '2659-anonymous-share-projects' of git.curoverse.com:arvados into 2659-anonymous-share-projects
Merge branch 'master' of git.curoverse.com:arvados into 2659-anonymous-share-projects
Merge branch '2659-anonymous-share-projects' of git.curoverse.com:arvados into 2659-anonymous-share-projects
2659: add publicly accessible pipeline instance and template to the list of objects to be expected in the project.
2659: Do not turn on anonymous config by default in test mode; instead make the tests set it as needed.
4951: send mail to request shell access to user
Merge branch 'master' into 4951-request-vm
Diagnostics testing: The test fails because there are 2 Run buttons in the Inputs tab right after choosing inputs. Click on Components tab before clicking on the Run button.
4951: add ajax handling for request shell access button.
4951: add tests for request shell access feature.
4951: trailing white space
4951: when a request for shell access is made, remember this in html5 localStorage so that on future revisits of the page (in the same browser) we can show an info message to that effect rather than showing the request button again.
Merge branch 'master' into 4951-request-vm
4523: preparing to merge master
Merge branch 'master' into 4523-full-text-search
Merge branch 'master' into 4951-request-vm
4951: add test assertion to verify that request access button was no longer displayed after a request was made.
4951: actually add test assertion to verify that request access button was no longer displayed after a request was made.
Merge branch 'master' into 4523-full-text-search
4951: localStorage lookup much simplified and handled during page loading.
Merge branch 'master' into 4951-request-vm
4951: remove unused id from link
Merge branch 'master' into 4523-full-text-search
4523: slightly modified full text search condition.
5110: use full text search operator '@@' instead of 'ilike'
Merge branch 'master' into 4523-full-text-search
4523: update full text search to append ':*' for each word to support a collection file search such as dir_name_part space file_name_part.
Merge branch '4523-full-text-search' into 5110-workbench-full-text-search
5110: clean up pipeline instances integration tests a bit. and remove a few of the tests in
4523: update full text search record_filter to not append :* for prefix matching; instead expect client to append it as needed.
4523: update tests to include :* in the filters sent rather than expect server to append it.
Merge branch 'master' into 4523-full-text-search
Merge branch '4523-full-text-search' into 5110-workbench-full-text-search
5110: append :* to search filter to enable prefix matching; server no longer does this and expects clients to append it as needed.
Merge branch 'master' into 4523-full-text-search
4523: add search test looking for '_' which is ignored by full text search.
Merge branch '4523-full-text-search' into 5110-workbench-full-text-search
5110: full text search seems to treat '_' characters as spaces. Thus, when searched for 'pipeline_1', objects matching 'pipeline'
Merge branch 'master' into 4523-full-text-search
Merge branch '4523-full-text-search' into 5110-workbench-full-text-search
Merge branch 'master' into 4523-full-text-search
Merge branch 'master' into 5110-workbench-full-text-search
Merge branch '4523-full-text-search' into 5110-workbench-full-text-search
5110: update comment
refs #4523 : Add postgres full text search support
Merge branch 'master' into 5110-workbench-full-text-search
Merge branch 'master' into 4951-request-vm
4951: Incorporate review feedback. Always show the request button, along with the "sent" message if applicable. Include timestamp in sent message. Some text updates as well.
Merge branch 'master' into 5110-workbench-full-text-search
5110: update couple more tests to do a single visit to page than visit home and then visit page.
5110: update a comment in test; this comment was reflecting ui from before project tabs.
closes #5110
Merge branch 'master' into 4951-request-vm
closes #4951
5185: in project data collections tab, show description when available; otherwise display modified time.
4951: button_to cannot be used to submit; instead change to link_to
5185: do not display description at all in Project -> Data collections tab because of concerns such as the description may be too big or has textile etc.
5185: remove the unnecessary test because another test is covering the description column check already.
closes #5185
Merge branch 'master' into 4951-request-vm
4951: update test to also verify that an email was sent when request shell access button was clicked.
4951: fix comment in test
5096: if only collection name or description is being updated, do not include manifest_text in the request.
5186: collection properties attribute is a hash
5186: a few more test assertions.
closes #5186
5189: include owned repositories in the list of repositories returned in manage_account.
Merge branch 'master' into 5189-manage-acct-shows-own-repos
5096: more efficient update method override.
Merge branch 'master' into 5096-wb-collection-name-or-desc-update
refs #5096
5197: write a failing test
5197: update the collection_owner_uuid_name_unique index to use expires_at is null where clause.
5197: updated migration script
Merge branch 'master' into 5197-collection-name-owner-unique
5197: down migration working
Merge branch 'master' into 5197-collection-name-owner-unique
5197: needed one more udpate
Merge branch 'master' into 5189-manage-acct-shows-own-repos
5189: a little cleanup
closes #5189
5220: do not show "Sharing and permissions" box when an anonymous user is viewing a shared collection.
closes #5220
closes #5197
3761: Run pull list worker, which processes pull reqests from the list.
3761: additional tests
Merge branch 'master' into 3761-pull-list-worker
3761: use SignLocator
5349: Reverted "Time.iso8601(current_job[:created_at]" back to "current_job[:created_at]". All tests and manual testing passed and no negative side effects are observed.
Merge branch 'master' into 3761-pull-list-worker
3761: code refactoring
3761: improved tests with delays
Merge branch 'master' into 3761-pull-list-worker
3761: pass keepclient as an arg to RunPullWorker
Merge branch 'master' into 3761-pull-list-worker
3761: Pass pullq to RunPullWorker
Merge branch 'master' into 5349-timestamp-error-for-running-pipeline
5349: update test to use a new fixture.
closes #5349
5145: while combining colletions, append an integer for any duplicate files.
Merge branch 'master' into 5145-combine-collections-repeated-filenames
5145: add tests to verify combine collections with duplicate filenames
Merge branch 'master' into 3761-pull-list-worker
3761: pass keepClient to Pull method
3761: log errors in RunPullWorker instead of Pull method.
3761: include names in test data initializations
3761: verify ReadContent and PutContent outputs.
3761: Improve the tests to RunTestPullWorker for each test and verify the queue is empty at the end of test.
3761: move test assertions from teardown method into performTest function for better readability of code.
3761: update GetContent function to use read_content test data info.
3761: (WIP) add integration test for pull_worker
3761: integration test WIP
5374: in collection show page, hide checkboxes for anonymous user view.
5190: tell user to not use hyphens in tooltip for repository name editable attribute
Merge branch 'master' into 3761-pull-list-worker
3761: get keep services
Merge branch 'master' into 5374-hide-collection-checkboxes-anonymous
Merge branch 'master' into 5374-hide-collection-checkboxes-anonymous
closes #5374
Merge branch 'master' into 5190-no-hyphens-in-repo-names
5190: updated tooltip
closes #5190
Merge branch 'master' into 5145-combine-collections-repeated-filenames
5145: better pattern matching for manifest text files
5145: revert back fixture update and instead add a new fixture for test combining of collections.
3761: need at least 1 replica
Merge branch 'master' into 3761-pull-list-worker
3761: a little refactoring
3761: Working!!!!
5304: include nulls in resutls when filter operator is "!="
4426: add google search in doc topnav
Merge branch 'master' into 4426-search-documentation
3761: add a test with invalid data manager token and expect the item to be rejected.
Merge branch 'master' into 3761-pull-list-worker
3761: Expand DiscoverKeepServers method to return service_roots and use it in test
Merge branch 'master' into 5145-combine-collections-repeated-filenames
closes #3761
Merge branch 'master' into 5304-include-nulls-when-notequal-filter
closes #5304
5145: small updates
Merge branch 'master' into 5145-combine-collections-repeated-filenames
4426: name attribute is needed for google search
Merge branch 'master' into 4426-search-documentation
4426: convert the search icon to a button
Merge branch 'master' into 4426-search-documentation
closes #4426
Merge branch 'master' into 5145-combine-collections-repeated-filenames
closes #5145
5145: include manifest text in assert error message to identify the cause of test failure in jenkins.
refs #5145
5418: topnav brand name is clickable only when user is logged in.
5418: fix assert formatting
Merge branch 'master' into 5418-brand-unclickable-anonymous
closes #5418
3761: We do not need to MakeArvadosClient to create the pullq worker. Otherwise, we need the ARVADOS_API_HOST env variable in keep.
refs #3761
5383: use db current time in place of Time.now
Merge branch 'master' into 5383-api-db-current-time
5383: add test that creates an object and does a lookup using that object's created_at timestamp.
5383: blob.rb uses db_current_time as well. since it is using static methods, the module needs to be "extended"
Merge branch 'master' into 5383-api-db-current-time
Merge branch 'master' into 5383-api-db-current-time
Merge branch 'master' into 5383-api-db-current-time
5383: update db_current_time to return date with more precision. Update log_test to use db_current_time instead of Time.now to ensure if there is a slight difference in time between db time and server time, the test does not fail.
5383: revert back db_current_time update to Time.now in api scripts and middleware rb files.
5383: revert db_current_time back to Time.now in application_controller as the usages were for error tokens and request profiling only.
5383: postgres current_timestamp is the start of transaction and clock_timestamp is the current time at the statement execution. updated db_current_time to use clock_timestamp and reverted back all updates to log_test.rb
5383: remove unused include statement from log_test (it is now the same as in master with no updates in this branch).
5383: add one more test that updates an object twice and verifies the updated_at times are different.
Merge branch 'master' into 5383-api-db-current-time
5383: minor test assertion updates
5375: add some of the mime types not supported by mime_types gem to mime_types.rb initializer.
5375: add test to verify that collection#show page enalbles / disables file viewing correctly.
Merge branch 'master' into 5383-api-db-current-time
5383: better test assertion
closes #5383
Merge branch 'master' into 5375-preview-collection-text-files
5383: Add back Time.parse(). Without it the value returned would be of the format 2015-03-18 13:37:11 UTC,
Merge branch 'master' into 5383-api-db-current-time
refs #5383
Merge branch 'master' into 5375-preview-collection-text-files
5375: after much hair pulling, found the syntax to replace MIME::Types.add(MIME::Type.from_array... since from_array is deprecated; now using MIME::Types.add(MIME::Type.new...
5375: enhance the mime_types initializer to add a new type only when that type is not yet supported by the MIME types gem.
5375: white space
3762: Run trash worker. Add tests to test the trash worker.
Merge branch 'master' into 5375-preview-collection-text-files
5375: improved mime_types initializer. enhance collections_helper to use mime_type sub_type instead of relying on suffix.
5375: update the fixture pdh with the value got from keep put.
Merge branch 'master' into 3762-delete-list-worker
3762: Expose Discovery func in arvadosclient. Use Discovery to get defaultTrashLifetime in trash worker.
Merge branch 'master' into 3762-delete-list-worker
Merge branch 'master' into 5375-preview-collection-text-files
5375: move the mime type tests into a new collections_helper_test.
5375: and add the new test file
5375: small updates on comments
closes #5375
5375: added javascript and rtf application types to white-listed application types after observing that these were disabled when testing in staging.
refs #5375
Merge branch 'master' of git.curoverse.com:arvados into 3762-delete-list-worker
3762: undo the api server discovery document update. trash worker can instead use permission_ttl in place of defaultTrashLifetime because these two will be the same.
3762: create NewWorkQueue instances of pullq and trashq in handers_test and remove this logic from handlers.
Merge branch 'master' into 3762-delete-list-worker
5534: When a pipeline has long running jobs with hundreds of thousands of log lines, the log line display is timing out
5247: anonymous access documentation
5247: white space
5556: Added select() to @logs in CollectionsController#show to avoid trasporting the unneeded log properties.
5556: add test to invoke collection#show with and without include_accept_encoding_header_in_api_requests config parameter.
5420: address checkbox selection caching issue with Safari and Chrome.
Merge branch 'master' into 5420-collection-checkbox-selections
Merge branch 'master' into 3762-delete-list-worker
closes #3762
Merge branch 'master' into 5534-limit-log-lines
closes #5534
Merge branch 'master' into 5247-anonymous-doc
5247: some caption updates.
Merge branch 'master' into 5556-collection-show-timeout
5247: couple more caption updates.
refs #5247
Merge branch 'master' into 5556-collection-show-timeout
closes #5556
5493: wiring for 'getting started' popup
5493: fix a bug in user profile handling where any pre-existing prefs are being lost.
5493: (thanks to tom) modal paging is awesome.
Merge branch 'master' into 5420-collection-checkbox-selections
5493: update user_profile test to expect getting_started modal for users with no previous profile configuration;
5493: add test for the getting_started help menu item.
5493: remove the helper method update_getting_started_shown_user_pref, and instead perform prefs update in the view itself.
Merge branch 'master' into 5493-getting-started-modal
5493: add pager-count where text such as "1 of 10" is shown.
5493: update tests to use updated fixture active_no_prefs_profile_no_getting_started_shown
5493: do not show "Public pipelines and datasets" link in help menu if config parameter is not set.
5493: getting_started modal is served from layout html
5493: improve test to verify Next button disables when last page is accessed
Merge branch 'master' into 5493-getting-started-modal
Merge branch 'master' into 5493-getting-started-modal
5493: test uncovered an issue with getting_started_shown handling; fixed it.
5493: one more stale update
Merge branch 'master' into 5420-collection-checkbox-selections
Merge branch 'master' into 5420-collection-checkbox-selections
5420: remove search box autocomplete off update so that this can be addressed in a separate issue.
closes #5420
5365: Do not link to job, log, output in pipeline instance if the user, mainly anonymous user, cannot read those objects.
Merge branch 'master' into 5365-not-link-unreadables
5493: minor refactoring
5365: add integration test to verify Log tab and other unreadable properties are disabled in anonymous view.
Merge branch 'master' into 5365-not-link-unreadables
5365: additional test assertions for pipeline log tab display
5365: couple more integration tests to access Log tab from within the Job page.
5365: white space
Merge branch 'master' into 5365-not-link-unreadables
5492: add "Home project" link to my account menu.
5565: show trimmed down version of breadcrumbs in anonymous view.
Merge branch 'master' into 5565-breadcrumbs-in-anonyymous-view
Merge branch 'master' into 5492-home-project-in-my-account
closes #5492
Merge branch 'master' into 5565-breadcrumbs-in-anonyymous-view
5565: show parent project hierarchy in breadcrumbs in anonymous view; not just current objects's parent.
closes #5565
5365: job link in pipeline instance page
Merge branch 'master' into 5365-not-link-unreadables
5365: refactor link to object if readable logic
Merge branch 'master' into 5365-not-link-unreadables
5365: preload jobs during pipeline display to improve performance.
5365: preload output collection uuids and jobs to help improve performance when loading large pipelines.
Merge branch 'master' into 5365-not-link-unreadables
5365: trailing white space
5365: Additional test verifications for Log tab display.
Merge branch 'master' into 5493-getting-started-modal
5365: add @distinct handling to workbench select queries and use this to preload portable_data_hashes as well.
5365: preload editable component inputs
5365: add controller tests for preload_for_pdhs and collection_for_pdh methods
Merge branch 'master' into 5365-not-link-unreadables
5365: do not link to docker_image_locator if not readable; also preload these.
5365: updated signature for link_to_arvados_object_if_readable
5365: missed one updated signature update
5365: add back data-toggle along with link disabled for Log tab.
closes #5365
5365: when the attrvalue is not a uuid, link_to_if_arvados_object
refs #5365
Merge branch 'master' into 5493-getting-started-modal
5493: remove confusing btn class on explanation button spans to not show hand mouse cursor
5493: that did not sit well
5493: public pipeline wiki page link added in page 3
5493: make popup size fixed
5493: popup size and a typo
5493: popup size
refs #5493
5647: add a hover dropdown login item.
5440: remove usused getting_started images
closes #5440
Merge branch 'master' into 5647-login-dropdown
5647: minor tweak to dropdown text
5647: add icon to the new login dropdown link
5647: as a team we need to learn to weigh options with their worth; aka, this login hover drop-down should have been implemented as a simple tooltip.
5647: minor text update
closes #5647
5417: add "unreadable inputs provided" visual cue for an editable pipeline instance.
5417: when input is unreadable, present visual cue even when it is not editable by the user.
5417: add additional test updates to verify Run button status when input is not readable.
5417: fix typo
5417: a few changes from based on review feedback. also, account for dataclass=File type when checking if input is readable by current user.
Merge branch 'master' into 5417-not-start-pipeline-with-unreadable-inputs
5417: update created_at time on pipeline_in_publicly_accessible_project, so that it is newest and always first in the list.
5417: revert unreadable-input check for links since no such case exists.
5417: in case of a File dataclass with unreadable input and uneditable pipeline, the Run button should be disabled.
5417: one more test condition for dataclass=File
Merge branch 'master' into 5417-not-start-pipeline-with-unreadable-inputs
closes #5417
5573: The activity page retrieves too much data that it does not need from the server. Added a limit clause on the request to server.
5753: show name column in project#show page when the object supports name.
5620: add ready handler for input
5675: include Subprojects tab in project#show for anonymous users.
5179: update files method to move dir parts to streamname.
5179: update arvados ruby sdk gem version in workbench.
5663: update documentation to use "puts rand" in place of "rake secret"
5620: added comment to input trigger event
closes #5620
Merge branch 'master' into 5753-job-name-column-in-project-page
closes #5753
Merge branch 'master' into 5573-user-activity-page-performance
closes #5573
5720: ajax new request check update
Merge branch 'master' into 5179-dirs-in-filenames
5179: one more test with multilevel manifest text
closes #5179
refs #5179 : gem version in workbench
Merge branch 'master' into 5720-ajax-loading-error
Merge branch 'master' into 5675-project-subprojects-in-anonymous-view
Merge branch 'master' into 5663-doc-rake-secret
closes #5675
Merge branch 'master' into 5663-doc-rake-secret
closes #5783
Merge branch 'master' into 5663-doc-rake-secret
closes #5663
5780: Display message in chooser popup if one is provided. Use this to show a message in sharing popup
Merge branch 'master' into 5720-ajax-loading-error
5796: omit file_names column from collections search index.
5796: add down migration
closes #5796
Merge branch 'master' into 5720-ajax-loading-error
Merge branch 'master' into 5780-sharing-popup-message
closes #5780
5826: check readable only when it is a data collection pdf or uuid.
5827: check readable only attrvalue is a string
closes #5826
5622: when there are too many collections matching a pdh, show only 20 of them.
closes #5622
5826: revert the previous update that checked if readable only for pdh or uuid;
5826: use present? instead of !empty?
closes #5826
5843: rename objects that contain xx_owner_uuid_name_unique constraint when they are removed from a subproject and the home project already has another object of the same type and name.
4458: breadcrumbs now visible in small window sizes; also, use user icon instead of email for my account menu.
Merge branch 'master' into 5843-subproject-deletion-owner-name-constraint
closes #5843
Merge branch 'master' into 4458-breadcrumbs-in-small-screens
closes #4458
5735: use inline editing for project description x-editable textarea.
Merge branch 'master' into 5735-edit-description-box-size
5735: use the inline css for description only so that any future inline css declarations are not impacted with this.
Merge branch 'master' into 5735-edit-description-box-size
5735: exclude btn* attributes from span tag.
5735: use data- for button text and button class custom attributes.
5735: add a new nonhtml_options parameter to render_editable_attribute method,
5735: add comment
Merge branch 'master' into 5735-edit-description-box-size
5735: move tiptitle into nonhtml_options
closes #5735
4467: add "select all" and "unselect all" buttons to project contents.
4467: add test for "Select all" and "Unselect all" buttions in project tabs.
3454: use configured default docker image when none found in a job's runtime_constraints.
Merge branch 'master' into 4467-project-contents-select-all
4467: Move select_all_items and unselect_all_items methods into selection.js
5682: move logic to check if '/' in filename into each_file_spec method from files method so that it is applied in all contexts using the each_file_spec method.
5682: update workbench arvados gem version
5720: when jqxhr readyState == 0 or status == 0, it could be that the user has navigated away from the page; hence return instead of showing error.
closes #4467
Merge branch 'master' into 5682-download-file-error
closes #5682
refs #5682 : update arvados gem version
Merge branch 'master' into 3454-default-docker-image
closes #3454
5645: added open graph meta tags to page
5720: when jqxhr readyState == 0 or status == 0, instead of skipping show "Cancelled" message.
5720: when jqxhr readyState == 0 or status == 0, instead of skipping show "Cancelled" message.
Merge branch 'master' into 5720-ajax-loading-error
4717: add read_only column for keep_services data model.
4717: use keep_services -> read_only flag in go sdk.
5645: add og:site_name to the list of open graph meta tags added to the page.
Merge branch 'master' into 5645-open-graph-meta-tags
5645: updated comment
closes #5645
Merge branch 'master' into 4717-read-only-keep-services-flag
4717: writable roots in go sdk
4717: just a couple minor tweaks
4717: use keep_services -> read_only flag in python sdk.
Merge branch 'master' into 4717-read-only-keep-services-flag
6064: display only 5 objects for which this collection is the output in source_summary.
closes #5720
Merge branch 'master' into 4717-read-only-keep-services-flag
4717: rename writableRoots as writableLocalRoots.
closes #4717
6061: add workbench integration_performance test folder.
refs #6061
Samuel Shapiro (2):
added initial python wh file
added PuTTY instructions to docs site
Thomas Mooney (1):
Support Docker v1.10+ hashes that include sha256: prefix.
Tim Pierce (582):
Updating install and use documentation.
Initial build files for Docker images with Arvados support.
Use PostgreSQL, not MySQL.
Post-install configuration script for PostgreSQL.
Rename "cfi" docker images to "arvados"
Merge branch 'master' of git.clinicalfuture.com:arvados
Splitting out containers into arvados/base, arvados/postgresql and arvados/api.
Merging the postgres container back into the api container (simpler
Specify all configuration settings in config.yml.
Bug fixes.
Configure Apache and Passenger.
Fix passenger filename.
Initial checkin for building warehouse-apps.
Start the API container with supervisor, running postgres & apache2.
Use supervisord to start postgres + apache2 when the api
Rename config.yml to config.yml.example (easier to keep real configuration
Copied Apache vhost config from qr1hi.
Merge branch '1578-api-server-in-docker' of git.clinicalfuture.com:arvados into 1578-api-server-in-docker
Set nameservers at config time.
Merge branch '1578-api-server-in-docker'
Precompile Rails assets.
Merge branch 'master' of git.clinicalfuture.com:arvados
Break out deploying vs. installation instructions.
Merge branch 'master' of git.clinicalfuture.com:arvados
Deploy containers with working copy of sources.
Merge branch '1579-keep-server-in-docker' of git.clinicalfuture.com:arvados into 1579-keep-server-in-docker
Build an arvados/warehouse container with locally checked out warehouse-apps
Add supervisor to start keepd when container is launched.
Add prerequisites for Keep server described in
Install warehouse-apps into arvados/warehouse directly from github source.
Deploy containers with working copy of sources.
Initial checkin for building warehouse-apps.
Build an arvados/warehouse container with locally checked out warehouse-apps
Add supervisor to start keepd when container is launched.
Add prerequisites for Keep server described in
Install warehouse-apps into arvados/warehouse directly from github source.
Merge branch '1579-keep-server-in-docker' of git.clinicalfuture.com:arvados into 1579-keep-server-in-docker
Added Makefile to track dependencies for Docker images.
Merge branch 'master' of git.clinicalfuture.com:arvados
Updated Makefile for more accurate dependency management.
Merge branch 'master' of git.clinicalfuture.com:arvados
Warehouse docker image no longer requires local checked-out source.
Fixed Makefile dependencies, added docker_build script to work around
Improve Makefile rules.
Fix dependencies.
Converted README to real markdown.
Improve Makefile rules.
Fix dependencies.
Converted README to real markdown.
Ruby 2.0.0-p247 -> p353 (now installed as stable).
Add --api, --sso, --workbench, --doc, --keep options to run.sh.
Changes to permit keep_server containers to authenticate to api_server.
Mount separate volumes for each keep container launched.
run.sh bug fixes.
API_SUPERUSER_TOKEN must be named API_SUPERUSER_SECRET to be autogenerated.
Superuser script must be run as ./script/create_superuser_token.rb.
Add support for ARVADOS_API_PORT.
run.sh prints instructions/settings for running a test suite.
Reorganized code to support 'start' command.
run.sh: add "stop" command, also remove old containers before trying to start them
run.sh: bug fixes
Added 'test' command to run.sh.
Adding 'run.sh test api' to run API server Rails tests.
Merge branch '1587-fix-test-suites-in-devsandbox'
Fix port flags for "run.sh start".
Added Keep tutorial (refs #1647)
Fixed formatting.
Added skeleton doc files for all API resources.
Linking all generated doc pages to index. (refs #1608)
Updated table headings, removed redundant fields.
Updated documentation for clarity.
Remove extra `shift' screwing up argument processing.
Added skeleton doc files for all API resources.
Linking all generated doc pages to index. (refs #1608)
Updated table headings, removed redundant fields.
Merge branch '1608-api-documentation' of git.clinicalfuture.com:arvados into 1608-api-documentation
Reorganize navbars.
Reorganizing API documentation to support new navbar (refs #1651).
Fix bug in default argument types.
Undo `docker build' hack fixed in docker 0.7; add git repo to support JobsController.
Added skeleton doc files for all API resources.
Linking all generated doc pages to index. (refs #1608)
Updated table headings, removed redundant fields.
Added skeleton doc files for all API resources.
Linking all generated doc pages to index. (refs #1608)
Updated table headings, removed redundant fields.
Reorganize navbars.
Reorganizing API documentation to support new navbar (refs #1651).
Fix bug in default argument types.
Merge branch '1608-api-documentation' of git.clinicalfuture.com:arvados into 1608-api-documentation
Adding REST method documentation stubs (resolves #1608)
Merge branch '1608-api-documentation'
Normalize API documentation names, fix links, etc. (refs #1608)
Fixed more broken links, normalized styles. (refs #1608)
Add `make realclean' target for removing cached Docker images from a build.
Merge branch 'master' of git.clinicalfuture.com:arvados
Fix makefile syntax for `realclean' target.
Fix makefile brittleness.
Merge branch 'master' of git.clinicalfuture.com:arvados
Added gen_api_method_docs.py used to generate API documentation files.
Added `gen_api_schema_docs.py' used to generate schema documentation pages.
Documentation for `Log'.
Document a convention for using Links to define tags on Collections.
Use docker build -q to fix Docker build failures.
Adding `tag' command: initial commit.
Added 'add' and 'remove' functionality.
Bug fixes.
Bug fixes (API requires request to be delivered in request_body)
Complete `arv tag add' and `arv tag remove' support (fixes #1530)
Merge branch '1530-arv-tag-commands'
Command-line clients use $HOME/.config/arvados for config settings.
Command-line clients use $HOME/.config/arvados for config settings.
Merge branch '1510-arv-tag-unit-tests' of git.clinicalfuture.com:arvados into 1510-arv-tag-unit-tests
Adding arv-tag unit tests.
Command-line clients use $HOME/.config/arvados for config settings.
Adding arv-tag unit tests.
Clean up tests with "assert_tag" method, remove non-working test. Fixes #1510.
Merge branch '1510-arv-tag-unit-tests' of git.clinicalfuture.com:arvados into 1510-arv-tag-unit-tests
Updated `arv' command line format options. Default now generates JSON. (refs #1889)
Merge branch 'master' of git.clinicalfuture.com:arvados
Updating API parameter documentation (refs #1901)
Clients use $HOME/.config/arvados/settings.conf for configuration. (fixes #1943)
Merge branch '1943-client-sdk-config-files'
Cache the API discovery doc for at least 24 hours (refs #1923)
Updating dependencies to satisfy RubyGems.
Updating dependencies.
Python SDK uses ~/.config/arvados/settings.conf like other SDKs.
Rearranging modules to eliminate recursive imports.
Rearranging modules to eliminate recursive imports.
Resolving scoping problems. Python unit tests now pass. (fixes #2036)
Merge branch '2036-fix-python-sdk' of git.clinicalfuture.com:arvados into 2036-fix-python-sdk
Fix some arv unit tests.
Fix build.sh bugs.
Update api/production.rb.in with changes from services/api.
Added missing workbench configuration directives to Docker (fixes #2072)
Merge branch '2072-workbench-docker'
Update docserver build process for zenweb (refs #1786)
Merge branch 'master' of git.clinicalfuture.com:arvados
Renamed run.sh -> arvctl, added per-service "stop" and "restart" commands (fixes #2099)
Correct comments to account for new verbs.
Rename to "arvdock" per code review.
Merge branch '2099-CORRECT-docker-stop-and-restart'
Updated Docker documentation to reference "arvdock". (refs #2099)
Merge branch '2099-CORRECT-docker-stop-and-restart' (documentation update)
Update arvdock instructions.
Update Docker-based services to ruby-2.1.0.
Fix documentation & workbench build permissions (refs #2221)
Merge branch 'master' into 2221-complete-docker
Minor doc fix.
Merge branch 'master' into 2221-complete-docker
Add build infrastructure for Perl SDK.
Update Perl install documentation.
Merge branch 'master' into 2221-complete-docker
Remove dependencies on warehouse and IPC::System::Simple (refs #2221).
Exclude any api/log files from API docker server, in case user is running an API server locally (refs #2221).
Build all containers from working source. (refs #2221)
Merge branch 'master' into 2221-complete-docker
Update doc for running crunch-job in local mode. (refs #2221)
New script install.rb (install all dependencies and build Arvados). refs #2221
Merge branch 'master' into 2221-complete-docker
Code review for 1844-blob-signature: documentation, slight refactoring (refs #1844)
install.rb: generate config.yml file on demand if necessary. Refs #2221.
Merge branch 'master' into 2221-complete-docker
Add build.sh wrapper for build.rb, to make sure Ruby is installed. (Refs #2221)
Make build.sh executable. (refs #2221)
Fix bugs for Ubunto 12.04 and 13.10. (refs #2221)
Force ruby1.9.3 install. (refs #2221)
Merge branch 'master' into 2221-complete-docker
Fix bugs introduced from merge with master. (refs #2221)
Update instructions to use build.sh. (refs #2221)
Merge branch 'master' into 2221-complete-docker
Small documentation fixes for installing. refs #2221
Merge branch 'master' of git.clinicalfuture.com:arvados
Merge branch 'master' into 2221-complete-docker
install_sdk.sh: install the arvados-cli gem and all prerequisites. (refs #2221)
Merge branch 'master' into 2221-complete-docker
Fix RVM install bug (rvm-installer --ruby doesn't work in Docker?) refs #2221
Save logs to a temp file and commit to Keep via 'arv keep put'. Refs #2221.
Merge branch 'master' into 2221-complete-docker
Merge branch 'master' into 2221-complete-docker
collate_output: close the 'arv keep put' input fd before checking for output. Refs #2221
Remove legacy 'thaw' code (refs #2221).
Incorporate code review comments (refs #2221, fixes #2325)
Reorganize build tools to be less confusing.
Clone git repository properly in API server container. (refs #2221)
Initial checkin: simple Keep server handling block reads only. (refs #2291)
Adding unit tests for GetBlock. (refs #2291)
Added unit tests TestGetBlockOneKeepOK and TestGetBlockFail (refs #2291)
Unit test TestGetBlockCorrupt. (refs #2291)
Incorporating comments from code review (refs #2438, refs #2291)
Improve readability of unit tests. (refs #2291)
Incorporating code review comments (refs #2291, refs #2438)
Merge branch '2291-new-keepd-read-blocks' (fixes #2291)
Unit test for FindKeepVolumes (refs #2295)
Merge branch 'master' into 2221-complete-docker
Addressing code review comments (refs #2485, refs #2295)
Merge branch '2291-new-keepd-read-blocks'
Added PutBlock and PutBlockHandler (refs #2292)
PutBlock creates dirs as necessary when storing new blocks (refs #2292, refs #2449)
Typo fix.
Added TestPutBlockOK. (refs #2292, refs #2449)
Added TestPutBlockOneVol and TestPutBlockCorrupt. (refs #2292, refs #2449)
Doc fixes.
Set umask to 022 when creating config files. (refs #2221)
Use quotemeta to protect shell escapes (refs #2221, #2325)
Resolving style comments from #2325. (refs #2325, refs #2221)
Use a simple 'open' for fetch_block (freeing up an extra filehandle). refs #2325, refs #2221
Semicolon fix (refs #2325, refs #2221).
Merge branch '2221-complete-docker' (closes #2325, closes #2221)
Fix $keep variable scope (refs #2221)
PutBlock saves a block on only the first available volume (refs #2292)
Merge branch 'master' into 2449-keep-write-blocks
Add command-line flags. (refs #2463)
Added IsFull() to check for free space before writing. (refs #2292)
Merge branch 'master' into 2449-keep-write-blocks
Check for MD5 collisions and corrupt blocks. (refs #2292)
Merge branch '2449-keep-write-blocks' into 2449-keep-flags
Merge branch 'master' into 2449-keep-write-blocks
Added /index handlers. (refs #2561)
Update for code review (refs #2292).
Merge branch 'master' into 2449-keep-write-blocks
Added status.json handler. (refs #2561)
Merge branch '2449-keep-write-blocks' into 2449-keep-index-status-handlers
Write status output in JSON. (refs #2561)
Added TestPutBlockCollision (refs #2292).
Merge branch 'master' into 2449-keep-write-blocks
Added todo for TestPutBlockWriteErr (refs #2292)
Merge branch '2449-keep-write-blocks' into 2449-keep-flags
Merge branch '2449-keep-write-blocks' into 2449-keep-index-status-handlers
Merge branch 'master' into 2449-keep-write-blocks
Merge branch 'master' into 2449-keep-flags
Updating docs for code review (refs #2463)
Added ReadAtMost function for reading request data (refs #2292)
Update documentation for command-line flags (closes #2463, closes #2591).
Merge branch 'master' into 2449-keep-index-status-handlers
Merge branch 'master' into 2449-keep-write-blocks
Merge branch '2449-keep-write-blocks' into 2449-keep-index-status-handlers
Fix nil map bug and status.json route. (refs #2561)
Code review comments (refs #2292)
Clean up error handling (refs #2292).
Merge branch 'master' into 2449-keep-write-blocks
Merge branch '2449-keep-write-blocks' into 2449-keep-index-status-handlers
Simplify GetNodeStatus to return only disk usage stats for volumes (refs #2561)
Merge branch 'master' into 2449-keep-write-blocks
Merge branch '2449-keep-write-blocks' into 2449-keep-index-status-handlers
Added rudimentary GetNodeStatus test. (refs #2561)
Merge branch '2449-keep-write-blocks'
Merge branch 'master' into 2449-keep-index-status-handlers
Added IsValidLocator to filter /index output.
Add a device_num field to status.json output.
Merge branch 'master' into 2449-keep-index-status-handlers
Check that specified Keep volumes actually exist.
Merge branch '2449-keep-index-status-handlers'
Reorganize Keep code in preparation for dividing into modules.
Adding a Volume interface and UnixVolume implementation.
Bug fix: GetBlock must report a CorruptError immediately.
Made KeepVolumes a slice of Volume objects, not strings.
Added UnixVolume unit tests. (refs #2620)
Moved the MD5 verification check back to GetBlock.
Updated UnixVolume unit tests (refs #2620).
Added -serialize flag.
Cleaned up unit tests. (refs #2620)
Cleaned up unit tests. (refs #2620)
Make sure an IOHandler is running when -serialize is on.
Reorganized Volume code for easier comprehension.
Corrected name for volume_test.go.
Merge branch 'master' into 2620-keep-serialize-io
Incorporating code review comments (refs #2620)
Replaced KeepVolumes with a VolumeManager interface.
Merge branch '2620-keep-serialize-io' (closes #2620)
Added permission helper functions.
Update docs. (refs #2328)
Incorporating code review. (refs #2328)
Resolve code review (closes #2328)
Merge branch '2328-keep-permission-hints'
Add --permissions-key flag.
Check GET permissions in the HTTP handler.
Added permission flags and unit tests.
Reset enforce_permissions between tests. (refs #2328)
Require enforce_permissions and API token as necessary.
Split out HTTP handler tests into their own file.
Added handler_test.go. (refs #2328)
2328: do not expose keys to /bin/ps
2328: simplify tests, permission_ttl variable
2328: fix TestPutHandler permission_ttl
2328: handler_test.go refactoring
2328: restrict all /index requests to superuser
2328: incorporate code review comments.
2328: more 80-column fixes
2328: code review comments.
Merge branch '2328-keep-permission-flags' (closes #2328)
2328: fix log.Warning bug
2755: Verify permission signatures on create.
2770: terminate on SIGTERM.
2755: Verify permission signatures on create.
Merge branch '2770-keep-sigterm' (closes #2770)
Merge branch '2755-api-collection-permissions' of git.curoverse.com:arvados into 2755-api-collection-permissions
2755: fix merge conflicts (refs #2755)
2755: incorporate code review
Merge branch '2755-api-collection-permissions'
2755: fix blob signing bug. (refs #2755)
Merge branch 'master' of git.curoverse.com:arvados
2755: add support for signed locators in the Python SDK.
2755: incorporate code review.
2829: add X-Block-Size response header
2865: collect garbage after each GET and PUT.
Merge branch '2755-python-sdk-permissions'
2755: add support to arv-put for signed manifests.
2755: code review.
2755: code review.
2755: comment doc fix.
2755: add unit test for Collection.normalize_uuid.
Merge remote-tracking branch 'refs/remotes/origin/2755-python-sdk-permissions-TC' into 2755-python-sdk-permissions
Merge branch 'master' into 2755-python-sdk-permissions
Merge branch '2755-python-sdk-permissions'
2865: reduce Keep memory usage.
2865: add traffic_test.py for testing Keep performance.
2857: fix flaky time-sensitive tests
Merge branch 'master' into 2857-collection-tests
2857: strip signatures from manifests before testing equality.
Merge branch '2857-collection-tests'
2934: add limits for crunch job log generation.
Merge branch '2934-limit-crunch-logs' of git.curoverse.com:arvados into 2934-limit-crunch-logs
2934: add launch_crunch_dispatch to integration test
2934: count events and bytes in-memory
2934: add "bar" repository to test fixture
Merge branch 'master' into 2934-limit-crunch-logs
2934: add crunch_dispatch_test repository
2934: document stderr messages for commit tests
Merge branch '2934-limit-crunch-logs'
2936: improve progress bar for failed jobs
2934: add crunch-dispatch default settings
Merge branch '2934-limit-crunch-logs-bugfix'
2936: fix job progress bars
Merge branch '2936-job-progress-bar'
2873: permission links are owned by root
2873: add /permissions API method
2873: changes for code review
3151: use 2-week default permission TTL
Merge branch '3151-permission-ttl'
2985: display project descriptions inline
2873: more code review changes
Merge branch '2985-shared-projects-descriptions'
Merge branch '2873-permission-links-ownership'
3072: add an empty collection in db seeds
3072: anonymous group can_read empty collection
3072: rendering "Empty Collection" in default_name
Merge branch '3072-empty-collection-bugfix'
2769: reorganize REST handlers
Merge branch '2769-keep-delete-request'
2985: add "move project" to "Selection..." dropdown
2985: use move_selections_into_project action
2769: implement DELETE.
2769: code review comments
3261: fixes for Docker build process
2769: code review comments
2769: more code review comments
Merge branch '2769-keep-delete-request'
3448: check block timestamp before DELETE
3448: minor bugfixes
3448: code review comments.
3448: unit tests for deleting new blocks
3448: add error checking
3448: add error checking in volume.Touch()
3448: unit tests, bug fixes
3448: update tests with "untouchable" volumes
Merge branch '3448-keep-put-timestamps'
3414: adding PullHandler and Replicator
3414: update for code review
3414: rename replicator to pull_list
3414: update comment for code review
3414: fix log typo
Merge branch '3414-keep-pull-handler'
3699: add arv-copy
3699: support pipeline templates
3699: arv-copy supports collections
3699: added copy_pipeline_instance
3699: copy docker images
3663: StreamFileReader.read() returns just one chunk
3663: update test_file_reader unit test
3699: add arv-copy to list of installed scripts
3699: accept collection hash as uuid
3699: bug fixes
3699: test storing the empty collection (edge case)
3699: skip components which have not yet run
Merge branch '3663-collection-reader-performance'
3823: bug fixes for arvados-bcbio-nextgen.py
3699: define copy_git_repo
3699: fix bugs
3699: add copy_git_repo step to copy_pipeline_instance
3699: typo fix
3699: rename repositories in copied pipeline instance
3699: add support for --recursive and --project_uuid
3705: add BlockWorkList and unit tests
3705: replace pullmgr with a BlockWorkList
Bug fix: manifests with extra spaces
3699: bug fixes and feedback
3699: bug fixes
3699: fix copy_git_repo
3699: use a consistent destination branch for copy_git_repo
3699: bug fix (return value for copy_git_repo)
3699: fix inconsistent function parameters
3699: allow pipeline instances without templates
3699: copy_pipeline_template requires dst_git_repo
3699: delete pi['uuid'] before committing
Merge branch 'master' into 3699-arv-copy
3699: ensure unique names for new pipelines
3699: use /scratch instead of /tmp for git copies
3699: typo fix
3699: back out attempts to change tempfile.tempdir
3699: report success/failure unambiguously
3699: copy collection properties, name, etc.
3705: describe BlockWorkList flow more explicitly
3705: update for code review
3699: code review
3699: update --src, --dst, --dst-git-repo args
3699: more help text for --src and --dst options
3699: document required options --src and --dst
3411: set default collection expiration policy
3699: update script_version and supplied_script_version
3411: code review
3705: drop unnecessary current_list pointer
3705: use helper functions in tests
3705: drop unnecessary 'items' from BlockWorkList
3705: use %v to format interface types
Merge branch '3705-keep-blockworklist'
3699: bugfix (renamed repository_map -> local_repo_dir)
3411: code review feedback
Merge branch '3411-expire-collections'
3699: code review
3411: added new collection fixtures owned by active_user
Merge branch '3411-expire-collections'
3411: fix failing tests
3699: misc improvements
3927: add NOT NULL constraint to User.owner_uuid
Merge branch '3927-user-owner-uuid-not-null'
Bug fix: Added missing owner_uuid to user fixture.
Bug fix: migration requires CurrentApiClient
Merge branch 'master' into 3699-arv-copy
3705: rename BlockWorkList -> WorkQueue
Merge branch '3705-async-work-queue'
3413: added TrashHandler
3413: update for code review
3413: code review
3782: load only 1 MB of job logs
3413: code review update
Merge branch '3413-keep-trash-list'
Merge branch 'master' into 3699-arv-copy
3901: improve component rendering error reports
bug fix: args_retries => args.retries
3901: allow job to be either Hash or Job
Merge branch 'master' into 3901-component-rendering-errors
3901: added unit test for pipeline rendering
Merge branch 'master' into 3901-component-rendering-errors
Merge branch '3901-component-rendering-errors'
3782: restore default of no maxbytes
3961: reset object UUID when ensure_unique_name is set
3782: print log lines compatible with the log viewer
3782: keep log viewer code in _show_log.html.erb
3961: test creating subprojects under home with duplicate names
3961: more explicit assertions and error text
Merge branch '3961-uuid-assignment-not-permitted'
4038: add "fixes #" to acceptable merge commit messages
Merge branch '4038-commit-hooks-accept-fixes'
3961: fix FuseSharedTest
Merge branch '3961-uuid-assignment-not-permitted'
3782: check for Content-Range header in response
4000: fix copying pipeline instances with newer templates
3782: bugfix and added integration test
4000: tighten up functional test
4000: update both functional tests.
Merge branch '4000-rerun-pipeline-changed-template'
Merge branch 'master' into 3699-arv-copy
3699: added --force option
Merge branch 'master' into 3782-large-job-logs
3782: added state to job fixture
3782: code review feedback
3699: copy_collection should process the full manifest line
3782: fix FuseSharedTest
Merge branch '3782-large-job-logs'
3825: write log output directly to a pipe.
3687: add functional test for sharing within a group
3825: code review
Merge branch '3825-crunch-pipe-to-arv-put-final'
3699: expose to the user that a pipeline has been copied
3699: add retry support
3699: add 'force' argument to pipelines
3699: bug fix: use destination permission hints
Merge branch 'master' into 3699-arv-copy
3699: report on collection copying progress
3699: bugfix: some leftover recursive/force options
3699: allow script_version to be a branch
3699: collection copying bug fixes
3699: bug fix
3699: bug fixes
3699: copy docker image links
3699: figure out correct docker image to fetch
4015: use portable_data_hash to populate chooser
4015: test recording collection hashes for pipeline inputs
4015: fix pipeline instance tests
Merge branch 'master' into 4015-collection-chooser-portable-data-hash
Merge branch '3699-arv-copy'
3699: fix namespace clash: rename module to arv_copy
Merge branch '3699-arv-copy'
4015: simplify integration tests
4015: update FUSE mount tests (again)
Merge branch '4015-collection-chooser-portable-data-hash'
4088: filter collection file view by regex
4088: code review feedback
4088: fix alignment for right-floating input field
4088: filter only on files
Merge branch '4088-collection-show-files-filter'
Fix the goldang consarned FUSE tests again.
4040: update FUSE driver tests with stable fixtures
4040: clarify variable names
4088: use filterable.js to filter on client side
4040: code review feedback
Merge branch '4040-fuse-tests'
3857: replaced httplib2 with 'requests'
4088: code review feedback
Merge branch '4088-collection-show-files-filter'
3857: implement proxy_timeout
4294: added min_nodes config parameter
3857: code review feedback
3857: fixed import order. d'oh
4294: code review feedback
4294: additional unit tests
3857: fixed comment
Merge branch '3857-python-sdk-timeouts'
Merge branch '4294-node-manager-min-nodes'
3857: fix python-requests dependency
Merge branch '3857-python-sdk-timeout-bugfix'
3609: fix "NameError: name 'rcode' is not defined"
Merge branch '3609-run-command-bugfix'
4595: add documentation for list --filters
Merge branch '4595-document-arv-filters'
4621: implement lean_uri_escape
4621: collate_output pipes to python
4621: code review feedback
4621: drop unused variable $output_in_keep
4621: undo lean_uri_escape changes
Merge branch '4621-crunch-memory-usage'
4465: add API/Workbench Github links to help modal
4465: added api_version_text helper.
4465: test for regex link targets
4465: add trailing /? to regexes
Merge branch '4465-workbench-version-links'
4269: added job validation forbidding collection uuids
4269: clean up uuid regex matching
Merge branch '4269-no-collection-uuid-in-script-params'
4499: Normalize manifest in one_task_per_input_file
Merge branch '4499-one-task-per-input-file-normalize'
4598: add crunch-failure-report.py
4598: correct search criteria for Crunch job output
4598: bug fixes, added full stats collection
4598: fetch logs from Keep, more failure reporting
4598: take out some debug reporting and --match option
4598: remove more dev/debugging features.
4598: added failure types and short names
4877: don't delete /dev/stdout
Merge branch '4877-dont-delete-stdout'
4598: formatting and calculation fixes (code review)
4598: account for queued and cancelled jobs, fix sorting
4598: catch exceptions more aggressively when looking up pipeline names
4598: rename script
4598: actually rename this time
Merge branch '4598-crunch-failure-stats'
4922: force google-api-python-client install
Merge branch '4922-google-api-client-force-install'
4889: close the child stderr pipe before returning
5010: remove_items sets the expired_at date if available
4889: added CopyPipeToChan unit test
4889: improve error message, unit test
Merge branch '4889-crunch-deadlock'
5010: updated functional tests for new trash behavior
5010: fixed integration test
Merge branch '5010-trash-button-for-collections'
4138: support for Google Cloud Engine.
4138: code review feedback
4138: general GCE fixes
4138: GCE fixes
4138: updated unit test
Tom Clegg (4826):
un-serialize data passed by google api client
do not force ssl in dev environment
orvos-ify Collection part of api spec
add metadata resource
rename metadata_class
rename created_by_client
more initial metadata implementation
uncamelcase params
add collections.uuid
add nodes#create and nodes#ping
save Node even if ec2 create failed
add "find metadata by target kind + uuid" refs #1236, and generalize some code into ApplicationController
add pipeline resource. refs #1357
add pipelineInvocation resource. refs #1357
update dnsmasq database when node gets hostname
fix crash if no dnsmasq_conf_dir configured
use an after_update hook to keep DNS updated
fix some error message suppression in node#ping
implement generic resource#update
return errors as json, improve handling of nested params
improve error handling
remove debug message
resume logging errors
tweak stack trace log
add /nodes status page
fix status name
update gems
do not show ec2-run command for instances already up
load deploy/assets so assets get compiled at production
update Gemfile.lock
add configurable compute node domain
fix param name in node#ping
in /nodes, click row to show info[]
use acts_as_api
add public/assets cache to .gitignore
add status attribute to Node api response
copy config files and bundle install --local before deploy:assets:precompile
add shared "common" template with kind and etag
add api template for pipelines, collections, metadata
eliminate extra layer {type: ...} in API responses
add pipeline_invocations html view
add api template for pipeline_invocation
include nameservers in Node api responses
improve default nameservers list
add missing require
remove extraneous html tag
add /collections html view, dry up CollectionsController
stub Orvos::Base, add collection example
add back pg gem
add /orvos/v1/schema
make Collection.new.save work
add a little error checking for save() and save!()
stub Metadatum, Pipeline, PipelineInvocation
add Specimen resource
stub Specimen
tweak uuid generators - use last N digits instead of first N
add Project resource
stub Project
fix missing route
add boilerplate for Project resource
remove extraneous Metadata#create
fix error handling
fix serialized fields, improve error handling
add uploaded_datasets#create API. refs #1247 refs #1248
provide metadata head_kind, head_uuid in api response
beginnings of upload->pipeline status page
fix missing find_object_by_uuid
fix error reporting
put AMI id in config file instead of hard coded
tag ec2 nodes; add --client-token to ec2-run command for idempotency
rename metadata attributes to head/tail
rename metadata attributes to head/tail
implement general params[:where] and eager loading of associations
easier access to metadata, and objects connected by metadata
skip abstract classes in schema controller
use new metadata access
cache metadata
add OrvosBase#dup, offload some filtering to server
add import script
support where:{attr:[a,b,c]} query, check :eager param more carefully
reorganize API helper a bit, refactor pipeline status page as /factory_jobs
fix query encoding in API calls
delete collections/index view
add error pages
pass api_token through to orvos
stub authentication check
add hardcoded_api_tokens.rb.example
ignore editor backup files
add Log resource
add Logs resource
add missing timestamp and route, set default event_at=Time.now
add omniauth.rb.example
add User model features and uuid index
send tokenless user through orvos login instead of giving up
use factory_jobs#index as front page, disallow robots
connect user authentication to API client authorization
install omniauth.rb during deploy
set format=json explicitly when calling api
use Accept header instead of post arg to ask for json
fix missing user variable
set created_by/at and modified_by/at attributes when saving objects
add permission checks
add example graph of user/project/permission metadata
host d3 js locally
move html into app/assets
Revert "move html into app/assets"
give up on assets for static d3 example
fix error response
handle invalid token more gracefully
move development.rb to .example
fix up factory_jobs status page
start pipeline after importing
add Pipeline#dependencies
handle non-db-column API response fields a bit better
fix use of OrvosBase#new when unpacking lists
move created_by to owner
fix exception when current_api_client==nil
fix check for owner==current_user
fix ownership-change permission check
fix OrvosModel#index permissions
fix db query
rename Metadatum to Link, head to head_uuid, tail to tail_uuid
add some admin privileges to make bootstrapping possible
rename Metadatum to Link, head to head_uuid, tail to tail_uuid
tweak eager loading
tweak eager loading
add Specimen#properties
fix default Link#properties
add PipelineInvocation#active, and try to set success=false when needed
bit more robust in error display
fix (Time.now - nil) exception
improve error handling and reporting
basic d3 graph at /collections
fix unsupported where() style in find_object_by_uuid
do find_object_by_uuid after setting up api token
tighten up automatic uuid assignment
create content-addressed collections with uuid=locator
add /orvos/v1/users
fix permissions to accommodate nodes#ping and user_sessions#create exemptions
move development.rb to development.rb.example
add /users/current method
add User resource, restrict generic resource pages to admin users
spice up collections graph a little
add bootstrap
support return_to after logout
fix login bug
add Log out button, hide admin-only nav links if not admin
add tabular /collections
add table/graph nav pills
move logout link to right
allow current_user => nil for pages that do not require login
support user and API client using different URIs for same API endpoint
add deploy files for ant install
add Job and JobStep resources
fix setting modified_* and default owner when creating objects
add Job#priority
add jobs and jobsteps
add permission logging
fix "home" link
add some nil checks
add Job#dependencies
support limit parameter in #index
limit data page to 1-3000 collections
add Collections#show page
include data_size and files list in Collection api response
fix commit-id -> redmine link
make bootstrap tabs really work
human readable data size
limit 100 instead of 1000
obey eager() and limit() in all()
default to data page instead of factory_jobs
use real instead of test data size
abandon "bold megabytes" feature
split jobs into provenance chart and job list
add Files pane on Collections#show
fix storage=cache pill coloron Source data tab
add orvos_resource_list#to_hash
use JSON.dump so timestamps load back as Time, and Oj.load for fast loading
add logs#show
update server-assigned attributes after saving objects
show log#event_type
add generic #show page for orvos objects and use it for Log and User
make resource_class_for_uuid more convenient and robust
fix misleading label
fix double render
link to pipeline invocations from factory_jobs page
add links to generic specimen and project #show
use old owner, not new owner, when checking permission to change attributes
sort lists on modified_at by default
use generic orvos link helper in collections#index
use Enumerable and to_ary for implicit conversion to array instead of forcing caller to use where()
do not hyperlink collections that do not have corresponding objects to look up
do not require user login for node#ping
set modified_by_user=nil instead of breaking if no user is logged in
check serialized attribute types during create/update
render pipeline_invocation components as job/progress table
emphasize local part of uuid
be more selective about invoking array-of-strings display on generic object#show page
fix broken orvos_model#initialize
allow re-saving an existing collection without complaint
do not delete uuid if provided in new object (notably collection)
accept json-encoded pipeline[components] param
let pipeline_invocation#active be a regular attribute
Rearrange pipeline_invocation handling
set exome regions
add PipelineInvocation#properties
show raw components json on pipeline_invocations#show
add first,last,[] methods
move vcf pipeline stats code to a helper, generate stats when pipeline finishes, cache in #properties
tweak factory_jobs display
show "sinfo" results on /nodes page
add pipelines#show
add region_padding=10
add a Link from specimen to collection when pipeline finishes (instead of storing summary info in the pipeline_invocation)
add fill_ftp_accounts script
add picard-casm statistics to factory job summary
add chromosome_calls and inferred_sex to factory job summary
prepare pipeline summary CSV and save in specimen->result link. refs #1239
ignore Keep hints when verifying collection uuid == md5sum
use spot instances for compute nodes
refuse to change ec2_instance_id once an EC2 instance has claimed a Node
rename Project to Group
rename Project to Group
add script/dispatch_jobs.rb and related Job features
upcase job command parameters when passing to whjobmanager
restart dispatch_jobs service on deploy
fix no-tasks-todo -> success logic
add Jobs#resource_limits attribute. refs #1417
Do not try to start a job when too few slurm nodes are idle. refs #1417
Fix slurm node counting, and pass along min_nodes to salloc. refs #1417
fix dispatch_jobs restart in 3260a10f0a9396b1ad0b8abe34f9808927311876
add output to Job api response. refs #1417
record started_at, and command_version reported by whjobmanager. refs #1417
Add Job#resource_limits to API response. refs #1417
add /jobs page. refs #1417
enable auto job links
clean up /jobs page. refs #1417
add helpers: current_group_permissions, groups_i_can(:read). refs #1415
show progress bar and tasks summary on /jobs page. refs #1417
current_group_permissions, groups_i_can(:read). refs #1415
relocate permissions code, use resulting :read permissions in #index. refs #1415
add Job#show template. refs #1417
add group#index and group#show pages. refs #1415
generate discovery document at /orvos/v1/rest. refs #1406
rename job command to script. refs #1416
fix cache keys
rename job command to script. refs #1416
rename pipeline invocation to pipeline instance. refs #1416
rename pipeline invocation to pipeline instance. refs #1416
rename pipeline to pipeline template. refs #1416
rename pipeline to pipeline template. refs #1416
rename job steps to job tasks. refs #1416
rename job steps to job tasks. refs #1416
fix exception when tasks_summary[:running] == nil
add per-user and per-session default_owner. refs #1415
use new permission methods to check model transactions. refs #1415
fix call to api_client.uuid when api_client is nil
handle parameter types :object, :datetime, :text. refs #1405 refs #1406
improve error checking
add create method. refs #1406
generalize output logic. refs #1405 refs #1406
add update method. refs #1406
add delete method. refs #1406
fix list logic. refs #1405 refs #1406
add Link to known-types list
use ORVOS_API_TOKEN environment variable in rails console
add "usual orvos object details" tab on collections#show page
add Collections tab to groups#show page
use label instead of btn for labels on collections#show
add names and keep flags to groups -> collections list. refs #1408
tweak table border
show PGP huID tags. refs #1408 refs #1419
add wh-pipeline. refs #1424
fix rdoc. refs #1424
add /pipeline_templates index page. refs #1420
tweak generic object display
rename wh-pipeline to wh-run-pipeline-instance. refs #1424
accept pipeline template uuid in --template argument, improve docs. refs #1424
add pipeline_instances#index, fix #show to work with new-school pipelines. refs #1421 refs #1422
create/update PipelineInstance while working. some code cleanup. refs #1440
show pipeline instance uuid in plain text status. refs #1440
do not include ApiClient* in discovery document. refs #1405 refs #1406
add config.git_repositories_dir. refs #1445
Support searching jobs by script_version_descends_from. refs #1445
Tweak git commit lookup. refs #1445
pass component[:script_version_descends_from] to server when searching for jobs. refs #1445
fix crash on non-string job#script_parameters
hack search on serialized fields. refs #1447
pass component[:script_parameters] to server when searching for jobs. refs #1447
tweak serialized attribute search. refs #1447
add --create-only and --instance arguments. refs #1448
set instance.success when finished. refs #1424
add jekyll doc tree
fix "active section" style
add ssh keys page
tweak front page of doc
rename ssh keys -> ssh access
add provenance, using git, using keep pages
use arvados.org favicon
highlight active side nav
add a bunch of API pages
add job pages
add Keep vocabulary
fix ssh link
split User into User Guide and API Reference
use bootstrap nav-list instead of hacked nav-stacked
tweak front page
xref to list of resources
fix up api index
rename api resources pages
add api/Links
add api/Jobs
fix titles of api/Jobs and api/Links pages
fix links, rename orvos
rename explorer -> workbench
add api/Collections and unlink missing resources on index page
rework front page
tweak page titles, add link to arvados.org
mobile fix for doc side nav
fix doc front page header image
fix doc front page subtitle word wrap
fix doc front page title/text alignment
fix doc header alignment some more
fix doc header alignment some more
change favicon
fix favicon path
add "incomplete" alert boxes
rename projects
fix migration
clean up workbench menu bar
cope better with unexpected pipeline structure
tweak front page title kerning
start ruby+sinatra keep server
change keep server to sinatra modular style
add PUT and SHA1 support
start ruby sdk
fix *List id and description in discovery document
add KeepDisk resource
report filesystem uuid to arvados
add controller-specific routes to discovery doc
accept arvados host/auth parameters via constructor
fix keep_disks/ping routing & discovery
use proper ping method
keep_disks#ping
more ping info
add job tasks docs
accept authorization token in Authorization request header
start python sdk
add attributes to job_task api response
remove duplicate resource parameter
serialize JobTasks parameters attribute
python sdk JobTask convenience methods
add crunch_scripts/hash
copy whjobmanager from warehouse-apps
start Perl SDK
catch and propagate errors during #update
fix response type for #index
add properties to pipeline_instance api response
add script/update_node_attributes.rb
support filters like where={"attribute":null}
improve error messages
support order param
add /arvados/v1/jobs/queue
move queue and system user logic out of dispatch_jobs
fix sym/string comparisons
add job_task attributes
start lifting job manager over from free factories to crunch
fix empty-string error report
fix job_task qsequence and delete_all usage
note created_by_job_task when queueing new tasks
drop shim for errant discovery doc
supply entire client-provided object if discovery doc does not describe structure
remove lockfile
add "where" param to discovery doc
crunch-job fixes
ignore client-provided kind and etag during update
rename outputkey to output
fix job_task update call
reject API calls with unsupported parameters
add "order" parameter to discovery document
run job tasks in qsequence order in first pass
retry requests after BadStatusLine
remove old cruft
fix job output hash key
fix up quoting
fix "is undef" test
use IPC::System::Simple::capturex instead of system()
tweak node failure detection
check child exit status
check validity of script_version during job#update
use gmtime instead of unixtime to report timestamps
fix logging of script_parameters and resource_limits
add AuthorizedKey resource
add authorized_keys
update bundle, bring in bootstrap-editable
add authorized_keys views and editable fields
tweak authorized_key#new
make authorized_keys editable on index page
add andand
fix up editable stuff
implement destroy method in arvados model proxy
add insecure-https config option, improve error reporting
more x-editable
fix redirect/error behavior for session timeouts
add VirtualMachine resource
add VirtualMachine resource to workbench
include indivually permitted objects in #index responses
add admin cheat sheet
expose Keys, VMs, Jobs tabs to non-admin users
include links that reference me in links#index
add users#index, allow non-admin users#show
add links#index, allow non-admin links#show
do not render attributes of non-owned objects with x-editable
fix current_user in model
add /authorized_keys/get_all_logins
add Repositories resource, fix authorized_user attr name, some wb fixes
always clear thread.current session info
remove less critical nav items
smarten up authorized_keys#get_all_logins response
add user->repo permission to cheat sheet
show cli update example on generic object#get page
include csrf token with x-editable ajax
improve error reporting
move authorized_keys#get_all_logins to virtual_machines#get_all_logins
add repositories#get_all_permissions
add front page dashboard
add tutorials and references to home page
fix up load_where_param
fix up wh tool
rename wh to arv
update "application name" identifier
rename secret_token.rb -> secret_token.rb.example
update cli sdk doc, tweak json output options
fix up formatting
add unverified SSL certificate note
fix formatting
add ApiClient#is_trusted, show list of api tokens in Workbench
add python example to generic workbench object display page
pluralize resource name in python example
nav menus
whitespace
update to new nav menu
add trusted client info to install doc
fix up cli/curl example code
always use https in workbench curl examples
use arv instead of curl in admin cheat sheet
port crunch dispatcher from whjobmanager to crunch-job
fix http status for "not logged in" response
handle auth tokens with api_client=0 better
more crunch install instructions
remove emacs tmp file
remove stub notice
update job intro
fix here-docs
fix crunch script path
add missing attributes to keep_disk api response
add CollectionWriter and Keep classes to Python SDK
make dependency install instructions paste-friendly
avoid sending html redirect to api client
tweak daemontools scripts
tweak daemontools scripts
obey RAILS_ENV if given
configurable setuid for crunch jobs
update docs
work around sudo environment sanitizing
send git-archive from ctl to compute node instead of running git-pull on compute
add likely job dependencies section
show full uuid even in uuid column on index pages
fix dev job id
fix attempts to save dev job status in arvados db
remove some leftovers from subversion support
update dev job docs
use find_or_create_by instead of rescue RecordNotUnique
use which to find crunch-job instead of hoping sudo preserves PATH
add Reader classes, local storage mode, and basic test suite
hash and grep examples for new reader/writer tools
fix for bare repos
fix KeyError in os.environ check
add missing newline at end of manifest
reader fixes
docs and fixes
add transparent bz2 decompression, some tests and fixes
add transparent gunzip support, fix string concatenation performance
fix permissions and bugs, use decompressed_name in output
add jobs#log_tail_follow
document passenger streaming setup
add user#is_active flag
activate admin users created before is_active flag existed
user#is_active fixes
link JobTasks api page
notify admin about new users
fix grammar
add util class
add trait resource
fix editable User attributes
fix active/admin editables
fix editable + serialized-hash attributes
update dispatch docs
remove "create new api client authorization" button
add Admin menu with Users and Groups
add Human resource
fix human/humans inflection
fixes and docs for testing crunch jobs locally
add pgp-survey-parse
fix up default site name
add workbench favicon
add pgp-survey-import
chdir out of supervise dir when starting job
document redis dependency
update running and finished_at attributes at completion
use the same API host for all examples
more specific info about VMs page
remove "add a new VM" button from non-admin VM index page
collections does not need to be admin-only
add Data menu
add "first job" tutorial
remove stub pages
point to doc.arvados.org tutorial if no local tutorials are set up
fix doc link
fix tutorial bugs
update example configs
restrict visible-because-references-me to permission and resource links
show login names and config instructions on VMs page
add repository usage example and doc link
fix inflections
start pgp data tutorial
add identifier link to conventions list
rename foreign uuid attributes
rename more foreign uuid attributes
document adding crunch user
rename more foreign uuid attributes
update example collection hash
reword "change api endpoint to suit" reminder
fix up collections index/show
show provenance
update provenance link conventions
remove special group/show stuff
restore uuid column
remove special group/show stuff
update uuids for pgp data example
quote yaml title
fix word wrap
update language on job intro page
use pattern as regexp
fix is_locked_by reference
fix is_locked_by reference
extend pgp data tutorial
fix nil bugs
hide expired and job-specific auth tokens from workbench list
pipeline_templates and pipeline_instances are not admin-only
clarify monitoring jobs until success=True
remove extra newline
add output & code notes to job tutorial
add util functions, fix up tmp dirs
lazy check for # files actually retrieved
add transparent decompression option to util.collection_extract
add GATK2 VariantFiltration example crunch script
add piwik info to workbench install doc
link to "ssh access" doc page from "authorized keys" workbench page
open up specimens page index and show pages
add "arv keep check"
update git commit hash link target
add GATK VariantFiltration tutorial
tweak example templates
comments
add CollectionReader.manifest_text()
add collection-merge script
fix job dependencies -- grok arrays and hashes in script_parameters
prepend to PYTHONPATH instead of replacing
fix namespace
add usr/local/arvados/src pythonpath
strip extra newline
link job and collection uuids to workbench pages
rename wh-run-pipeline-instance to arv-run-pipeline-instance
add "arv pipeline run" -> "arv-run-pipeline-instance" wrapper
fix command line parsing
add new-pipeline tutorial
new dashboard content
use "nano" instead of pseudocode "edit"
fix pipeline component names
fix "pipeline run" command
count accessible objects instead of owned objects on dashboard
count files instead of specimens on dashboard
hide account summary bar if user is not activated yet
use find() instead of where() to get a single object
add --no-reuse and --no-reuse-finished options to arv-run-pipeline-instance
say "interrupt" nicely instead of a stack trace on ^C
fix backward-compatible env var
fix bash "read -d" usage
fix top nav extra padding with responsive + small window
clean up SDK examples
show copy-paste-friendly current token at top of API Tokens page
fix example ssh command
add auth scopes
gitignore arvados-cli gem
fix exception in get_all_logins when user has no authorized_keys
put back :find_object_by_uuid in virtual_machines controller
add links beside editable object uuids
lock CRUNCH_DISPATCH_LOCKFILE while running, if provided
update crunch install doc
fix uuid column heading on jobs index
tone down css on jobs index
clean up jobs page, add expand/collapse
fix spelling of arv argument
use ssh -T to silence pty warning
allow admin to change object UUIDs
fix :id -> {uuid} subst in discovery doc when generating from rails routes
replace compute_node_security_group config with generic compute_node_ec2run_args
Revert "remove extra newline"
add crunch_scripts/file-select
add zipball_extract
allow caller to override std* pipes in run_command
fix infinite loop in flush_data
use . as default stream name
fix stream names in CollectionWriter
add one-task-per-stream utils
mkdir -p in collection_extract()
fix out of scope function use
fix os.path.dirname usage
add util.listdir_recursive()
overwrite existing files in stream_extract
fix extra empty stream in all_streams() response
maintain directory structure in collection_extract
add CollectionWriter.write_directory_tree()
escape spaces in manifest filenames
Modify API discovery document and CLI tool so resource objects can be
pass api_token and pipeline_instance in request body instead of query string
During task setup, if it exists, run /crunch_scripts/install instead
update "correct answer" hashes in collection tests - see 3f431d852579ee433197fb1c29e6db374ccb4b9e
test API update semantics, confirm put/get cycle preserves components{} hash
do not munge hash keys in client-provided request. fixes #1516
update API tokens doc to match current API tokens page
fill in intro-to-git page
move crunch-job to arv-crunch-job and add to arvados-cli gem
add ruby wrapper around crunch-job
fix exception when script_version is nil
add local job debugging tutorial. closes #1515
Fix exception if api_client_authorization does not reference a valid user.
Ignore failure in data-migration part of old schema migration.
fix down-migration: add_column should have been remove_column
add a few fixtures and tests for API. refs #1517 refs #1510
Migrate the test database to 0 instead of trying to "drop database".
add note about running crunch jobs with no git repo at all
add crunch scripts for Real Time Genomics pipeline
add Real Time Genomics pipeline template
rename jobs.resource_limits to runtime_constraints
set/unset ARVADOS_API_HOST_INSECURE in cli setup instructions
upgrade gem versions in Gemfile.lock
fix infinite loop if CONT signal received while todo is empty
fix infinite exec loop
allow crunch-job caller to override CRUNCH_TMP. fixes #1593
update node info[:slurm_state] when running sinfo. fixes #1606
send queue/worker stats while waiting for job to start. fixes #1591
improve error message when crunch-job is not found
store the last ~512 KiB of job log messages in a redis buffer. fixes #1589
in log_tail_follow, emit recent messages at top of live stream. fixes #1590
improve error message when API response is valid JSON but not a hash
include "href" key for each object in API response
upgrade gem dependencies
upgrade gem dependencies
upgrade to latest Rails 3.2
Remove "test user needs createdb privilege" workaround.
Add "not null" constraint to timestamp columns.
Preload all models at startup. Otherwise, in non-production mode,
Use current_user to select API response template. fixes #1594
fix tests and behavior for nodes#list as called by inactive user. refs #1594
add crunch_worker_state to Node api response. refs #1586
Catch AbstractController::ActionNotFound instead of deprecated
add log_stream_href attribute to Job resource
add generatedAt field to discovery document
update discovery document generator for rails 3.2
delete cached discovery document during boot, so doc matches running code
add discovery document test
fix loop label syntax error introduced in 7c4460b82b8abadc5144f758a9f36ce0f42893e8
reduce default #attempts per job task from 9 to 2
fix over-compensation for attempts counter when task fails on suspect node
Use more current slurm environment variable name in error message scraping
fix error message scraping regexp
Allow all active users to retrieve list of keep nodes.
Obey ARVADOS_API_HOST_INSECURE and use the debian trusted CA list if
Use httplib2 to connect to Keep instead of executing whget/whput
Add test case for reading and writing small Keep blocks
Use hash to determine probe order
turn on logging if ARVADOS_DEBUG environment variable exists
Allow all active users to retrieve list of keep nodes.
Obey ARVADOS_API_HOST_INSECURE and use the debian trusted CA list if
Merge branch '1600-native-keep-client'
Fix crash building api response when user is nil
add tests for node#ping, fix response when ping_secret is invalid
Include privileged attributes in node#ping response
Add Admin/Nodes page
Avoid Rails cache when polling database for latest node/job state
Move @redis.exists and .getrange outside .subscribe block. Otherwise
Revert anjlab-bootstrap-rails to version 2 because bootstrap-editable
Read pipeline template from local file if --template arg is not a uuid
Send job log messages to owner_uuid redis channel
Add /users/:uuid/event_stream action to subscribe to per-user channel
Ensure script_version matches before satisfying pipeline component with a job.
remove unused class DataReader
Add threading locks in Keep client
Document user#event_stream. closes #1605
add doc/Gemfile
Update development-mode crunch-job notes
Do not re-attempt failed job tasks, unless there is some reason to
Fix job script attribute (not script_name), and update script_version
Fix navorder of API resource pages
Move api/job-tasks to api/crunch-scripts and add some explanation
Send "Accept: application/json" header with API requests. refs #1604
Refuse to stream responses unless request Accept header has text/plain
Fix encoding issue so Python clients can store binary data in Keep
Add threaded Keep.put()
add script/create_superuser_token.rb
Fix up keep_disks/ping and add tests. refs #1581
Fix base class of ApiClient model so as_api_response works
Allow users to cancel a running crunch job by updating cancelled_at
Fix missing "my" scope.
Fix scalar gmtime() usage.
Avoid infinite poll loop triggered by (:running and !(:finished or :cancelled))
Avoid starting new jobs when TERM signal has been received.
Check refresh trigger in crunch-job instead of crunch-dispatch.
Update comments in config/environment/ files.
Fix config setting name.
Add job.cancel API.
Update crunch_refresh_trigger config setting name in tests.
Do not require filesystem_uuid in first keep_disks/ping request.
Use new-style Python classes (subclass object), so super() can work
Clear TASK_WORK before starting a task.
Show how crunch job tasks get their runtime parameters. refs #1513
Add arv-put, a whput replacement built on the Python SDK.
Add --as-stream test.
Add CollectionWriter.data_locators()
Add arv-put --as-raw option.
Add arv-get, a whget replacement built on the Python SDK.
Remove Rails default scaffold CSS.
Add delete button to generic Workbench index page.
Make the sample "Add authorized key" button non-clickable.
Command line usability improvements.
Fix sha1 test, dry up "remove ./tmp/foo" preparations
Improve compatibility with whput.
Use collection hash instead of entire manifest as symlink target
Allow caller to override close_fds in util.run_command()
Pass through stderr in real time by default in util.run_command()
Add arvados.util.clear_tmpdir() function.
Sort filenames when writing multi-level streams in
Accept data from a generator in arvados.CollectionWriter.write()
Accept empty string as root stream name "." in
Add arvados.getjobparam() convenience function.
Fix PermissionDeniedError when a non-admin user is the first to log in
Fix failing test by parsing timestamp correctly.
Check permission before deleting objects
Use some helpful HTTP status codes, not just generic 422.
Remove spurious "if".
Ignore tmp directory created by cli tests
Ignore config/environments/test.rb and production.rb
Add example scripts and libraries for bwa, samtools, picard, gatk2
Use body={...} instead of object_type=json.dumps({...})
Fix crash when destination directory supplied without trailing slash,
Use our own exception classes, instead of just Exception
Fail more gracefully on some common error conditions
Fail gracefully when fetching a raw block that does not exist
Support "arv-get {locator}/filename path/to/localdir" invocation:
Add arv-get -f (force overwrite) and --skip-existing flags.
Merge branch 'master' into 1646-arv-put
Clean up arvados_bwa module and add docstrings.
Support the use of groups to grant repository permissions.
Add "Create standard objects" page in install docs.
Rename doc page to match title.
Add python client library package: arv-get, arv-put, arvados module
Support invoking arv-get and arv-put via "arv get" and "arv put".
Add "Install client libraries" page to doc/install/.
Point "arv keep get/put" to arv-get/put and unsupport "arv get/put"
Merge branch '1646-arv-put'
Add docstrings to arvados_samtools module.
When running "arv-get -n", it is OK if destination files already exist.
Add GATK2 exome fastq->vcf pipeline example
Clear $CRUNCH_TMP/src when starting a job.
Add "crunch examples" and "crunch utility libraries" pages
Use git at git.[...] not git://git.[...] as default repository fetch_url.
Remove unused module import.
Fix reference to picard source distribution -> binary distribution.
Add input parameter tables and source code link for each crunch example.
Rename annotation DepthOfCoverage to Coverage
Revert "Rename annotation DepthOfCoverage to Coverage"
Fix gatk_tbz locator.
Store crunch-job log in a collection instead of just raw data blocks.
Fix left nav link order
Remove TMP/src.commit too, when removing TMP/src.
Handle unexpected server responses better, clean up usage/help code
Stream API responses for jobs.log_tail_follow and users.log_stream
Fix parsing of buffer_size parameter in jobs.log_tail_follow
Set jobs.log_tail_follow default buffer size in discovery document.
Obey ARVADOS_API_HOST_INSECURE if host came from ARVADOS_API_HOST env var
Add example ruby client: list idle/busy worker nodes and age of last ping
Fix some typos and long lines
Remove extra trailing newline in --raw mode output
Return empty-block locator in data_locators() for zero-byte stream
Change arv-put default behavior: write a manifest, even for a single file.
Propagate API errors as exceptions
Return API response objects as Models (with save() etc) instead of Hashes
Add new user activation + setup script. closes #1738
Fix up usage error message
Accept an API version argument to arvados.api(). refs #1772
Add Python SDK doc page. closes #1754
Update favicon to Dax
Use arvados.api('v1') accessor instead of arvados.service. refs #1773
Specify api_version when instantiating Arvados client. refs #1773
Specify apiVersion when instantiating Arvados client. closes #1773
Redirect to a suitable page when HTML response requested for create/update.
Do not ignore attributes received with create request.
Fix exception when no authorized_key hash provided in create request.
Allow self-activation after signing required user agreements.
Add --yaml output format choice to arv.
Fix return_to on generic "Create" button.
Do not self-activate, or show user agreements to, an uninvited user:
Store correct collection uuid in signature/click link.
Support user agreement viewing and signing in Workbench.
Divert processing when user agreements are needed, instead of just
Merge branch '1694-user-agreement'
Allow authentication from multiple OpenIDs using permission links with
Support pre-activation by creating a new user record and OpenID
Request activation if is_active==false but no user agreements seem to
Merge branch '1675-preactivation'
Do not set is_active flag for new users. They do that themselves now,
Fix unexpected nil in @required_user_agreements. refs #1699
Do not check for user agreement signature before showing user agreement.
Add "items_available" (integer) to generic list response.
Use items_available in API response instead of retrieving all items.
Fix error message: print the exit status, not the string "$?".
Link to show/download files from collections. refs #1748
Offer to show html files in browser, too. refs #1748
Fix error message: print the exit status, not the string "$?".
Merge branch '1748-workbench-view-files'
Merge branch '1694-user-agreement'
Serialize objects/arrays in query params with json.dumps() instead of str().
Remove json.dumps() from example and tutorial code.
Accept comma-separated list of locators in Keep.get()
Reword tag convention. refs #1508
Remove extra line breaks in textile paragraphs.
Read data from local file if an absolute filename is given for an
If there is only one user agreement, show it inline with an OBJECT tag
Fix mime-type lookup.
Check HTTP status before using API responses.
Ask API server for profile information. refs #1815
Provide response[_profile][request_time] if requested by client. refs #1815
Show profile data in log if config.profiling_enabled. refs #1815
Show some params in profiling output. refs #1815
Use operating system's bundled CA certificates if available.
Obey profiling_enabled config. refs #1815
Merge branch '1792-workbench-performance'
Fix check for whether adding response[:_profile] is safe. refs #1815
Fix up "age" column contents on workbench home page.
Fix up collection links on workbench home page.
Add "arv delete" example.
Add "Links" entry to Data menu. closes #1852
Add "arv delete" example.
Add "Links" entry to Data menu. closes #1852
Fix mime-type lookup.
Fix crash when a discovered parameter is not supplied. closes #1866
Remove debug message
Hide fetch_url column on repositories page. closes #1863
Add arv-tag to arvados-cli package
Update collections index to match front page table, and add tags column.
Add AJAX tag search box to collections.index page.
Fix crash if no search parameter given.
Update query in browser location bar when showing search results.
Unescape filenames from manifests. refs #1849
Allow admin client to create API tokens for other users.
Use symbol keys to access request parameters.
Add collection search box on home page.
Fix test setup.
Merge branch '1499-workbench-tags'
Switch to rounded search box style on collections page, to match home page.
Merge branch '1499-workbench-tags'
Fix sending resource body to server.
Lowercase model name in discovery document, like the real parameter name.
Obey permissions when retrieving object by UUID.
In collections.create, set owner to root and add a permission link
If a GET fails while looking up a friendly name, skip the hyperlink
With where={"any":["contains",X]}, search all text and string columns.
Reload the target object before updating if it is marked read-only
Fix incorrect find(), should have been find_by_uuid()
Fix user_agreements_controller.table_name so generic list method works.
Fix user_agreements.sign when collection uuid has +size hint.
Do not show user agreement inline unless site config says so. Remove
Inactive users can get a list of Keep disks.
Remove boilerplate comments and unused test fixtures.
Fix "ignore client-provided values for protected auditing attributes"
Do not attempt to load authentication config file unless it exists.
Use AJAX when adding a new authorized_key from the user home page.
Respond with an error status if save fails during create method.
Use render_error instead of custom error logic in update method.
Tweak error messages to render nicely as Rails "full messages".
Reduce boilerplate on error messages.
Accept DSA public keys ("ssh-dss"), not just RSA.
Add themes to .gitignore in apps/workbench
Merge branch '1880-check-duplicate-public-key'
Normalize collection UUID to {hash}+{size} before save and lookup.
Remove +K@{name} hints from collection uuid, link head/tail, job
Set cache expiry time on discovery document.
Cache discovery document in ~/.cache/arvados/discovery/.
Render token with ApplicationController#show instead of calling
Add link to "create standard objects", and delete obsolete tutorial
Update bundle. Avoid google-api-client 0.7.0 (rc2 is incompatible) and
Add section about adding Keep disks during install.
Fix circumventing usual permission mechanism in nodes.ping.
Merge branch '1922-cache-discovery-python'
Merge branch '1881-normalize-collection-uuids'
add list-inactive-users.rb script.
Always use string keys instead of symbol keys for serialized attributes.
Always accept properties and info as serialized attributes.
Update schema migration version.
Fix keep_disks.ping to accept a correct ping_secret in lieu of a
Add /keep_disks page to Admin menu.
Change underscores to camelcase in "kind" strings as documented.
Fix crash on error pages by removing before_filters that assume there
Provide UUID prefix used for each schema in discovery document.
Use discovery document to build infix->kind map instead of using a
Update tests to match recent permission changes; dry up "404 if no
Do not run ec2 commands in test mode.
Ensure we always respond with json (except the redirect at the root
Always use symbol keys for object attribute names and string keys
Replace Admin menu with System menu (users can see those things too).
Add virtualMachine test fixture.
Change underscores to camelcase in "kind" strings as documented.
Do not offer to let non-admin users create new keepDisks and nodes.
Remove Keep prototype (moved to branch 1747-keep-prototype-ruby).
Fix crash when no config file exists, and fix config access from Model
Add tests, fix dependencies so "bundle exec rake test" runs.
Send parameters in the request body instead of query parameters if
Add Gemfile.lock
Fix up dependencies, pin google-api-client at 0.6.x
Do not add body object to discovery parameters.
Add version string to arv (assume "gem install" makes the version
Remove request body object from parameters lists in *.create and
Merge branch '1946-request-body-vs-parameters'
Add pipeline_instances/compare page
Clean up /arvados/v1/../../discovery/v1/ hack before sending request
Show condensed data about pipeline instances side by side on "compare"
Fix regular expressions in database migration.
Remove +K at foo parts from collection uuids in jobs' script_parameters
Fix jobs.cancelled_by_*_changed? calls to jobs.cancelled_by_*_uuid_changed?
Skip "specified commit exists" validation when updating an existing
Add functional test for "skip script_version validation when not
Fix "show all pipeline templates" link target on dashboard page.
Add pipeline_instances/compare page
Show condensed data about pipeline instances side by side on "compare"
Use generic index page for /users
Add breadcrumbs bar to #index and #show pages.
Allow controller to override page name for breadcrumbs.
Remove extra breadcrumbs bar.
Merge branch 'master' into 1978-compare-pipelines
Add basic select-and-compare interface to pipeline_instances#index.
Fix crash when component has missing attributes (e.g., no job was run).
Fix scoping for error classes.
Add StreamFileReader.seek() method.
Add arv-mount.
Improve progress view on pipeline_instances.show page and add AJAX
Use AJAX/remote requests when deleting objects. Add delete button to
Refresh dashboard with AJAX every 60 seconds.
Show "loading" indicator in nav bar while waiting for AJAX requests.
Ensure util.mkdir_dash_p() fails if its target is a non-directory, and
Fix breadcrumbs crash if link has no tail_kind or head_kind.
Merge branch 'master' into 1977-provenance-report
Merge branch '1976-pipeline-progress'
Hide this/other/both legend if only one pipeline_instance is shown.
Ensure permissions can be propagated via group owner_uuid attribute.
Add tests for permissions enforcement in collections.provenance.
Move can_read permission logic into ArvadosModel.readable_by scope,
Merge branch '1977-provenance-report'
Merge top nav menus into brand dropdown, move breadcrumbs into top
Hide loading indicator after refreshing dashboard tables.
Merge branch 'master' into 1979-workbench-ui
Move all index content into Recent and Help tabs.
Use object type or friendly_link_name as browser title.
Move show-object content into tabs.
Fix (and add test for) 404 at repositories.get_all_permissions.
Remove old views/collections/show template.
Merge branch '1979-workbench-ui'
Fix script_parameters comparison on pipeline_instances.compare.
Skip repeated find() calls for the same object during a single page
Fix handling of pipeline components with no job/output yet.
Call super in pipeline_instances#show to make preview/js/json
Add progress bars to pipeline instances table on dashboard.
Fix up job/pipeline progress bars on dashboard.
Fix progress bars on pipeline_instances#show
Expand graph border to tab pane width.
Remove extra breadcrumb separator in "no breadcrumbs" mode.
Use .badge-alert to highlight notification badges.
Clean up collections index and show->files tables.
Highlight "Dashboard" instead "Users" in left nav when on
Rearrange thread_with_api_token around_filters so current_user is
Remove extra divider in notifications list.
Fix up pipeline_instances#index and #compare.
Fix up collections search box on dashboard.
Fix up collections#index search box with bootstrap3 styles.
Fix up tab titles.
Add space under tab row.
Add section headings to API examples.
Merge branch '1979-workbench-ui-TC'
Clean up groups/index and add delete buttons.
Fill col-sm-3 space with left nav, and move it to the bottom when
Fix top nav behavior when window is narrow.
Merge branch '1979-workbench-ui-TC'
Retrieve manifest_text from API server. If that fails, emit a warning
When retrieving a blob, if local Keep servers don't have it and the
Include log buffer in Job API response.
Fix dashboard auto-refresh, and show last line of log_buffer (if any)
Merge branch '2008-show-log-buffer'
Mock up add/remove tag UI.
Return log buffer "(not available)" instead of crashing if redis is down.
Merge branch '2070-read-remote-arvados'
Dismiss "paste your SSH key here" notification when key successfully saved.
Look in method[request][properties][$ref] for resource object
Replace weird "loading" animation with a nice plain glyphicon.
Apply usual parameter encoding logic to method[request] parameter too.
Show first 200 items on index page, not 1000.
Allow passing hashes as resource attributes. This lets crunch-job keep
Fill in missing example locators for crunch wrappers.
Fix git_dir path in CommitAncestor model. Fixes #2089
Fix pipeline_instances#show by renaming partial.
Fix terminology agreement between left nav and elsewhere in docs/pages.
Fix up legend colors and labels on pipeline_instances > compare > graph.
Limit "compare" selection mechanism to 2 or 3 pipelines. Closes #2101
Hide "Create" button on pipeline_instances and pipeline_templates pages. Closes #2098
Show owner name instead of uuid on pipeline_templates#index. Closes #2097
Merge branch 'master' into 2096-demo-feedback
Merge branch '2096-demo-feedback'
Change color of "running" job label to blue.
Fix 404 at api_client_authorizations.create_system_auth and add tests.
Fix crash when a script_parameter value is a Fixnum. fixes #2167
Normalize collection hashes provided by clients as attribute values
Fix crash in collections.provenance when a script_parameter is a Fixnum.
Provide manifest_text in collections.get API response. refs #2070
Hide left nav panel if current_user is not active. refs #2078
Prevent table headings from scrolling out of view on collections index
Move HeaderRowFixer out to application.js and application.css.scss
Output a blank line between pipeline status updates. closes #2182
Say when re-using existing jobs, even without debug mode. closes #2183
Hide top nav notifications and links to tokens/keys pages until user
Return only true or false for is_invited, never group uuid.
Fix crash if @notifications is nil.
Fix symbol key used in serialized attribute. closes #2186
Merge branch 'master' into 2179-demo-feedback
Render collection provenance graph top-down like other graphs.
Merge branch '2179-demo-feedback'
Merge branch 'master' into 2060-edit-tags-in-workbench
Fix up tag and "add tag" button display.
Add and remove collection tags with ajax.
Do not show "delete tag" button unless current user owns tag.
Light up "add tag" buttons on button hover, not cell hover.
Fix fixed-header-row hack. closes #2194
Merge branch 'master' into 2060-edit-tags-in-workbench
Fix breadcrumb-hiding on user agreements page. closes #2191
Lighten and add tooltips to "delete tag" buttons.
Update fixed header rows after new content arrives via ajax.
Merge branch 'master' into 2060-edit-tags-in-workbench
Add {repo}/arvados/sdk/python to Python search path during crunch job.
Fix namespace bugs.
Fix missing : separator in PYTHONPATH.
Install Python SDK from repository (if present) into virtualenv for
Fix more namespace bugs.
Merge branch 'master' into 2199-crunch-virtualenv
Re-order left nav and make labels consistent with arvados vocabulary.
Merge branch 'master' into 2199-crunch-virtualenv
Remove -h flag from arv examples in docs, now that it means --help.
Use %N instead of %n format and parse node ranges ourselves if slurm <2.3
Update Python SDK install step, add virtualenv dependency
Use rvm-exec to eliminate extra bash process provided by "rvm do". closes #2095
Use fusermount -z to force detach and avoid hanging on exit.
Consume all arguments after arv-mount --exec, and fork subprocess
Document pkg-config dependency for llfuse.
Update Python SDK docs to reflect new llfuse dependencies.
Merge branch 'master' into 2060-edit-tags-in-workbench
Merge branch 'master' into 2060-edit-tags-in-workbench
Fix files=nil -> files=[] in API response for empty collection.
Fix spacing of tag labels.
Turn off CSS animation on loading graphic; make it red instead.
Merge branch '2060-edit-tags-in-workbench'
Move left nav all the way to the top left corner.
Revert "Move left nav all the way to the top left corner."
Fix refresh feature on pipeline_instances/show.
Fix text color of active+hover nav link
Do not try to extract git archive if none was provided by calling process.
Add build.sh step to Python SDK install recipe.
Add build.sh step to Python SDK install recipe (part 2).
Set up virtualenv and install custom SDK only if SDK appears in
Fix usage message dependent on credentials. Remove unused argparse import.
Fix jobs.cancel and add integration test. closes #2258
Use require instead of rake_require. Fixes "Can't find
Document baseurl and change to a useful default.
Add signing and verification code for blob permissions
Add "Log in as user" button in Admin pane on users#show.
Read configuration from config.defaults.yml and config.yml.
Add "rake config:check" task. refs #2076
Add arv-normalize to python SDK package.
Obey ARVADOS_API_HOST_INSECURE
Add arvados-cli dependency
Fix crash when job script is nil.
Fix check for crunch-job missing from path, and run it as arv-crunch-job.
Merge branch 'master' into 1844-blob-signature
Merge branch '1844-blob-signature'
Fix CollectionReader treating any collection with additional +hints as
Support where=[[attr,operator,operand],...] parameter
Move where=[] to filters=[]
Advertise filters param in discovery doc.
Add ArvadosModel.filter() to support filters API
Avoid extra API lookup to get an object that is already loaded.
Use action_name instead of URI path as the default breadcrumb page name.
Add users/activity page.
Fix up start/end dates and display them in a more friendly way.
Fix incorrect API endpoints in api/schema/
Merge branch 'master' into 2076-check-config-at-boot
Rename config[.defaults].yml to application[.default].yml
Merge branch 'master' into 2189-log-in-as-user
Merge branch '2076-check-config-at-boot'
Merge branch '2189-log-in-as-user'
Rename load_config to zz_load_config so initializers/secret_token.rb
Add missing config variable to test.rb.example. Thanks, #2076!
Default to using a random secret_token in test mode.
Rename preload_all_models to zz_* so it runs after load_config.
Add arvados-cli dependency to Gemfile.
Add python-pip to Python SDK install dependencies
Fix namespace bug in write_directory_tree(max_manifest_depth=0).
Add instructions for installing with new versions of pip.
Remove PYTHONPATH hack.
Add api server package dependencies and update config instructions.
Fix left nav order in Install section.
Update comment to refer to correct default-config file.
Fix route method on user_agreements/sign.
Fix user_agreements/sign* routing again.
Report error message and propagate exit code if subprocess.call() fails.
Fix giving up on a pipeline prematurely when a component is waiting
Bump versions, change optional=false to required=true, set dataclass
Update Gemfile to use latest arv-run-pipeline
Remove some stumbling blocks in default/example configs
Fix configured uuid_prefix and force_ssl getting overwritten by application.rb
Add sqlite3 to Gemfile so database.yml.sample works unmodified
Fix crash in render_error(not_an_exception), and use raise() rather
Fix order of steps in install doc, add git_repositories_dir
Merge branch 'master' into 2257-inequality-conditions
Make integer attributes searchable, add test cases.
Merge branch '2257-inequality-conditions' into 2290-user-activity
Add "filters" param to discovery document, add deprecation note to
Merge branch '2257-inequality-conditions'
Tweak table styles and add Total row
Add link from users#index to users#activity. Not as good as showing it
Merge branch 'master' into 2290-user-activity
Use --no-clear-tmp flag to decide whether to remove of old temporary
Use flock() to prevent sharing crunch-tmp with another job in local dev mode.
Use the supported discovery document instead of the old schema#show shim.
Remove old schema#show method used by Workbench before
Merge branch 'master' into 2316-no-clear-tmp-flag
Merge branch '2316-no-clear-tmp-flag'
Merge branch '1750-use-supported-discovery-doc'
Fix up README, convert to textile.
Fix textile formatting.
Accept ...&filters=&... in query string as "none" instead of "unparseable JSON".
Add tests for limit and offset parameters. refs #2075 refs #2209
Add output_is_persistent attribute to job model.
Add output_is_persistent flags to example pipeline template.
Propagate output_is_persistent flag from pipeline to job.
Make sure persistence links get added for persistent components, even
Indicate which outputs are marked "persistent" on pipeline provenance graphs.
Avoid reusing per-task work directories.
Merge branch '2290-user-activity'
Merge branch '2209-test-limit-offset'
Fix reversed input/output in dependency explanation.
Add/explain output_is_persistent flag in example pipelines.
Mark output as persistent at end of job, if Job record says to do so.
Merge branch 'master' into 1932-job-output-persistent
Merge branch '1750-remove-old-schema-doc'
Update controller action name in functional test. refs #1750
Merge branch '2412-do-not-reuse-task-work-dirs'
Add Perl SDK page.
Add command line SDK page.
Fix broken links and duplicate titles.
Add Ruby SDK page.
Add list of in-progress/planned SDKs.
Ignore "cache directory does not exist" error when clearing cache at
Fix migration crash when script_version is nil.
Merge branch 'master' into 1932-job-output-persistent
Fix some whitespace complaints.
Move "empty blob" magic number into a Collection class method, remove
Clarify c_already_finished comments.
Add dependency on latest arv-run-pipeline-instance.
Fetch --tags, otherwise we miss out on commits that are tagged but are
Merge branch '1932-job-output-persistent'
Bump arvados-cli version to use output_is_persistent feature. refs #1932
Add repository=arvados now required by jobs.create.
Move keep mount point outside TASK_WORK directory to prevent conflict
Test items_available = items returned when limit is large.
Test no duplicate UUIDs are returned by collections.list.
Count distinct records for items_available, not rows generated by permission join.
Increase max #users shown on users/activity page.
Support group-wide administrator privileges.
Document group-level administrator semantics.
Add a system_group, and ensure it has can_manage permission on every
Add migration to create system group in existing installations.
Fix down-migration: delete the system_group and automatically
Merge branch '2498-group-admin'
Load ArvadosApiClient after configuration.
Fix searching on tables with no "name" column. Take care not to use
Merge branch '2447-fix-workbench-search'
Really check for duplicate items returned, instead of just calling a
Merge branch 'master' into 2209-correct-items-available
Fix counting results in test case (count != size for an ActiveRelation).
Merge branch '2209-correct-items-available'
Fix copy/paste error.
Remove Ruby 2.1.0 dependency by using Hash[x] instead of x.to_h.
Document capybara-webkit qt dependencies, and add phantomjs recipe.
Use bootstrap/theme style defaults for user setup modal and form.
Document capybara-webkit qt dependencies, and add phantomjs recipe.
Use bootstrap/theme style defaults for user setup modal and form.
Change password placeholder to something db:load_config can parse.
Fix permission error when adding system_group->new_user permission link.
Show editable pipeline instance name (instead of template name) as
Stash and restore current tab selections when replacing body content.
Fix tests: make sure system_user exists before making "pre-test users" list
Trim newline from output hash emerging from `arv keep put`.
Add "create collection" integration test
Get logs and archives directly from the given git repo. Skip git-clone
Update README for selenium+headless: install firefox+xvfb
Merge branch '2243-fix-pipeline-instance-heading'
Add comments to explain different ways we choose $repo.
Add/fix some shell quoting.
Merge branch '2338-maintain-active-tab'
Merge branch '2246-fix-git-clone-race'
Refactor pipeline_template -> pipeline_instance process.
Ignore jobs with no repository specified, instead of crashing.
Do not reuse cancelled jobs. Do not consider nil outputs from
Make arrays non-editable on pipeline instance page.
Send HTTP 500 if we end up in render_error without a more specific status.
Add token that is valid except that it points to a missing user.
Move test-api-server framework so it works in functional and unit tests too.
Add tests to expose some token handling bugs.
Fix exception when valid token points to missing user
Merge branch '2388-bogus-token-error-page'
Send HTTP 500 if we end up in render_error without a more specific status.
Add token that is valid except that it points to a missing user.
Move test-api-server framework so it works in functional and unit tests too.
Add tests to expose some token handling bugs.
Fix exception when valid token points to missing user
Add tests, enable environments/test.rb to prevent CSRF checks in tests
Update local copy of updated_at if the server provides it.
Ensure created/modified/updated_at are correct, add tests.
Merge branch 'master' into 2596-refactor-pipeline-create
Fix, and tests for, PipelineInstance#bootstrap_components
Remove bootstrap_components logic from Workbench, now that it happens
Merge branch 'master' into 2596-refactor-pipeline-create
Merge back-to-back condition blocks with the same condition.
Fix unwarranted use of instance variable.
Fix errant use of symbol vs. string
Merge branch '2596-refactor-pipeline-create'
Fix typo
Add instructions for Ruby and bundler. Add missing dependencies.
Add missing </span> tag.
Fix up SSH capitalization/typeface
Fix up MD5 and Git capitalization/typeface and some Git vocabulary
Fix missing --format=uuid, fix obsolete version=repo:commit example.
Add Postgres setup instructions. Fix missing "bundle exec".
Clean up .gitignore files
Fix here-doc usage.
Extricate flags controlling jobs.create behavior from the Job resource itself.
Accept minimum_script_version, excluded_script_versions, and
Use "filters" instead of "where" in examples and docs. Remove
Remove spurious "show" methods from API docs.
Remove spurious "destroy" methods from API docs.
Clean up API schema and method pages.
Fix up job create/cancel descriptions.
Describe logins and get_all_logins methods.
Remove updated_at from API responses.
Add dependency info to Workbench install page. Fix "is_trusted" recipe.
Fix configuration behavior in Ruby SDK.
Hide refresh button on "show pipeline instance" page.
Permit a job to update its output and log attributes even if those
Fix failing test in 8d6efcbfb559e377a8f21873b736390035bc44ed.
Add group_class attribute.
Expose generate_uuid class method, e.g., for making test fixtures.
Remove _kind columns from schema.
Set :null=>false explicitly on timestamp columns.
Add some test fixtures that are owned by folders.
Improve permission link examples.
Fill in "I'm in a folder" fixtures
Add owned_items action for groups and users.
Support limit and offset params in owned_items.
Rename include_managed to include_indirect.
Add docs for owned_items.
Add unit test for User#owned_items.
Build response objects based on item type rather than list type.
Add two missing sudo.
Add explanation comments to assertions.
Add tests, rename include_indirect to _linked, improve wording in
Support filters=[["attr","=",nil]]
Rename jresponse test helper to json_response.
Merge branch 'master' of git.curoverse.com:arvados into 2640-folder-api
Merge branch 'master' into 2505-update-docs
Improve examples, explanations, typography, tests
Remove excess controllers and routes. Add tests to detect accidental future additions.
Check sanity when applying filters, and provide useful error messages.
Fix admin user setup instructions, add some missing "bundle exec",
Add detail about what script_version should look like.
Merge branch '2409-remove-unused-controllers'
Fix & test edge cases (no script_parameters), fix pipeline instance fixture UUIDs
Add persistent/cache switch to collections index page.
Move persistent/cache toggle into a partial.
Add persistent/cache toggle to collections#show page.
Fix s/about/~/ substitution on dashboard.
Remove unused tutorial_group and tutorial_links queries.
Add persistent/cache toggle to collections table on dashboard.
Untabify.
Remove "Admin Guide" link from left nav.
Merge branch '2640-folder-api'
Add UUIDs in TR tags so automated tests can find them.
Merge branch '2505-update-docs'
Merge branch 'master' into 1969-persistent-switch
Add tests for persistent/cache switch.
Increase contrast between active/inactive sides of toggle switch.
Merge branch 'master' into 1969-persistent-switch
Fix crash when tags exist for items offered in selection widget.
Use name links instead of permission links to include objects in groups.
Expose names for owned_items as list_response.name_for() in Workbench
Convert joins to subqueries to fix duplicates in owned_items
Fix patch_paging_vars args.
Render group with folder view
Show folder contents with editable names.
Add sb-admin layout.
Add folders page, backed by groups.
Skip UserAgreement in owned_items. They are just collections again.
Merge branch '2640-folder-api' into 1970-folder-view
Fix error reporting, catch "=foo" in config file, add dev doc.
Merge branch 'master' into 1969-persistent-switch
Replace radio button group with a single click-to-toggle button.
Merge branch '1969-persistent-switch'
Merge branch '0000-ruby-client-config'
Rename owned_items to contents, remove user method, update docs.
Merge branch '2640-folder-api' into 1970-folder-view
Rename owned_items to contents.
Make "create folder" button work
Fix route name in nodes.create, add basic functional test.
Move folders into a separate controller. Add description-editing.
Fix no-side-nav mode.
Fix "create folder"
Merge branch 'master' into 2640-folder-api
Merge branch '2640-folder-api' into 1970-folder-view
Use separator instead of drop-down for less-used nav links.
Fix tests to work with new css/layout.
Move up more useful navs, make activity text smaller.
Less side-nav padding.
Fix "show all activity".
Add integration test for folders.
Fix up table columns, use explicit Open button instead of linking uuid
Render cards with real content from first few items.
Focus/hover side navs with white instead of arbitrary color.
Synchronize "show more" button style, turn off cards for now
Do not return Links with group contents
Update tests for new layout.
Add "Shared with" section to Sharing/Permissions panel.
Add "show object" button partials.
Put back missing parentheses.
Add test cases that produce predictable signatures with fixed expiry times.
Add partial unique index
Merge branch '2328-signatures-for-integration-testing'
Merge branch 'master' into 2640-folder-api
Fix log permission case broken in merge.
Use partial unique index for name links.
Merge branch 'master' into 2640-folder-api
Fix wrong infixes in fixture uuids.
Add unique and valid name constraints.
Merge branch '2640-folder-api' into 1970-folder-view
Name links do not need to be owned by tail_uuid.
Add "Copy selections to current folder" button to selection drop-down.
Merge branch '2640-folder-api' into 1970-folder-view
Add folder list to left nav.
Use class_for_display more.
Fix crash if item has no name (backported from 1970-folder-view)
Fix name_for method name.
Take into account owner_uuid and is_admin when exporting repository permissions.
Add shadow to left nav.
Unify button styles.
Add writable_by to Group API response.
Add "remove item from folder" buttons
Merge branch '2223-repo-owner-admin-perms'
Return empty list if no "links" supplied in api response. Add tests.
Return all links if link_class is not provided. Add docs.
Improve code formatting
Merge branch '2640-folder-api' into 1970-folder-view
Generate test coverage reports with SimpleCov.
Prevent ownership cycles.
Fix crashing "grant access to my own object" permission check.
Merge branch 'master' into 2640-folder-api
Do not try to record object_owner_uuid during earlier migrations.
Update to Rails 4
Use one client per thread, via a factory method, instead of a global.
Add client proxy hack so existing themes continue to work.
Skip simplecov if gem fails to load or NO_COVERAGE_TEST env is set.
Omit locally configurable files and testing code from coverage report
Merge branch '2640-folder-api' into 1970-folder-view
Update name in breadcrumb after editing elsewhere on page.
Name new folders "New folder", fix crash if folder name is nil
Implement "remove from folder"
Implement simple client-side folder contents search.
Edit objects after creating them with x-editable.
Behave nicer if supplied owner_uuid is not a valid uuid at all.
Fix tests, make model.editable? work for new records.
Fix ajax->refresh race condition in test case.
Add simplecov-rcov.
Add test for links_for + multiple link_classes.
Comment mysterious testing code.
Raise PermissionDeniedError more consistently
Merge branch '2640-folder-api' into 1970-folder-view
Raise PermissionDeniedError even more consistently
Merge branch 'master' into 2640-folder-api
Merge branch '2640-folder-api'
Fix broken test case.
Move contents action to GroupsController.
Merge branch '2640-folder-api' into 1970-folder-view
Remove confusing return value.
Change icon for virtual machines nav.
Use send(symbol) instead of send(string).
Ensure edited description really gets saved.
Remove duplicate ERROR_ACTIONS constant from merge.
Replace unfailable assert() with a comment explaining what it means
Add find?() method so "assert find?(...), msg" can be used in test cases.
Merge branch 'master' into 1970-folder-view
Merge branch '1970-folder-view'
Remove stray debug puts.
Remove Gemfile.lock from repo
Fix copy/paste error in gem build instructions.
Synchronize repository and crunch setup instructions with current behavior.
Merge branch '2733-coverage-report'
Fix dead links
Merge branch 'master' into 2744-update-dev-docs
Fix Perl and Python SDK instructions on crunch install page
Add wait_for_ajax after using x-editable in tests.
Clean up install docs
Fix prompt paths.
Fix up workbench and apiserver install docs.
Fix crash creating new user during initial login. Add test.
Refuse to run rake tasks without "bundle exec".
Document "filters" feature. Add missing "!=" and "not in" operators.
Clarify log methods page: say "log entry" instead of "log".
Improve tests: multiple values for "not in", check =, != results more carefully.
Merge branch '2744-update-dev-docs'
Trim newline from logfile hash emerging from `arv keep put`.
Fix pipeline instance state tests.
Fix python SDK test suite issues.
Fix Python SDK test suite issues.
2762: Protect owner_uuid referential integrity when changing uuids and
2762: Test assigning random non-existent owner_uuids to new objects.
2762: When deleting an object, delete permissions and fail if other
2762: Rename assign_uuid to has_uuid to reflect expanded scope.
2756: Use site-default x-editable mode for pipeline instance name.
Merge branch 'master' into 2762-owner-uuid-integrity
2808: Fix paging on jobs, pipeline_instances, api_client_authorizations.
2812: Fix "not in" filter so it can retrieve rows with null values.
2754: Hide unhelpful "Add a new job" button.
2673: Move auth to headers, avoid combining request body with GET method.
Merge branch '2673-no-request-body-in-get'
Add self-signed.{key,pem} to .gitignore files.
2638: Fix css syntax.
2673: Update bundle to ensure we get commit:7ff8586 at runtime.
2821: Add "delete" button to pipeline instances page. closes #2821
2673: Move auth to headers, avoid combining request body with GET method.
2673: Update bundle to ensure we get commit:67f1781 at runtime.
Specify required ruby version (>= 2.1.0) in gems.
2673: Use _method=GET hack to avoid combining request body with GET
Fix job state cleanup in crunch-job failure case.
Really write log messages to logfile. refs #2221 commit:ced5e21
2762: Move destroy_permission_links method to HasUuid.
2762: Finish enigmatic comment.
2762: Fix wrong class used in test case.
2762: Merge branch 'master' into 2762-owner-uuid-integrity
Merge branch '2762-owner-uuid-integrity'
2756: Fix ajax refresh on pipeline instance page.
2756: Fix "duplicate tab" behavior in Chrome.
2809: Merge branch 'master' refs #2809
2809: Fix load_config so it loads in time to set passenger configs.
2809: Turn off secret_key_base by default, instead of requiring it in config.
2809: Fix whitespace.
2809: Add note about harmless warning when installing the themes_for_rails gem.
2809: Remove debug printfs, add comment explaining why so much "permit!"
2809: Add wait_for_ajax after submitting forms, to avoid race condition.
2809: Upgrade to Rails 4.1
2809: Rearrange "run API server during tests" to suit Minitest 5.
2809: Remove some unnecessary sleep(0.1).
2809: Remove javascript and coffeescript boilerplate.
2809: Move coffee-rails out of :assets section to make tests thread-safe.
2809: Merge branch 'master' into 2809-workbench-rails4 refs #2809
2809: Merge branch '2809-workbench-rails4' refs #2809
2755: Defer to CollectionReader to get manifests instead of going
2809: Fix disappearing ArvadosApiClient in development mode. refs #2809
Remove debug log message. refs #1969
2809: Fix arvados_api_client usage in FileStreamer. refs #2809
Prevent admin from creating an object with empty string as UUID.
2352: Reset column information between adding a new serialized
Fix crash when saving api_client_authorization during migration.
2754: Add easy "Inputs" tab for providing required/described inputs for a pipeline instance.
2754: Allow editing the same parameter in two different places on the page.
2754: Fix up instruction text and formatting.
2754: Add integration test, fix selection bugs.
Remove obsolete secret_token.rb.
2760: Show folder hierarchy on regular page, not in left nav
2760: Add "move to a different folder" button to folders#show.
2760: Refresh permission cache after changing group uuid or owner_uuid.
2760: Add test case.
2760: Make editable? follow permissions obtained via groups/folders.
Fix crash in jobs.queue API. Add test case.
2754: Allow template to specify form label, fix up wording and formatting.
2754: Fix css selectors in test case.
2760: Use lambdas instead of defining methods with misleading scope.
2754: Make default parameter label more explicit.
2754: Merge branch '2754-easy-run-pipeline' refs #2754
2760: Use "My Folders" to indicate the root of my folder hierarchy.
2760: Merge branch 'master' into 2760-folder-hierarchy
2760: Disambiguate "hover" from "selected" style. Do not highlight
2760: Merge branch 'master' into 2760-folder-hierarchy
2760: Merge branch 'master' into 2760-folder-hierarchy
2760: Merge branch '2760-folder-hierarchy' refs #2760
2760: Exclude system objects like VMs and repos from folder behavior.
2754: Add pipeline template description attribute.
2760: Merge branch '2760-not-all-objects-in-folders' refs #2760
2862: Use ?format=js so browser does not think js and html views are
2892: Fix user agreement routing. closes #2892
2754: Add migration missing from commit:fe84c84827af6092784408629e4cde0663fe6af8
2893: Prevent symbol keys/values in serialized attributes.
2893: Add test for un-symbolizing existing db content.
2893: Add more tests for bogus serialized hash values.
2754: Merge branch '2754-pipeline-template-description' refs #2754
2754: Change "create pipeline" label to "run", to match others in same workflow.
2762: Do not try to auto-destroy referring links when destroying a model with no uuid. refs #2762
2931: Remove {dependent: :destroy} in ArvadosModel. HasUuid does that now. closes #2931
2755: Let the API server decide the correct collection UUID format.
2893: Add test cases. Stringify all serialized attrs coming from DB,
2893: Merge branch 'master' into 2893-no-symbols-in-db
2893: Merge branch '2893-no-symbols-in-db' closes #2893
2893: Fix accepting JSON-encoded components_summary in API calls. refs #2893
2939: Allow caller to pass additional parameters to create API.
2872: Remove name column from collections table.
2872: Start doing everything from folder perspective.
2872: Use data chooser for running pipelines. Many presentation fixes.
2872: Rearrange standard views to deemphasize "advanced" usage, add infinite scroll
2872: Make editable work for names in folder, choose data from current folder, tweak css.
2872: Fix "remove from folder"
2872: Add "delete folder" button.
2872: Rearrange folder index. Show collection tags and files in folder view.
2872: Add infinite scroll to chooser modal.
2872: Disable chooser OK button until selection is made, update run-pipeline wording.
2872: Add useful pipeline instance content summary.
2872: Fix css on collection tab line buttons.
2872: Show collection details in chooser.
2872: Add recent jobs and pipelines to folder index, use as root url.
2872: Fix some tests and bugs
2872: Fix bugs
2872: Add job summary partial.
2872: Fix up tests
2872: Ensure name link tail_uuid == owner_uuid. refs #2872
2872: Change default order of folder contents. refs #2872
2872: Fix attribute_editable? to play nicer with group ownership. refs #2872
2872: Add free text search in chooser.
2872: Add "clone and edit" button to pipeline instance page.
2872: Make infinite scroll work with regular window scrollbars too.
2872: Add selection checkboxes to folder view. Compare pipeline instances and remove multiple items.
2872: Ensure friendly_name is always a string.
2872: Filter by tail instead of owner in case api server rule is not in yet.
2872: Treat HEAD same as GET in case we are behind a proxy.
2872: Fix total_bytes returning nil.
2872: Fix crash on missing object.
2872: Fix empty name placeholders.
2872: Check "optional" flag too when looking for required inputs.
2872: Fix up log message display on pipeline instance page.
2872: Fix syntax
2872: Do not try to add name links for objects that have their own names
2872: Fix links to collections from folder view.
2872: Fix breadcrumbs on pipeline instance compare page.
2872: Make Show button say Show again. Fix links on collections index.
2872: Merge branch 'master' into 2872-folder-nav
2872: Ensure user agreement can be viewed before being accepted.
2872: Make "inactive" and "not logged in" behavior work on all pages, not just dashboard.
2872: Fix up login behavior.
2872: Tweak wording on pipeline inputs page.
2872: Enable OK/Cancel buttons on editable popup. Lose "close" button.
2872: Use collection name, not hash, for breadcrumbs/title/activity.
2872: On front page, only show recent pipelines and jobs that are in folders.
2872: Fix auto-disable/enable "Run" button on pipeline page.
2872: Bring back Graph pane on pipeline instance show page.
2872: Fix up pipeline instance table, show "resume" button if paused.
2872: Fix string+nil bug.
2872: Fix provenance graph absolute position.
2872: Hide unnecessary scroll bar
2872: Rename folders to projects
2872: Rearrange top nav with breadcrumbs/projects bar.
2872: Fix notifications menu colors.
2872: Hide projects nav bar when not logged in or inactive.
2872: Top nav gradient.
2872: Fix test for new nav.
2755: Add api server config to enable mandatory Keep signatures.
2872: Fix bookmark bar causing spurious window width.
2872: Merge branch 'master' into 2872-folder-nav
2872: Fix tests broken in merge.
2872: Show projectless jobs/pipelines in "recent" list too.
2872: Muster all the "move" and "add" buttons up to the right side of the tabs.
2872: Fix "remove" icon and pencil icon placement.
2872: Add "move to project" button, fix "copy to project" behavior
2872: Fix selector in test case.
2755: Fix unfinished comment.
2755: Preserve blob signatures in manifest_text passed to collections.create.
2755: Fix handling of stripped/orig manifest text.
2755: Merge branch '2755-require-keep-permission' refs #2755
2872: Fix bugs, tweak formatting
2872: Move project_tree code into helper_methods, fix up tests.
2872: Fix test, to match fixed fixture.
2961: Fix crash on pipeline_instances/compare.
2872: Fix provenance graph double-smart-scroll.
2872: Redirect to parent project after deleting a subproject.
2753: Fix param name (filter -> filters).
2872: Merge branch 'master' into 2872-folder-nav
2872: Add data binding for project names in dropdown.
2872: Merge branch '2872-folder-nav' closes #2872
Fix infinite scroll in chooser. refs #2872
Handle UrlGenerationError as an application exception rather than a
Fix crash on unexpected case of a collection whose owner_uuid is not root.
Remove spurious puts. refs #2884
Fix off-by-one-month in timestamp conversion. refs #2883
3099: Fix up loading spinners.
3099: Use appropriate tr>td markup if infinite scroll container is a table.
Merge branch '3099-spinner-assets' closes #3099
3174: Heed "common" section of config yaml the same way apiserver does.
3174: Round up, not down, when reporting file sizes in KB.
Merge branch '3174-arv-ls-size-fencepost' closes #3174
3185: Fix job validation failure with no errors given.
3220: Fix HTTP status codes.
3220: Adjust disk corruption behavior: If both corrupt and intact
Fix broken links to SSH key help page. no issue #
3113: Remove qsequence assignment shim. Clarify "do not load tasks
3220: Fix whitespace
3220: Merge branch '3220-http-status-codes' closes #3220
3214: "New project" link in top nav dropdown create a new project at top level, instead of inside the current project.
3214: Ensure current_user has write permission on the new owner when setting owner_uuid.
3214: Fix unclosed HTML tag.
3170: Root out evil runtime-generated javascript. Simplify tab loading.
3170: On pipeline instance page, refresh the tab panes rather than the entire page.
3214: Split independent unit tests into separate test cases.
3214: Readability: rearrange conditionals, and say "old" instead of "existing"
3214: Merge branch 'master' into 3214-permission-to-use-owner-uuid
Merge branch '3214-permission-to-use-owner-uuid'
3235: Top nav "search" button opens a collection chooser.
3170: Add comments.
3170: Merge branch 'master' into 3170-refresh-tab-pane
Merge branch '3170-refresh-tab-pane'
3235: Hide projects dropdown unless caller asked for it.
3170: Fix stale use of tab_pane_valid_state. refs #3170
3170: Use find().all() instead of page.all() to accommodate ajax content. refs #3170, #2044
Remove extra spinner. closes #3301
3235: Fix project dropdown, use it when choosing collections.
3235: Merge branch 'master' into 3235-top-nav-site-search
3235: Fix errant merge.
3182: Fix races by using find() instead of {sleep; page.has_text?}.
Merge branch '3182-workbench-test-race'
3138: Split body into a separate layout for wiselinks.
Fix extra row left behind after deleting a pipeline instance. closes #2929
3235: Split GroupsController#contents logic out to load_searchable_objects.
3349: Consolidate comments in application.default.yml
3349: Update to ruby 2.1.2
3349: Merge branch 'master' into 3349-install-docs
3349: Remove duplicate keys in default yml.
3349: Add default blob_signing_key for test environment.
3349: Do not override working "test" defaults in example config.
3235: Fix SQL query.
3235: Add test case.
Do not remove /dev/stdout on interrupt signal. No issue #
3286: Split Home/Projects thing into Home button and Projects dropdown.
Merge branch '3286-home-button' closes #3286
3351: Set PUT request timeout to 60 seconds.
3351: Do not use resume cache at all if --no-resume flag is given.
3351: Improve "already uploading" message (and send to stderr, not stdout).
3351: Retry failed threads (servers) if replication is too low after one pass.
2800: Allow api() caller to specify api host and token.
3351: Catch SSLError and show warning instead of stack trace.
3351: Clean up some excess indentation.
3351: Add comment about SSL timeouts.
3351: Fix CollectionReader usage in arv-ls.
3235: Show multiple object types in top-nav Search modal.
3235: Skip data-remote links in smoke test. They might not know format=html.
3036: Assign collection uuid without complaint if not provided by client.
3036: Add TODOs for mutable collection uuids
3235: Infinite scroll error handling: Show error message and "Retry" button
3235: Fix infinite scroll in Search chooser.
3235: Hide section header rows for empty sections. closes #3377
3235: Make filter-by-project play nice with infinite scroll.
3235: Fix multiple-select.
3235: Fix "no filter / all projects" option.
3235: Show project filter by default (i.e., unless by_project=false).
3235: Focus search box when chooser modal appears.
3235: Bind Esc key to close modals.
3235: Merge branch 'master' into 3235-top-nav-site-search
3235: Remove errant action exemption.
3235: Split GroupsController#contents logic out to load_searchable_objects.
3235: Fix SQL query.
3235: Add test case.
Merge branch '3235-all-contents' refs #3235
3235: Merge branch 'master' into 3235-top-nav-site-search
3235: Move filter-by-project to left side.
3235: Fix infinite scroll broken by merge.
Do not return non-project groups in groups#contents. refs #3235
3235: Show selected project in project-filter dropdown title.
3235: Fix hover highlighting on projects dropdowns.
Fix TypeError: Got an unexpected keyword argument "uuid". refs #3235
Do not show non-project groups in groups directory. no issue #
3235: Replace link with search widget in top nav.
Merge branch '3235-top-nav-site-search' refs #3235
3235: Fix attribute keys. refs #3235
3351: Propagate caller-provided timeout from KeepClient to KeepWriterThread.
Merge branch '3351-keep-timeout' closes #3351
3235: Merge branch 'master' into 3235-top-nav-site-search
3235: Do not bring up search dialog until enter/click in top nav
3235: Don't blow up when calling 404 from ActionsController.
3235: Fix missed filter update after paste event.
Report validation error messages in server log. refs #3415
3235: Merge branch 'master' into 3235-top-nav-site-search
3235: Update results as soon as search modal opens.
3235: Make collections show up in search results.
3235: Show names of named collections instead of their uuids, but support searching by uuid.
3235: Accept group_class="folder" as a project.
3235: Enable "rename folder to project" migration.
3235: Fix empty collection migration: ensure anonymous_group exists in
3349: Merge branch 'master' into 3349-install-docs
3349: More example/default config cleanup.
3425: When reading from arv-get, repeat sysread until error or eof.
Merge branch '3349-install-docs' closes #3349
3425: Smaller buffer.
Merge branch '3425-truncated-output' closes #3425
3261: Fix alternate SSO endpoint configuration.
3261: Fix condition for showing SSH usage message.
3261: Add Keep blob_signing_key configuration.
3261: Propagate configured RAILS_ENV to Workbench build tasks.
3261: Fix wrong test: wanted file size == 0, not string length == 0.
3261: Strip .in from list of generated files.
3261: Improve status reporting
3261: Fix permissions on run-keep script
Merge remote-tracking branch 'origin/3261-keep-docker-config' into 3261-keep-docker-config
3261: Fix wrong test: wanted file size == 0, not file size != 0.
3261: Fix wrong variable.
3261: Fix typo.
3261: Restore (and document) ability to use docker sso container.
3261: Turn on --enforce-permissions in run-keep.
3261: Reduce dependency bloat: process templates one dir at a time.
3235: Merge branch 'master' into 3235-top-nav-site-search
3235: Fix UrlGenerationError on 404 page.
3235: Show uuid without "unnamed -" prefix for nameless objects.
3235: Fix test
Merge branch '3235-top-nav-site-search' refs #3235
Fix display order of recent jobs and pipelines. closes #3148
3444: Remove support for obsolete group_class=folder
3444: Update documentation referring to obsolete group_class=folder
Merge branch '3444-no-folders' closes #3444
Merge branch '3463-make-sharing-dialog-with-users-more-useful' closes #3463
3499: groups/{user_uuid}/contents returns home project contents. closes #3514
3499: Update breadcrumbs. closes #3513
3499: Show home project at /projects/{user_uuid}. closes #3515
3499: Show Home at root of projects tree. closes #3517
3499: Merge branch 'master' into 3499-home-project
3499: arv-put collections in home project if no --project-uuid specified.
3525: Users can manage permissions on their own repositories
3036: Remove unnecessary code path.
3036: Merge branch 'master' into 3036-mutable-collections
Merge branch '3036-mutable-collections' refs #3504
3499: Improve Python SDK's ability to read apiserver's config files.
3499: Fix Python SDK tests to accept signed blob locators.
3499: Reconcile Python tests to new arv-put "home project" behavior.
3499: Update arv-put usage message to reflect new behavior.
3499: Merge branch 'master' into 3499-home-project
3499: Do not show "Move" and "Delete" buttons on Home project.
3499: Show "Home" in "My projects" panel on dashboard.
3499: Do not show "Sharing" tab on Home project.
3499: Do not show Home in breadcrumbs when viewing dashboard page.
3499: Remove "not in any project" warning.
Merge branch '3525-manage-own-repo' refs #3525
Merge branch '3499-home-project' closes #3499
Update tests and user setup methods. refs #3525
Hide preview pane on narrow screens. closes #3496
Add "wait_for_ajax" to avoid race condition. No issue #
2769: Add -never-delete command line flag.
Fix excessive API calls in collection chooser. refs #3489
Merge branch '2769-disable-delete-flag' closes #2769
3489: Indicate initially selected project in chooser. closes #3544
Fix repositories.get_all_permissions, add tests. closes #3546
3551: Fix source tree layout.
3551: Use keep{store,proxy} from the working dir to build keep containers
3551: Make keepproxy tests pass when permission signatures are enabled
3551: gitignore python/sdk/tests/tmp
3296: Remove session cache of user record to prevent synchronization problems.
3212: Support arv-run-pipeline-instance --no-reuse --submit
3531: Refactor project tab infinite-scroll. Sort jobs and pipelines
3531: Fix horizontal alignment of selectable and non-selectable things.
3531: Use dropdown caret instead of down arrow on selection dropdown.
3531: Remove debug limit.
3531: Add tests for sort order on "jobs and pipelines" tab.
3531: Hook infinite-scroll to resize events.
3531: Look up relevant portion of uuid->{user,group} map when loading projects tab.
3531: Fix name-editing behavior: preserve name_links when merging results.
3531: Fix test to accommodate more ajax.
3178: Add missing jobs.create and users.setup parameters to discovery document.
3178: Use different discovery doc cache files for different API servers.
3178: Discover required parameters for regular CRUD methods, too.
Merge branch '3212-pipeline-no-reuse' closes #3212
3570: Do not fail the job when crunch-job loses a locking race during startup.
3489: Remove debug comment.
3489: Degrade nicely if initial filter refers to an unreadable project.
3489: Add test for correct dropdown text.
3531: Do not assume order is always descending.
3531: Dry up tab pane rendering. Fix copy-and-paste error on tab_name.
3531: Add large limit parameter to ensure we get all of the user/group objects
3531: Make test case less confusing.
Merge branch '3570-crunch-race' closes #3570
3531: Move the tab_line_buttons sections out of the tab pane partials
Merge branch '3489-indicate-initial-project-filter' closes #3489
3570: Use exit code 75 to mean "temporary locking failure". Add comments.
3178: Merge bogus _setup_requires_parameters into existing class method.
Update PGP download link. No issue #
Update more PGP web links. No issue #
Merge branch '3570-crunch-race' closes #3570
3572: Set state to RunningOnServer if --submit given. closes #3572
Merge branch '3178-discoverable-parameters' closes #3178
3531: Fix ordering of combined results, add test. Anchor "desc" regexp.
3531: Pass through tab_pane to reused partials, so element IDs are really unique
Fix "undefined method `uuid' for nil:NilClass". closes #3576
Merge branch '3531-sort-jobs-with-pipelines' closes #3531
Fix repositories.get_all_permissions to report manage, write, and read. refs #3525
3354: Redirect link#show to "show named object".
Update Gemfile to use crunch-job from latest arvados-cli gem. refs #3570
3551: Fix more package paths.
3551: When testing keep clients, write keepstore/keepproxy logs to
3551: When invoking run_test_server.py during tests, pass stderr
3551: Behave correctly when Read() yields some data along with the EOF condition.
3551: When invoking run_test_server.py during tests, pass stderr
3551: Merge branch 'master' into 3551-go-layout
3551: Handle missing GOPATH and multiple-entry GOPATH sensibly.
3551: Set mode of generated files more aggressively (overrule umask).
3551: Remove chmod 755, no longer necessary now that config.rb does that.
3551: Use FileUtils.rm_r so deleting stale generated/{dir} works.
Report unexpected slurm states as "down". No issue #
Increase Capybara timeout to accommodate slow test server. No issue #
3551: Remove superfluous chmod(). Umask is 022 here.
3551: Revert "3551: When testing keep clients, write keepstore/keepproxy logs to"
Update Go tests to new layout, rename keepstore, add missing tests for keepproxy and Go SDK libraries. refs #3551
3551: Dry up pass/fail checks with a checkexit() function.
3551: Merge branch 'master' into 3551-go-layout
Merge branch '3551-go-layout' closes #3551
3551: Use mktemp for GOPATH. Update run-build-packages.sh with new Go recipes.
3551: Clean up temp dir.
3551: Test and install keepstore before Python SDK.
3551: Fix dependency cycle: Install Python SDK, then install+test Go stuff, then test Python SDK.
3551: Install Python and Go bits, force installation of Python test dependencies, *then* run tests.
Restore trailing newline in manifest format sanity check.
gitignore sdk/cli/vendor. refs #3551
Cleanup, add some error checks, add --skip and --only args.
Merge branch '2800-python-global-state' into 2800-pgs
2800: Merge branch 'master' into 2800-pgs
2800: Clean up exceptions, variable names, handling of discoveryServiceUrl vs host args.
3604: Fix theme support for welcome page
3604: Fix stale permission cache by using destroy instead of delete.
3604: Do not check for notifications if user is not even activated.
3604: Remove test for persistent/cache switch on wrong page.
3604: Verify permission cache behavior in unsetup test
3604: Add test for redirect to welcome page
3604: Fix user_agreements behavior, update tests to expect redirects.
3604: Fix up profile and redirect handling.
3604: Fix tests, and restore ability to view user agreement before completing profile.
3604: Fix user deactivation test.
Add missing "install fuse" and "test workbench".
Display elapsed time for each test/install.
Display progress by default when writing to regular files and stderr isatty. No issue #
3604: Remove unused style.
3604: Update comment to reflect lack of built-in default.
Do not throw away permission hints when writing data. refs #3261
Do not try to predict uuid of new collection. refs #3261 refs #3504
3604: Remove useless loop.
3604: Remove useless slice.
2800: Remove unnecessary sha1. Just use the tuple as an array key.
2800: Restore deleted comment which is still relevant.
Merge branch '3604-theme-and-new-user-process' closes #3604
2800: Sort import statements.
Merge branch '2800-pgs' refs #2800
Fix swapped install/test sections for fuse.
3114: Fix href in json response for generic #create action too. See 7485476.
3114: Merge branch 'master' into 3114-copy-to-project
3114: Update test case: searching for own uuid now shows home project.
3114: Follow return_to param, if given, when redirecting users#show to projects#show.
Merge branch '3114-copy-to-project' closes #3114
Install py test deps even if py tests fail. Make docs skippable. No issue #
Fix failure detection for python tests. No issue #
3610: Move live log display into its own tab, avoid clobbering with tab reload.
Merge branch '3610-pipeline-log-refresh' closes #3610
Treat success=true exitcode!=0 as success when collating output,
Add libpcre3-dev dependency. Sort package names in apt-get install command.
Update "standard objects" page to reflect change from keep_disks to
3171: Add tests for desired behavior. Start using FactoryGirl.
3171: Outdent giant "if ... else return self" construct.
3171: Do not follow permission graph through a User, unless permission on the User is can_manage. Restore usual permission model to user lookups. Add tests.
3171: Update tests to conform to new permission behavior.
3171: Add fixtures to make Workbench tests pass under new permission rules.
3171: Skip find_object_by_uuid during :create action.
3171: Do not use select param for database lookup during create/destroy actions.
3153: Add integration tests.
3171: Add administrator UI for toggling user<->role permissions.
3660: Show add/run buttons if project is writable.
Fix disabled Continue button on user agreements page. No issue #
3627: "My Projects" is now called "Home" and is always selectable in choosers.
Fix user agreements page when no signatures needed. Add tests. No issue #
3627: Grey out unselectable projects in chooser.
3627: More consistent icons/headings in projects dropdown and chooser.
3627: Really make Home project selectable in project chooser.
3627: Add hover highlight to Home project on dashboard.
3149: Support "any" and array in filter "column" element.
3149: Support "ilike" operator in record_filters.
3149: Fix operator precedence in SQL statements built from record_filters.
3149: Untabify.
3149: Propagate search string to Workbench server using filters param.
3149: Let Workbench server do all the filtering where infinite-scroll facilities exist.
3149: Fix user-controlled filters overwriting page-controlled filters.
3149: Fix infinite scroll not updating after retry/refresh.
3149: Do not preload results in search dialog. Let infinite scroll do it.
3149: Do not blow up on small devices/windows.
3149: Propagate filters from remote->dialog->infinite. Fix tests.
crunch-dispatch can run --jobs, --pipelines, or both.
Merge branch 'multiple-dispatch' no issue #
3171: Fix wording, update docs.
3171: Merge branch 'master' into 3171-group-membership
Merge branch '3171-group-membership' refs #3171
Do not blow up with pipeline has a plain value in a script_parameter. No issue #
Instead of munging TASK_WORK var, mkdir -p it (and therefore JOB_WORK)
Merge branch '3171-admin-groups' closes #3171
Update Gemfile.lock. refs #3405
3660: Clarify self-contradicting comment.
Merge branch '3660-project-editable' closes #3660
Do not blow up when stdout!=subprocess.PIPE. no issue #
Fix test fixtures breaking Workbench tests. refs #3171
3149: Explain what "any" does as a filter attribute.
Merge branch '3149-filter-any' refs #3149
3149: Clarify comments. Stick with a-b-c data keys where possible;
Do not blow up when tmp directory already exists. No issue #
Do not blow up in install stage when TASK_WORK is not set. No issue #
Remove uuid-guessing code from "combine selected files". refs #3504
Propagate query params to tab panes. No issue #
Move styles out of tab partial. No issue #
Set up arvados env vars when running arv-normalize. No issue #
3149: Merge branch 'master' into 3149-server-side-search
Merge branch '3149-server-side-search' closes #3149
Fix migration and api templates that rely on changes that did not end up happening. refs #3036
Fix test. refs #3149
Merge branch '3627-selectable-projects' closes #3627
Run the task setup script inside the container (if any), instead of
Ensure source_version is a string, even when it consists entirely of decimal digits. refs #3112
Fix label for API endpoint. Re-order fields. refs #3112
3354: Merge branch 'master' into 3354-render-textile
Merge branch '3354-render-textile' closes #3354
Revert arvados gems to versions that actually exist. refs #3342
Run given command even if src tree already installed. No issue #
Pass stdin to child process. closes #3751
Complain and fail if exec() returns. No issue #
3550: Improve startup time by removing excess api client instantiation.
3550: Add --run-jobs-here flag to arv-run-pipeline-instance.
3704: Force params[:filters] into native format so views and helpers can use it.
3637: Operate only on selections within this container, even if clippy has more.
3637: Propagate checkbox selection through chooser modal, remove session hack.
3550: Clarify help message, mention that there are limitations.
3550: Merge branch 'master' into 3550-local-pipeline
3550: Update docs to match new flag name. Add note about job re-use.
3877: Fix memory leak after "log truncated" decision. Clean up code.
3550: Remove extraneous "to" in usage message.
3550: Fix crash when --template loaded from local JSON file has no uuid.
Merge branch '3877-log-memory-leak' closes #3877
3550: Add missing .andand
3550: Add missing .andand
3550: Fix running local job repeatedly on subsequent loop iterations.
3550: Fix race conditions: avoid setting running/success/finished_at
3550: Merge branch 'master' into 3550-local-pipeline
Merge branch '3550-local-pipeline' closes #3550
Update Gemfiles to use latest arvados gem. refs #3550
3894: Accept WORKSPACE=foo on command line. Copy config from CONFIGSRC
3894: Do not create and abandon new temp dirs if caller has already
3901: Add missing error checks while copying git objects.
Merge branch '3901-cannot-find-commit' refs #3901
3894: Fix backward empty/nonempty logic.
3894: Use mktemp instead of $WORKSPACE/tmpgit for test repo
3894: Scrub PYTHONPATH
3894: Use gem install --user-install to avoid sudo/rvm requirement
3894: Fix sequence: do all installs, then all tests.
3894: gem version numbers reflect latest commit in the subdirectory
Use new --run-pipeline-here instead of --run-here flag, which no longer does what crunch-dispatch wants. refs #3550
Fix Gemfile.lock to use a real gem, not a dev build. refs #3550
3894: Fix up runtime dependency specs to make bundle comfortable.
3894: Merge branch 'master' into 3894-gem-version
3894: Add --help message.
3894: Use $GEMHOME tmpdir instead of user's $HOME for userspace gem installs.
3894: Use git log --first-parent to get nearest commit/merge on
3894: Remove outdated README.
3894: Move report_outcomes to a function.
3894: Remove bundle install stage from sdk/cli.
3894: Use readable timestamp (yyyymmddhhmmss) for python package
3894: Merge branch 'master' into 3894-gem-version
3898: Clean up job state hooks and validations.
3898: Add error messages to assertions. Use assert_raise.
3898: Populate state column using SQL transaction instead of load/save N times.
3894: Use --first-parent and readable timestamp in fuse package as
3894: Fix -n that should have been -z. Flatten condition.
3894: Add comments about CONFIGSRC in --help text.
3894: Set arvados version in Gemfile.lock to one that exists.
Merge branch '3894-gem-version' refs #3894
Merge branch '3894-improve-run-tests' closes #3894
3898: Assume a job Failed if it is locked but state fields indicate Queued.
3898: Assume a job Failed if it has started but state fields indicate Queued.
3898: Merge branch 'master' into 3898-job-state-attr-TC
Merge branch '3898-job-state-attr-TC' closes #3898
Revert "Bugfix: do not try to overwrite docker_image_locator if it is already set." refs #3898
Fix another -z that should have been -n. refs #3894
Report crunch_worker_state=down for nodes not occupying a worker slot. refs #3605
3960: Remove unused has_permission? method.
3960: Make editable? return false or true, never nil.
3960: Fix readonly/writable flag on repo list. Add tests.
3889: Clean up counter checks. For now, warn instead of failing.
3960: Merge branch 'master' into 3960-show-more-repos
Merge branch '3960-show-more-repos' closes #3960
3991: Set workbench_address in docker config.
3991: Run passenger script using bundle exec.
3991: Handle "no workbench_address configured" more gracefully.
3991: Use db:structure:load and db:seed instead of db:setup: the database already exists here.
3991: Change docker default to "no root url redirect". Make "not found" message more friendly.
3991: Fix config checks in admin_notifier.
3991: Merge branch 'master' into 3991-docker-workbench-address
Merge branch '3991-docker-workbench-address' closes #3991
Fix typo in error message. Refs #3961
3616: Use ActionController::Live to avoid buffering entire response in memory.
3616: Use "ensure" to close response stream.
Merge branch '3616-live-stream' closes #3616
3782: Isolate file_enumerator stubbing to the current test. Introduce mocha.
3775: Fix crunch-job exit status: exit non-zero after "freeze"
3782: Remove obsolete helper method.
3782: Merge branch 'master' into 3782-stub-file-content
3782: Use stubbed file content for "inactive user can retrieve user agreement" test.
3782: Merge branch 'master' into 3782-stub-file-content
Merge branch '3782-stub-file-content' refs #3782
3775: Merge branch 'master' into 3775-fetch-git-repo
3775: Fix syntax error 2 commits ago.
3775: Remove stagnant $job_has_uuid flag. Every job has a uuid. The
3775: Run local/dev jobs on code from arbitrary remote git repositories.
3775: Add TODO comments in suspicious-looking collate_output corners.
3775: Clean up reporting of exit codes: say "0 with signal 2", not "512".
3775: Remove redundant log message.
3775: Merge branch 'master' into 3775-fetch-git-repo
3687: in User#can?, consider permission links with head_uuid pointing directly at the target.
3687: OK here just means this loop is OK, not everything is OK!
3687: Add comment about redundant lookup.
3882: Test response status and #cancel method.
3882: refactor update-or-cancel code.
3687: Merge branch 'master' into 3687-grant-permission-on-repo-TC
Merge branch '3687-grant-permission-on-repo-TC' closes #3687
3782: Restore config settings changed during tests. (copied from API server)
3782: Stub IO.pipe() with StringIO instead of stubbing content with KEEP_LOCAL_STORE.
3782: Avoid using a huge read buffer when client requests a huge range. Add test.
3775: Recognize ssh short form without @ symbol. Expand comments.
3782: Merge branch 'master' into 3782-stub-io-pipe
Merge branch '3782-stub-io-pipe' refs #3782
3828: Remove duplicate "write pidfile" block.
3828: Use defer to close pidfile. Avoids leftover pidfile if Listen fails.
3828: Wait for listener to start before connecting to it. Fix test
3775: Merge branch 'master' into 3775-fetch-git-repo
3775: Set state=Running when creating a Job in local mode.
Merge branch '3775-fetch-git-repo' closes #3775
Merge branch '3828-keepproxy-race' closes #3828
3775: Update perlpod. Use items_available. Be more conservative when
3775: Update comment
3775: Update comment
3775: Fix SDK usage.
Merge branch '3775-fetch-git-repo' closes #3775
4044: Clean up channel and pipe usage.
4126: Preserve hash form when populating component parameters.
4044: Add comments to "continue" statements.
4044: Merge branch 'master' into 4044-crunchstat-wait
Merge branch '4044-crunchstat-wait' refs #4044
3826: Remove old cruft. Dry up OpenAndReadAll.
3826: Fix up CPU usage accounting.
3826: Add host-level network IO monitoring, and placeholder for task-level.
3826: Clean up cgroup id strings. Output stats for all interfaces.
3826: Use /proc/PID/net/dev to get container net stats.
3826: Print cumulative net io as well as interval io.
3826: Tweak log format.
3826: Remove duplicate log message, add missing crunchstat: log prefix.
3826: Skip net/dev lines with unexpected # tokens.
3826: Change cpu time logging format to match net traffic.
3826: Add a couple of easy unit tests.
3826: Look for stats files while polling, to get per-container stats that appear after first poll.
3826: Tweak memory stat log format to match cpu and net.
3826: Fix up block IO stats.
3826: Add TODO comment on FindStat.
3826: Move cpu stats into a function, fix interval reporting.
3826: Move cpu-counting into a function.
Resolve race condition: wait for response before checking mock expectations. refs #3782
4219: Accept signatures even if other hints are present in URL.
4219: Use a precompiled global regexp for signature verification.
3826: Use correct variable Sscanf. CPU stats work again.
3826: Change FindStat to OpenStatFile to eliminate redundant Stat()
3826: Just modify lastSample maps in place, instead of returning the supplied pointer.
3826: Fix up formatting cf. gofmt
3826: Roll up Open() attempts into a slice and a loop.
4219: Add test for missing size hint. Dry up "known signature hint" consts.
Merge branch '4219-verify-with-hints' closes #4219
4229: Get rid of sleep() in test
4229: Add missing Close() calls
3826: Reverse fields in "stat not found" warning.
4189: Look up owner's writable_by when API did not provide writable_by for the object in question.
4189: Admit in writable_by that an object is writable when its owner_uuid is writable.
4189: Add writable_by to User API response, so writable_by is available for every owner_uuid.
Merge branch '4229-close-files' closes #4229
4062: Merge branch 'master' into 4062-infinite-scroll-repeat-TC
4062: Do not clobber next_page_url filters with local filters.
3826: Clean up slice initialization.
3826: Use global logChan. Use ioutil.ReadFile.
3826: Use strconv instead of scanner + sscanf.
3826: Fix confusing reuse of variable name.
3826: Merge branch 'master' into 3826-crunchstat-netstats
4266: Fix log processing performance.
4266: Clean up streambuf.split(newline) loop.
4266: Fix OOM when stderr has a lot of data with no newlines.
4266: Tweak long-line handling.
4266: Add missing "next" statement, oops.
4266: Use each_line as enumerator instead of building an array with lines.each.
4266: Add word missing from comment.
4266: Merge branch 'master' into 4266-log-performance
Merge branch '4266-log-performance' refs #4266
3824: Show stdout/stderr if shell cmd fails in install script.
3826: Merge branch 'master' into 3826-crunchstat-netstats
Merge branch '3826-crunchstat-netstats' closes #3826
Add services/crunchstat to test suite. refs #3826
Report "swap" memory statistic if kernel provides it. refs #4185
4297: Do not update_node_status more than once per second.
4297: Cache results of git-fetch-pack, git-tag, and API token
3824: Use a docker volume for TASK_WORK, clean up env var and tmp dir handling, improve logging.
3824: Add dnsmasq to compute images, to make docker-in-docker work. refs #3333
3824: Do not say "There are jobs in the queue ahead of this one" if queue_position is unknown.
3824: Use keep volumes big enough to store the arvados/jobs docker image. Do not clobber settings.conf if not starting apiserver.
4297: Use save! instead of save to ensure failures get logged.
4297: Rename @have_tags -> @job_tags, @have_commits -> @fetched_commits. Add comments.
4297: Merge branch 'master' into 4297-dispatch-load
3706: Silence fallback-to-keep warning, show both errors (API and Keep) if both fail.
Merge branch '4297-dispatch-load' refs #4297
3706: In CollectionReader, try fetching manifests before/after API
3706: Do not leave zero-length file segment at end of stream after finish_current_file().
3706: Consolidate more regular expressions into util package.
3706: Remove automatic normalization. Add --normalize option to
3706: Merge branch 'master' into 3706-keep-warning
3824: Fix up logging messages.
3824: Add sources.list file missing from commit:0f9aaf1
3824: Remove redundant addition of arvados package repo and key.
4091: Use <=N instead of <N+1. Pass "uuid not in" filter to API instead of adding custom filter params.
3824: Use Data::Dumper instead of custom quoting to show srun/task exec args.
3824: Merge branch 'master' into 3824-crunch-container-setup
3824: Bring back $! in error message, in case it's helpful.
4088: Add usage docs to filterable.js
3824: Set mode on /etc/fuse.conf in compute image.
3824: Remove redundant "apt-get update", standardize on "apt-get update -qq" and "apt-get install -qqy".
3824: Import GPG key needed for RVM installation.
3706: Use normalize() in arv-mount.
4088: Add usage docs to filterable.js
Merge branch '4088-filterable-docs' refs #4088
Merge branch '4359-project-paging' closes #4359
Fix google-api-client version deps. No issue #
3824: Fix comment to state that tasks must not assume $TASK_KEEPMOUNT is /keep.
3824: Merge branch 'master' into 3824-crunch-container-setup
Merge branch '3824-crunch-container-setup' closes #3824
3706: Catch and return exceptions in _populate_* methods to dry up _populate()
3706: normalize() returns None instead of self. Add equally convenient, and more Pythonic, manifest_text(normalize=True) feature.
3706: Put --normalize and --max-manifest-depth arguments in a mutually_exclusive_group.
3706: Really make the API server return the wrong data, instead of just saying so.
3706: Two blank lines after class definition.
3706: Prefer "if x" to "if len(x) > 0".
3706: Prefer join() to many string concatenations.
3706: Clarify test case.
3706: Process directory trees in sorted() order, instead of the order
3706: Mutating normalize() does not return self.
3706: Prefer "is None" and "is not None" to "== None" and "!= None".
3706: Add back "collections.get => 404" to "fallback to Keep" test.
3706: Remove superfluous conditional.
3706: Merge branch 'master' into 3706-keep-warning
3706: Add (skipped) test for broken max_manifest_depth=0 feature.
3706: Oops, the test was broken, the feature works fine.
Merge branch '3706-keep-warning' closes #3706
Update version dependency to ensure CollectionReader has a normalize() method. refs #3706
4389: Fix infinite loop when ownership trail terminates at a self-owning user.
4389: Add debugging tools to dev bundle.
4084: Move all pane state into $pane, flatten panes with multiple entry points.
4084: Fix up whitespace (4-space indent, like the other *.js)
4084: Skip the "inactive pane" optimization when the div is not a tab pane.
4084: Use assert_text, not assert page_has_text?, to make tests reliable.
4084: Merge branch 'master' into 4084-log-pane-refresh-TC
4084: Ignore bubbling arv-log-event events, just process each once at the original target.
4084: Update comments.
4084: Remove excess loaded class.
4084: Fix dashboard refresh.
4084: Add comment.
4434: Test whether sort order matches application locale, instead of testing a locale-dependent collation rule.
4389: Merge branch 'master' into 4389-breadcrumbs-infinite-loop
4389: Add test.
4084: Update views that were still using pane-anchor.
4389: Merge branch 'master' into 4389-breadcrumbs-infinite-loop
4084: Fix wrong state name: jobs are Running, pipelines are RunningOn[...].
Merge branch '4389-breadcrumbs-infinite-loop' closes #4389
3400: Remove ArvadosResourceList#all
4434: Merge branch 'master' into 4434-collation
4434: Assume only that if a<b are lowercase ascii, aX<bY for any X, Y, collation.
4024: Test that items_available takes filters into account.
Expand websocket config comments. refs #4233
4434: use select! instead of &= to preserve duplicates.
2853: Use rendezvous hashing to select probe order.
2853: Use rendezvous hashing to select probe order in Python library.
2853: Fix instance ID in test fixture hostnames. Clean up brittle tests.
2853: Add "reference set" test to check probe order agreement between implementations.
2853: Add tests for reference set and some edge cases.
2853: Update tests to survive ServiceRoots being a map[string]string.
2853: Use the same keep_service UUIDs so tests behave reliably.
2853: Add comment.
2853: Remove shuffledServiceRoots() wrapper.
4024: Fix "compare pipelines" button.
3400: test limit between server maxpagesize and 2x server maxpagesize.
Merge branch '4434-collation' closes #4434
2853: Merge branch 'master' into 2853-rendezvous
Export vars provided on command line, e.g., ARVADOS_DEBUG=1. No issue #
2853: Remove redundant test case.
4363: Fix filename munging. Add tests.
2853: Use Md5String() in getWeight().
2853: Fix reference to reference set.
2853: Avoid using reserved word "hash" as a variable name.
2853: Use mock from KeepClientServiceTestCase for rendezvous tests
2853: Merge branch 'master' into 2853-rendezvous
Merge branch '2853-rendezvous' closes #2853
4363: Reject NUL characters in filenames in CollectionWriter.
4363: Test that munged filenames show up in os.listdir().
4363: Test edge cases as directory names, too.
4024: Remove unused /pipeline_instances/?search=x support.
4024: Add test for double-load bug.
4024: Use filterable-control input values (if any) when first loading dynamic content.
Relax activesupport dependency to accommodate 4.x. refs #4388
4388: Fix javascript errors.
4388: Update bundle
4388: Fix typo
4024: Merge branch 'master' into 4024-pipeline-instances-scroll
4024: Merge branch '4388-workbench-update' into 4024-pipeline-instances-scroll
3400: Do not fetch_multiple_pages in #index actions.
4024: @limit override must happen before find_objects_for_index.
3400: Do not fetch API results just for the sake of looking up resource_class.
3824: Run build_script (and create *_WORK) inside the container.
3824: docker issue 3172 is closed, build works without -q.
Merge branch '4388-workbench-update' closes #4388
3824: autorestart non-detached services regardless of exit code.
3824: Create "all users" group in seeds.rb instead of asking a human to do it.
3824: Check whether there is anything to install before trying to lock $destdir.
Merge branch '3824-task-work' closes #3824
4024: Make items_available, result_limit, and result_offset work for empty result sets.
4552: Fix conflict between ensure_unique_name and signed manifests.
4552: Use faster database_cleaner strategy.
4552: Tidy up ensure_unique_name block.
4400: Silence git error messages during package install (we ignore them anyway)
4533: Add /database/reset API to roll everything back to fixtures.
4533: Update bundle.
4533: Reset fixtures after testing user deactivation.
4363: Remove redundant test.
4363: Validate utf-8 encoding of manifest_text.
4363: Explicit UTF-8 for test string.
4363: Merge branch 'master' into 4363-less-filename-munging
4533: Reload database seeds during reset.
4533: Fix crash on nonexistent (or non-User) owner_uuid of role group.
4533: Add more tests for disabling reset feature in config file.
4363: Specify ASCII encoding in test case.
4363: Accept manifest_text in a non-UTF-8 encoding that is equal to its UTF-8 encoding.
4363: Fix size hints for multi-byte manifests (backported from f78caf9).
4363: Fix flaky test case (race with filterable.js debounce timer).
Merge branch '4363-less-filename-munging' closes #4363
4552: Set permit_unsigned_manifests=false needed for test.
Merge branch '4552-collection-unique-name' closes #4552
4233: In crunch-dispatch, log timestamps in UTC in the same format crunch-job uses.
3824: Merge branch 'master' into 3824-docker-fixes
Merge branch '3824-docker-fixes' refs #3824 refs #4186
4400: Do not add ".commitdate.hash" tag if a build tag is already set.
Fix syntax error in whitespace. refs #3824
Fix wrong variable assigned. refs #3824
4400: Fix whitespace. Fix tagging in nodemanager, too.
4400: Dry up test recipes for python bits. Tweak "extra test args for suite X" feature.
Merge branch '4400-python-install-herring' refs #4400
4533: Merge branch 'master' into 4533-remote-reset
4533: Set group_class="role" on "All users" group.
4400: Fix syntax error.
4651: Accept "false" as false for a boolean param. Reject bogus strings.
Merge branch '4400-dry-python-tests' refs #4400
4651: Fix KeepDisksController test failures.
4651: Update NodesController _ping_requires_parameters format.
4651: Merge branch 'master' into 4651-boolean-params
4651: ensure_unique_name param is not required.
4533: Add integration test.
4651: Fix exceptions. Accept 0 for false, 1 for true.
4533: Merge branch 'master' into 4533-remote-reset
Merge branch '4651-boolean-params' closes #4651
Update arvados-cli Gemfile.lock. No issue #
Ignore failure at "easy_install *.egg". refs #4400
4533: Use Rails.env instead of ENV["RAILS_ENV"]. Add comment in test case.
3781: Set CORS headers in API responses.
3781: Add tests for CORS headers. Turn off CORS headers at OPTIONS /login etc.
3781: Set CORS headers in keepproxy responses.
3781: Add POST method for writing without knowing MD5.
3781: Add allowed headers. Respond to OPTIONS at any path.
3781: Fix error propagation.
3781: Remove js cruft from api server.
3781: Add browser->api/keepproxy angular app as Upload tab on collections#show
3781: Merge branch 'master' into 3781-browser-friendly-servers
3781: Merge branch '3781-browser-friendly-servers' into 3781-browser-upload
3781: Fix OPTIONS routing.
3781: Merge branch '3781-browser-friendly-servers' into 3781-browser-upload
4533: Merge branch 'master' into 4533-remote-reset
4533: Reset fixtures after each test case by default.
4533: Move "database reset does not break basic operations" into its own test.
4533: Disable database-reset for all diagnostics test classes.
4533: Skip one-fixture-reset-per-test in a few test classes.
4533: Move all controller tests into controllers/, merge overlapping class defs.
4533: Replace enable_remote_reset config knob with check for existence of non-fixture users.
4156: Install dependencies to user gempath, not system gempath, even
4595: Fix nodes.list?select=[...] crash when is_admin. Also, do not ignore where/limit/order.
4595: Merge branch 'master' into 4595-node-list-select
Merge branch '4595-node-list-select' closes #4681
4533: Remove config flag from default section, too.
4533: Consolidate database-reset tests. Fix up ensure/reload_routes! blocks.
Merge branch '4533-remote-reset' closes #4533
Merge branch '3781-browser-friendly-servers' refs #3781
4533: Pass CORS test when no omniauth provider is configured. refs #4533
4699: Fix crash when collection has tags but no name. closes #4699
4156: Remove Gemfile.lock because this is a gem, not an application.
4533: use "extend" when calling module methods from class methods. refs #4533
4533: Lock all tables before deleting contents.
Merge branch '4533-lock-tables-first' refs #4533
4525: Call enable_disable_selection_actions once per container to
4533: Fix wrong lock mode. refs #4533
3781: Merge branch 'master' into 3781-browser-upload
3781: Report as KiB instead of K.
3781: Disable start button when there is nothing to do.
3781: Fix up comments.
3781: Fix error message, and add actual error detection.
3781: Use ===, !== instead of ==, !=
3781: Deobfuscate.
3781: Use $.each for simple loops.
3781: Fix up dropdown: match appearance of other buttons, tweak wording,
3781: Fix push-to-bottom-when-Done so it works on the last item, too.
3781: Use regexp instead of split: avoid making big arrays when working on big manifests.
3781: One less Regexp.
4465: Link to arvados.org revision history instead of github.com code view.
4156: Merge branch 'master' into 4156-bundle-install-user
3781: Fix more {==,!=} -> {===,!==}.
4533: Lock all tables, not just the ones with fixtures. refs #4533
Merge branch '4156-no-gemfile-lock-for-gem' refs #4156
3781: Tweak link text. Remove redundant title attributes.
3781: Do not portray "stopped" state as an error if it was reached by clicking Pause.
3781: Add Uploaded state, fix handling of Pause during appendToCollection.
3781: Add ng-cloak style.
3781: Merge branch 'master' into 3781-browser-upload
3781: Update tests.
4533: Wait up to 10s for "lock tables". refs #4533
Reset listener=nil before running main() from test cases, so
3781: Merge branch 'master' into 3781-browser-upload
3781: Fix exaggerated claims of being finished.
3781: Fix use of committed flag. That is now called state==="Done".
3781: Go to "Done!" state when the last upload completes despite a late call to stop().
3781: Fix progress% (100, not NaN) and manifest format (>=1 data locator) for zero-byte files.
3781: Fix trigger() usage: second argument is an array of handler args.
3781: Add test cases: empty files, renaming, and error reporting.
3781: Add singletest function.
3781: Merge branch 'master' into 3781-browser-upload
3781: Fix test that assumes only one empty collection is readable.
Merge branch '3781-browser-upload' closes #3781
4156: Fix install destination for gem install --user-install. Make tmp-gem-home discovery work under rvm.
4156: Do not use --user-install -- just install to the GEM_HOME we selected.
4156: Become rvm-aware.
4481: Fix ambiguous "Keep id" -> "locator" in example scripts.
4156: Merge branch 'master' into 4156-bundle-install-user
4754: Move perf/prof deps to :test/:performance groups.
4156: Move "bundle install" into separate install_ task like apiserver. Remove one more GEMHOME after merge.
4156: Update comments, remove excess cut, tweak GEM_HOME-reporting.
4156: Uninstall old gems just before building new ones. Avoid an extra source of truth for the "gems we install" list.
Merge branch '4156-bundle-install-user' refs #4156
4515: Add controller tests for search dialog.
Ignore .eggs/ (*.egg isn't enough: .eggs/README.txt gets installed too, as a human-readable .gitignore.)
Use gem uninstall --force to skip dependency prompt. refs #4785
Restore scroll on projects menu. Do not offer "Add project" button in projects dropdown in chooser dialog. closes #4811
Remove excess heading and divider. No issue #
Merge branch '4515-search-empty-project' closes #4515
4875: Use range in for loops.
4875: Let the OS choose port numbers for fake servers.
Merge branch '4754-performance-TC' closes #4754
4523: Add "store long string" test.
4523: Fix whitespace.
4523: Fix down-migration.
4523: Remove dev-only checks in migration.
4523: Fix column order to match migration order.
4523: Dry up migration and test cases.
Merge branch '4523-owner_uuid-index' refs #4523
Fix version strings to comply with PEP-440. No issue #
Set RAILS_ENV consistently. Support "sevices/keepstore_test=-v" etc. No issue #
4875: Merge branch 'master' into 4875-keepclient-test-race
Fix go test args. No issue #
Merge branch '4875-keepclient-test-race' closes #4875
Fix whitespace, cf. gofmt. refs #4875
Remove cruft. No issue #
3021: 4399: Convert some tests from selenium to phantomjs. Restart Headless less.
3021: Silence "invalid regexp" errors while typing regexp; put input in "has-error" state instead.
3021: Skip angular init if angular is not loaded.
3021: Clean up headless/selenium/javascript choices.
4523: Merge branch '4523-search-index' of git.curoverse.com:arvados into 4523-search-index
4523: Description is not unsearchable after all.
4523: Increase string length limit. (Postgres allows nearly 1MiB here, no need for a smaller limit.)
4523: Simplify migration.
4523: Do not search :text fields at all.
4523: Restore indexes accidentally deleted from structure.sql.
4523: Come back down to earth enough to make tests pass.
3021: 4399: Refactor headless stuff into a module. Clear up new/start/stop use.
3021: Use headless helper in performance and diagnostics tests, too.
3021: Wait for dialog to close before asserting page transition.
Rename {workbench,apiserver}_test to {apps/workbench,services/api}_test,
3021: Fix assertion broken in 9c10212.
3021: Wait for shown.bs.modal before trying to click butttons in the modal.
3021: Add random part to magic string.
3021: Merge branch 'master' into 3021-more-phantomjs
3021: Use selenium to land on #Advanced tab.
Merge branch '3021-more-phantomjs' refs #3021
3408: Propagate API error messages to caller.
Merge branch '3408-go-sdk-api-errors' refs #3408
3021: Fix phantomjs races by waiting for pages to appear. refs #3021
Make angular shim minify-safe. No issue #
Diagnostics really do need selenium. refs #3021
Add a magic pseudoclass to body, instead of appending a magic div. Selenium seems to like this better. refs #3021
3021: Fix races in performance test.
3021: Do not drop/create API database at startup: run-tests.sh does that for us.
3021: Update bundle.
3021: Optimize manifest parsing.
3021: Load test fixtures during api install.
3021: Add workbench_profile target. Rename _performance to _benchmark.
3021: Allow '--only X' to override '--skip X'.
3021: Add web-inspectable profiling mode.
3021: Do not check user notifications unless they are going to be displayed.
3021: Check for errors when installing PyYAML.
3021: Do not install *.egg after running tests: "pip install PyYAML" does what we need.
3021: Merge branch 'master' into 3021-workbench-perf
3021: Install docutils (unlisted dependency of python-daemon).
Merge branch '3021-test-perf' refs #3021
Fix crash on missing return_to param.
3021: Add a performance test.
3021: Call Rails.application.eager_load! only once, not every single
3021: Do not compute etag for initial model state unless/until actually needed.
3021: Use Oj to encode API responses, and to decode them in tests.
4523: Clear up manifest-signing vs. locator-signing.
Reduce file_names limit to 2^12. refs #4523.
3021: Remove reference to deleted speclist arg.
3021: Update Gemfile for Ruby SDK bugfix.
3021: Do not show user notifications on error pages.
3021: Merge branch 'master' into 3021-workbench-perf
3021: Revert "3021: Do not drop/create API database at startup: run-tests.sh does that for us."
3021: Do not drop/create API database at startup if run-tests.sh has done that for us.
3021: Reset API fixtures before running the first test case.
Merge branch '3021-workbench-perf' refs #3021
Update bundle. refs #3021
3021: Replace more "render json:..." with faster Oj.
3021: Use Marshal dump/load to save @old_attributes. Otherwise, hashes
3021: Merge branch 'master' into 3021-api-performance
Merge branch '3021-api-performance' refs #3021
Fix preview pane disappearing on narrow displays. closes #4918
4472: Include output of failed tasks when generating job output manifest.
5037: Remove connection pool. Now, api(cache=False) refers only to the discovery document cache.
5037: Mention where this cache is, instead of describing what a cache is.
Merge branch '5037-python-sdk-thread-safe' refs #5037
5011: Add replication argument to CollectionWriter.
5011: Add --replication argument to arv-put.
5011: Add --replication option to CLI docs.
3021: gitignore rotated log files.
5011: Use replication=2 if unspecified.
5011: Fix run_and_find_collection so it works without --portable-data-hash.
5011: 3410: Export client-specified redundancy as replication_desired.
3021: Add comment about IPv4/IPv6 trap in websocket client.
3021: Propagate unhandled exceptions back to the caller instead of returning None. Show a warning message only when falling back to polling.
3021: Refactor run_test_server.
3021: Round up various implementations of "run test servers" into an arvadostest module. Update to work with new run_test_server.py.
3021: Leave an API server running while test suites run.
3021: Use run_test_server.py instead of the special workbench way of starting an API server.
3021: gitignore **/vendor
3021: Try bundle install --local before resorting to downloading the internet.
3021: Let integration tests write to the api server log before rotating it.
3021: Cache packages with bundle package --all
3021: Fix up some error checking.
3021: Use `passenger stop` to shut down passenger. Restore database fixtures after running DatabaseCleaner-powered tests.
2659: Send anonymous token as a reader token with all requests.
2659: Fix crashes when current_user is nil.
2659: Fix up 404 page.
2659: Fix crash in /users/current when not logged in.
2659: Set group_class=role on anonymous group, change name to "Anonymous users".
2659: Reload seeds (system_user et al.) after cache is cleared. Be more careful about errors while creating seeds.
2659: Use Collection.creatable? to toggle selection actions.
2659: Fix indentation.
2659: Toggle features according to editable? and creatable? instead of current_user.is_active. Fix up whitespace.
2659: Back out unneeded condition. Fix up whitespace.
2659: Remove obsolete "don't create collections in workbench" rule.
2659: Update test to match changed error message.
2659: Remove obsolete "don't create jobs in workbench" rule.
2659: Check anon config at runtime instead of boot time in skip_before_filter. This makes it possible for test cases to toggle behavior by changing configuration on the fly.
2659: Do not ask to skip_around_filter(only:...) and skip_around_filter(if:...) for the same filter, because that doesn't work.
2659: Add separate test for "run pipeline" notification when anonymous token is configured, skip it for now.
2659: Fix anonymous token test.
2659: Token fail during download is 404 when anon token is enabled, 302 otherwise.
2659: Fix infinite recursion while creating system_user.
2659: Fix up chicken-and-egg conditions in seed setup.
2659: Split skip_around_filter back into two separate cases. 063c546 was misguided here.
2659: Use assert_operator to help debug unreliable test.
2659: Use a partial instead of a switch block for the not-so-generic "create object" button. Don't show one at /jobs or /collections.
5011: Use None, not 0, to signify default replication.
5011: Sync local_store_put/get() signatures to put/get(). Add docstrings.
5011: Convert comment to docstring.
Merge branch '2659-anonymous-share-projects' refs #2659
3021: Fix misspelled variable.
3021: Tidy up (and document) the choice of exactly which server we expect to run/reset/stop.
3021: Set up a proxy-only keepclient manually, instead of communicating with global/env vars.
3021: Remove ARVADOS_KEEP_PROXY support. (If you need to interfere
3021: Fix use of inaccessible global in atexit handler.
3021: Don't worry about env vars, run_test_server.py does that.
3021: Check whether a randomly selected port is available before using it.
3021: In start(), if a stale server is already running (but we can't reset() it), kill it.
5011: Merge branch 'master' into 5011-arv-put-replication
5011: Dry up three different solutions for mocking keep_services().accessible().
Merge branch '4472-save-partial-output' closes #4472
5011: Fix up docstring.
5011: Add tests and comments for redundancy vs. replication_desired attrs.
5011: Add forward compatibility, so we don't need backward compatibility later.
3021: Set env var to empty string instead of removing it. Unsetenv isn't worth introducing a dependency on Go 1.4.
Avoid optional disk cache usage if it is owned by a different user. refs #2659.
3021: Print reassuring messages about expected warning messages during "bundle install --local".
3021: Reset env vars in any reset(), not just in start().
3021: Give up earlier if ARVADOS_TEST_API_HOST is set but reset() fails.
3021: Use kwargs to pass options through make_mount(). Dry up FuseTagsUpdateTest.
3021: Remove unused variable.
3021: Reduce race potential by resetting database after fuse unmounts.
3021: Make sure any unexpected openssl output ends up on stderr.
3021: We use passenger in dev and test environments, too.
3021: Merge branch 'master' into 3021-leave-api-running
3021: Use bind(0.0.0.0, 0) instead of grep /proc/net/tcp to find a port number.
5011: Make docstring less scary.
5011: Merge branch 'master' into 5011-arv-put-replication
Merge branch '3021-leave-api-running' refs #3021
5011: Fix comment about when local_store_* methods get used.
Merge branch '5011-arv-put-replication' closes #5011
Merge branch '3021-leave-api-running' refs #3021
5037: Use a newer Python SDK (with no connection cache) instead of cache=False.
Use virtualenv to get a newer setuptools that doesn't munge Python versions. No issue #
5030: Move log graph javascript into its own file.
5030: Hide cpu/mem/io graph until data points arrive.
4958: Detect old browsers and recommend upgrading.
4958: Avoid squishing dax's aspect ratio when window is narrow.
4958: Add tests.
5175: Do not add +sha1 tag unless --sha1-tag flag given.
5176: Fix handling of escape sequences in manifest filenames.
5176: Add spaces-in-filenames tests.
4523: Remove unneeded else.
4523: Fix up whitespace.
4523: Add tests for some edge cases.
5175: Revert change to find_packages(), it gets confused by run_test_server.py.
5715: Sort imports.
Merge branch '5175-pep-440' refs #5175
5176: Fix broken fixtures, add tests.
5176: Update bundles.
5176: Update Gemfile.lock.
5037: Merge branch 'master' into 5037-nonocache
5037: Remove git tags from SDK version restrictions.
5183: Disable animation on modal. (It seems to cause intermittent test failures.) refs #5183
Fix javascript errors when closing modal. refs #5183
4958: Rename test file.
4958: Test WebSocket support.
4958: Tweak wording. Fix margins.
4958: Merge branch 'master' into 4958-old-browser-alert
Merge branch '4958-old-browser-alert' closes #4958
3410: Rename redundancy -> replication columns.
3410: Add default_collection_replication to config and discovery doc.
3410: Add replication attributes (and rules about updating them) to model and docs.
3410: Add tests for replication attributes.
3410: Fix munge_manifest_locators!: don't skip locators that have no +hints.
4951: Add localStorage to browser dependency checks.
4951: Rearrange js to make use of Rails built-ins. Add generic AJAX error handler.
4951: Empty out ActionMailer::Base.deliveries so we can count deliveries caused by the current test.
Merge branch '4951-request-vm-TC' refs #4951
Fix undependable test (incorrectly assumed a particular job would be on the first page of /jobs). No issue #
Default to TESTOPTS=-v for rake test.
Make tmp/cache during workbench install. No issue #
Clean up and exit on SIGINT.
4839: Use node[crunch_worker_state], not node[info][slurm_state], to make shutdown decisions.
Merge branch '4839-worker-state' closes #4839
5200: Enable anonymous file view/download.
5200: Do not fail tests if application.yml is missing: defaults are enough to pass tests.
5200: Fix overly sensitive fuse test. (Do not fail every time a fixture is added.)
5011: Use a threadsafe wrapper to feed side effects to mocks.
5011: Fix unreliable test.
Merge branch '5030-hide-graph-until-data' closes #5030
Merge branch '5037-nonocache' closes #5037
Merge branch 'master' of git.curoverse.com:arvados into 3408-production-datamanager
Merge branch 'master' into 3408-production-datamanager
Merge branch '5011-thread-safe-test' refs #5011
Ensure result order is predictable, even if client-provided orders do not specify a complete ordering.
3410: Rename maybe_clear_redundancy_confirmed to maybe_clear_replication_confirmed.
3410: Update comments.
3410: Fix overly sensitive test.
3410: Fix search index so [old versions of] tests pass after down-migration.
3410: Update comments.
4759: Use ISO 8601 timestamps with fractional seconds in API responses.
3410: Merge branch 'master' into 3410-replication-attrs
Merge branch '3410-replication-attrs' closes #3410 refs #5011
4759: Ignore args to as_json.
4759: Add test for inequality filters.
4759: Add functional tests for timestamp precision.
Merge branch '4759-timestamp-precision-TC' closes #4759
3408: Merge branch 'master' into 3408-production-datamanager
5043: Accept long stderr lines from crunch tasks.
5043: Use Go's log package to serialize writes. Lose logChan.
5043: Split long stderr lines rather than consume unlimited memory.
5349: Add test to protect bugfix in a1d70a2116673ac46f4946115bdb5048137689a2. Fix regression.
5043: Remove errant +1 from error message.
Merge branch '5043-crunchstat-long-lines' closes #5043
5261: Move project name and description into a Summary tab.
5261: Update tests.
5261: When redirecting during an AJAX request, send the target URI in
5261: Redirect to destination project's Collections tab when copying/moving collections.
5261: Fix {redirect_to @object} usage.
5105: Add API stub helpers and basic ArvadosApiClient unit tests.
5105: Move displaced comment.
5105: Treat not-logged-in AJAX requests as errors, instead of redirecting to login prompts.
5105: Tidy up test case.
5105: Really call the report_issue action, instead of fetching "/",
5105: Remove unnecessary self.included/class_eval stuff.
5105: Skip database reset where we use a mock instead of the real API.
5261: Move project name from Summary tab back to heading area.
5261: Rename Summary tab to Description.
5105: Include form data with report_issue request.
5261: Change placeholder text from "New group" to "New project"
5261: Merge branch 'master' into 5261-project-description
5261: Fix DST transition bug: browser timezone != test suite timezone.
Merge branch '5261-project-description' closes #5261
5182: Advise adding CORS headers to proxy error responses.
5182: Improve error reporting in uploader.
5182: Silently skip Angular initialization if injector() returns null.
5105: Restore previous token in an ensure block.
5105: Better assertion for tab-loading error display.
5073: Document docker limitations. Bring "overview" up to the top of the install section.
5073: Update ruby version to 2.1.5
5105: Clarify "no double-layout" asserts. Remove redundant "not on welcome page" asserts.
5182: Tweak error messages.
Merge branch '5105-ajax-redirect' closes #5105
Merge branch '5073-docker-limitations' closes #5073
Merge branch 'master' into sigint
Merge branch '5182-cors-error-reporting' closes #5182
Merge branch 'sigint' closes #5178
5207: Change expires_at column type from date to datetime.
Merge branch 'patch-1' from github.com/daftano. No issue #
5416: arv-git-httpd provides HTTP access to git repositories, using Arvados token authentication.
5416: Add options to docs.
Merge branch 'master' from github.com/daftano. No issue #
5416: Add integration tests.
5416: Allow users with repo write permission to update modified_at.
Merge branch '5207-expires-at-time' closes #5207
5416: Add path variants test. Skip cloning arvados repo when not needed.
5416: Merge branch 'master' into 5416-git-auth-token
5416: Add arv-git-httpd.
5416: Add --retry flag: when a test suite fails, prompt to try that suite again.
5416: Add arv-git-httpd to install docs, api config, and discovery doc.
5198: Remove href attributes from remote:true links to prevent ctrl-click et al. from doing the wrong thing.
5276: Remove redundant test teardown.
5276: Improve job log graph tests.
5276: Reveal and update the log graph as soon as the first data point
5276: Use using_wait_time 0 instead of evaluate_script.
5276: Name log fixtures. Fix over-sensitive test.
5276: Merge branch 'master' into 5276-job-graph-phantom-tooltip
Merge branch '5276-job-graph-phantom-tooltip' closes #5276
5198: Update comment.
5198: Merge branch 'master' into 5198-remote-link-ctrl-click
Merge branch '5198-remote-link-ctrl-click' refs #5198
5416: Add BasicAuth compatibility shim for go1.3.
5416: gofmt formatting changes.
5416: Merge remote-tracking branch 'origin/5416-arv-git-httpd' into 5416-arv-git-httpd
5416: Fix overly sensitive error message regexp.
5416: Improve variable names & filenames. Report original error if git fails with nothing on stderr.
5416: Add permission-denied tests.
5416: Merge branch 'master' into 5416-arv-git-httpd
Merge branch '5416-arv-git-httpd' refs #5416
Merge branch '5416-arv-git-httpd' refs #5416
5416: Prefer repos stored locally by uuid, fall back to name. Update tests to suit 4523 fixtures.
5414: Add client support for Keep service hints.
5414: Clean up Locator / MakeLocator() API.
5414: Add TestGetWithLocalServiceHint.
5414: Use reflect.DeepEqual instead of comparing serialized maps.
5414: Do not blow up if local config file is empty.
5416: Do not blow up tests when git global config is uninitialized.
5416: Improve logging.
Merge branch '5416-git-config' refs #5416
5416: Terminate connections on the configured test database (not necessarily arvados_test).
5414: Fix up formatting.
5414: Add test case for unrecognized/future hints.
5414: Control initializer load order using require_relative instead of alphabetical filenames.
5414: Rename zz_load* to load*; alphabetical names are not needed any more.
5414: Rename zz_preload* to preload*; alphabetical names are not needed any more.
5414: Merge branch 'master' into 5414-keep-service-hints
Merge branch '5414-keep-service-hints' refs #5414
Fix loop label that became disconnected in bae652c, causing crash on "next THISROUND". refs #5642
If provided, use TMPDIR instead of /tmp. No issue #
Fix indentation. Remove errant unclosed <p> tag. refs #5493
3145: Add -readonly flag. Accept multiple -volume arguments.
Merge branch '3145-readonly-volumes' refs #3145
5416: Run keepproxy, arv-git-httpd, and SSL proxy services during Workbench test suites.
5416: Support testargs for go bits, like "services/arv-git-httpd_test=-check.vv".
3126: Accept remote http/https/git url as repository attr in jobs.create/save.
3126: Update API docs.
3126: Do not try to do anything with job.repository in crunch-dispatch if the commit is already in internal.git.
3126: Always act as current_user in Commit.find_commit_range.
3126: Fix test. Avoid excess "git init" by probing with "git branch" first.
3126: Delete git cache dir in test teardown. Remove no-op test.
3126: Handle "no local repository" case better.
3126: Use localhost to simulate API failures. reset_application_config in teardown
3126: Skip some unnecessary db resets.
Merge branch '3126-remote-git-url' closes #3126
Revert errant change in f8e6cb3. refs #3126
5416: Fix comment.
5416: Browse git repository contents in workbench.
5416: Run arv-git-httpd and nginx ssl proxy in test suite.
5416: Do not override git urls for remote hosted repos.
5416: Add read-only clone_urls attribute to Repository resources, deprecate push_url and fetch_url, tidy up config settings.
5416: Disable repository browsing (and skip tests) if git version is suspected unreliable.
5416: Remove second trailing slash in breadcrumbs link.
5416: Use http://foo:bar@host:port/ instead of credential helper.
5416: Fix protected method that should have been public.
Merge branch '5416-ssl-proxy' refs #5416
Merge branch '5416-browse-repo-tree' refs #5416
5416: Test with HEAD in current tree, not master: jenkins index has no master.
Fix typo. No issue #
5739: Remove unnecessary sqlite3 dependency.
Merge branch '5739-no-sqlite3' closes #5739
5562: Use pycurl library (instead of requests) for Keep transactions.
5562: Add fake keepstore server with configurable problems.
5562: Enable TCP Keepalive for Keep requests. refs #5822
5562: Return UA to pool if it worked well enough to get an HTTP status.
Merge branch '5562-pycurl' closes #5562
3145: Respond to PUT without reading request body if no volumes are writable.
5562: Use WRITEFUNCTION instead of WRITEDATA for compatibility with older pycurl. refs #5562
Merge branch '5836-remote-api-server-errors-made-obvious' closes #5836
Merge branch '3145-close-early' closes #3145
5562: Use static method. Fix "TypeError: _socket_open() takes exactly 5 arguments (4 given)".
Merge branch '5562-pycurl' refs #5562
5724: Add blobSignatureTtl to discovery doc. Add config comment/explanation.
5724: Rename -permission-* flags to -blob-sign* to match apiserver. Tidy up some time-related code.
5724: gofmt fixes.
5893: Add realm="git" to WWW-Authenticate header, cf. rfc2617
5893: gofmt fixes. Fix some logs to print as strings, not byte arrays.
5893: Use git credential helpers for arv-git-httpd tests. Fix workbench helper.
5893: Remove "disable repo browsing with git2" stuff.
5724: Update reference to keepstore flag, now -blob-signature-ttl.
Merge branch '5724-blob-signature-ttl' commit '6fc44a6' refs #5724
Merge branch '5893-helper-stdin' closes #5893
Merge branch '5724-blob-signature-ttl' closes #5724
5523: Say "notice" instead of "error" when stats file changes/appears/disappears.
5748: Write index data to http.ResponseWriter, instead of using string
5748: Clean up comments and variable names.
Merge branch '5748-keepstore-leak' refs #5748
Merge branch '5523-stats-error' closes #5523
5745: Serialize writes and data reads, but allow concurrent requests
5745: Fix test order dependency (restore mocked methods), tidy up test data.
Merge branch '5745-serialize-content-only' closes #5745
4882: If -cgroup-cid flag is given, only read container stats. If not given, only read host stats.
Merge branch '4882-no-host-when-container' closes #4882
5748: Use a buffer pool instead of calling runtime.GC() during each GET.
5748: Return the real decoder error for unparseable trash and pull requests.
5748: Check for errors when writing pidfile. Keep it locked while running.
5748: Log clock time for each request.
5748: Update keepstore install docs (-max-buffers, -blob-signing-key).
Merge branch '5748-max-buffers' refs #5748
5748: Put() always takes a non-nil buf.
5748: Time out leak tests after 20s.
5748: gofmt fixes.
Merge branch '5748-max-buffers-leak-TC' refs #5748
5720: Leave existing content in place (and schedule another reload) if a tab-refresh gets cancelled.
5748: Remove runtime.GC() obsoleted by bufferpool. refs #5748
5748: Remove unused import. refs #5748
Tidy up sanity checks. Add checks for fuse.h, pyconfig.h, nginx.
5766: Remove "remove dependencies" step. Detect existing PyYAML faster.
Merge branch 'more-sanity-check-less-download' refs #5766
5416: Add arv-git-httpd.
5416: Add --retry flag: when a test suite fails, prompt to try that suite again.
5416: Terminate connections on the configured test database (not necessarily arvados_test).
5416: Run keepproxy, arv-git-httpd, and SSL proxy services during Workbench test suites.
5416: Support testargs for go bits, like "services/arv-git-httpd_test=-check.vv".
Tidy up sanity checks. Add checks for fuse.h, pyconfig.h, nginx.
5766: Remove "remove dependencies" step. Detect existing PyYAML faster.
6087: Get database time only once per manifest-signing/verifying event, rather than once per locator.
Merge branch '6087-collection-timing' (early part) refs #6087 refs #6092
6094: Avoid SIGPIPE by consuming DATA section even when it's not needed.
6094: Consider arvados_sdk_version (not just script_version) when
Merge branch '6094-install-script-sigpipe' (early part) refs #6094
6087: Use app-configured key by default for blob signing and verification.
6087: Compute portable_data_hash only once during check_signatures.
6087: Use HTTPClient's compression feature (instead of adding the
6087: Add big-manifest tests, with some finer-grained performance numbers on stderr.
6094: Propagate install script stderr+stdout to job log.
Merge branch '6094-install-script-sigpipe' refs #6094
6098: Recreate full text indexes with leading spaces, to persuade Postgres to actually use them.
Merge branch '6098-full-text-index' refs #6098
6146: Retry install (max 3 attempts) if install script fails with no error messages.
Merge branch '6146-retry-install' refs #6146
6146: Exit TEMPFAIL early (without failing the job) if worker nodes cannot run a trivial command.
Merge branch '6146-job-runtime-sanity' refs #6146
In install script, log archive hash before running tar. refs #6146
6146: Ignore SIGPIPE while feeding data to tar. Rely on close() retval instead.
Merge branch '6146-ignore-tar-sigpipe' refs #6146 refs #6094
Tell tar to read to EOF (even if it detects trailing NULs).
6146: Add dns_server_update_command. Update docs & tests for DNS update hooks.
Merge branch '6146-dns-update-command' refs #6146
Update example dns_server_update_command. refs #6146
Remove non-existent migration from structure.sql. refs #3036
6146: Improvements to "kill srun process if slurm task disappears" feature:
6146: Use new SLURM_JOB_ID env var instead of old SLURM_JOBID
6146: Better log message.
Merge branch '6146-log-squeue-lost-tasks' refs #6146
6087: Use app-configured key by default for blob signing and verification.
6087: Compute portable_data_hash only once during check_signatures.
6087: Use HTTPClient's compression feature (instead of adding the
6074: Speed up db query by using octet_length() instead of length(). closes #6223
6074: Clear any existing ActiveRecord select() before adding our own,
6074: Never exceed the configured max_index_database_read (even by one
6074: Update config docs to match new max_index_database_read behavior.
6074: Use each instead of find_each, so our order() and limit() constraints are respected.
6087: Add big-manifest tests, with some finer-grained performance numbers on stderr.
6087: Strengthen "manifest_text is not lost in update" test.
6087: Reset changed-attrs list after saving. Fix only-send-changed-attrs logic. Add tests.
6087: Remove unneeded CollectionsController#update special case.
6087: If attributes are accessed but not loaded due to select(), raise instead of returning nil/{}/[].
Merge branch '6087-collection-timing' closes #6087
6087: Fix MissingAttribute firing for new records during changes_applied. refs #6087
6087: Fix MissingAttribute check, and change it to a debug warning for now
Merge branch '6074-collections-index' closes #6074
6146: Document how --steps really works. Simplify squeue output format and parsing.
Merge branch '6146-log-squeue-lost-tasks' refs #6146
6203: Merge pdh validations into one method. Update comments. Add tests.
6203: Eliminate unneeded variable.
6203: Apply special case only to a 0-byte manifest: don't ignore white space.
6203: Remove redundant split before each_line.
6203: Use each_line instead of split.each.
6203: Fix cheating test.
6203: Use faster =~ instead of match.
6203: Remove unused vars. Remove unnecessary newline manipulation.
6203: Remove unused vars. Remove unnecessary newline manipulation.
6203: Accept (and discard) hints in client-provided portable_data_hash.
6203: Add tests for LOCATOR_REGEXP. Fix regexp to reject "++" and trailing "\n".
6203: Fix loophole allowing locators in bogus manifests to be accepted
Merge branch '6203-locator-regexp' refs #6203 refs #6277
Use default word wrap (no mid-word line breaks). refs #6057
Update arvados in bundle. refs #6203 refs #6277
6277: Catch whitespace errors, and "." and ".." in paths. Rename valid? to validate!.
6222: Precompile all regexps. Remove wasted effort in GetBlockHandler.
Merge branch '6222-precompile-regexps' refs #6222
Do not blow up if HOME is not set. refs #2256
6272: Add blank line to indicate index response EOF. Error out of data manager if not received.
Merge branch '6272-index-eof' closes #6272
6277: Simplify manifest-building loop, fix up truncation tests.
6218: Refactor profiler as a decorator
6440: Display AJAX errors as text, not HTML. closes #6440
6377: Create user-specified tmpdirs if necessary. refs #6377
6377: Don't try to uninstall arvados* gems if they are not installed.
5677: Edit structure.sql after db:structure:dump: avoid warnings when loading, and editwars when committing.
Merge branch '5677-plpgsql-warning' closes #5677
6222: Add memory stats to status.json.
6377: Missing application.yml is not an error.
6377: Support loading omniauth config from application.yml.
6377: Use fork of themes_for_rails with fixed gemspec.
6377: Fix reversed condition in "site application.yml is optional" fix.
6377: Avoid "incompatible encoding" errors by using blank template0 instead of site-specific template1.
6377: Update ruby to 2.1.6. Remove bundler version pin (not needed since ca1d997).
6377: Fix crash if no asset pipeline.
6377: Update docs to use application.yml instead of omniauth.rb for SSO config.
Fix typo. No issue #
Merge branch '6377-dev-startup' refs #6377
6279: Move webshell action into VirtualMachinesController, tweak formatting & config.
6218: Generate profiling data for a few arvados.collection.Collection scenarios.
Merge branch '6218-python-sdk-perf' closes #6218
Merge branch '6222-memory-status' refs #6222
6279: Move keyboard assets to /public/webshell/
6279: Do not set content-length headers in XHR.
6279: Do not retry a request that "timed out" in less than 2 seconds.
6279: Provide feedback that authentication token has been sent.
6279: Hide "reconnect" button because it doesn't auto-login.
Merge branch '6279-web-shell-client' closes #6279
6610: Include all user->VM login permission, even for users without SSH keys.
6362: Change config precedence order.
Merge branch '6610-login-without-ssh-key' closes #6610
6362: Update bootstrap gem.
6362: Allow config/application.yml to override environments/*.rb in apiserver, too.
5898: Wait for servers to start *listening* before running integration tests against them.
Merge branch '5898-listen-race' closes #5898
5898: Use lsof instead of fuser -- fuser 1234/tcp just hangs on some systems.
Merge branch '5898-listen-race' refs #5898
Merge branch '6362-asset-config' refs #6362
Revert "6362: Update bootstrap gem."
Revert "6362: Change config precedence order."
Revert "Revert "6362: Change config precedence order.""
Revert "6362: Allow config/application.yml to override environments/*.rb in apiserver, too."
6438: Drop wget dependency. Update ./configure flags for Ruby 2.1.6.
6438: Bring back package dependencies for "build Ruby from source" option.
6438: Add some missing steps to rvm recipe. Install rvm as root.
6438: Add specific commands for nginx16 and postgresql92 Software Collections.
6438: Add missing "apt-get update". Give a command to add the package repo.
6438: Use urandom instead of md5(hostname) for UUID. Avoid using Ruby at this stage.
6438: Merge debian and ubuntu sections. The same recipe works for both.
6438: Use uuid_prefix (more) consistently.
6438: Add missing steps to SSO installation.
6438: Back out dependency on lsb_release command.
6438: Make puma run script relocatable, set ARVADOS_WEBSOCKETS.
Merge branch '6438-multiplatform-install-guide-TC' refs #6499
6663: Fix using default owner_uuid in repositories#create. refs #6652
6663: Fix "duplicate public_key" test: OK if the key being updated has the same public_key.
6663: Fix repository permissions.
6592: 4534: Re-enable integration tests, now that run-tests.sh brings up an API server.
6592: Do not overload $Job temporarily.
6592: Exit RETRY_UNLOCKED if clean-work-dirs phase fails.
6592: Add crunch-job integration tests.
6663: Add explanatory comments and fix some unhelpful variable names.
6663: Admins and repository owners can_manage, not just can_write
6592: Build and install the Perl SDK so sdk/cli tests can use it.
6592: List runtime dependencies for Perl SDK.
Do not mess with PERLLIB in crunch-job tests; let run-tests.sh do that.
Merge branch '6663-bugfixes' refs #6663
Merge branch '6592-retry-if-cleanupfail' closes #6592
Merge branch '6592-test-perl' refs #6592
Merge branch 'crunch-dispatch-docker-bin-env' of https://github.com/wtsi-hgi/arvados into 6838-docker-path
6838: Document CRUNCH_JOB_DOCKER_BIN in example run script. Sync up with new default in sdk/bin/crunch-job.
Merge branch 'git-commit-debug' of https://github.com/wtsi-hgi/arvados
Whitespace
6663: Document git setup.
6838: If env var override is not set, let crunch-job use its default.
6598: Fix misreported "running tasks" stat.
Merge branch '6598-crunch-progress-stats' closes #6598
6663: Use a more obviously fake token.
Merge branch 'master' into 6663-git-server
6663: Update default git_repositories_dir to match install guide.
6663: Fix wrong bash prompt in example transcript.
6663: Clarify default vs. example git_repositories_dir.
6663: Clarify port 22 goes to sshd.
6663: Clarify "gitolite installed in X" vs. "hosted repos stored in X/repositories".
6663: gitolite, not arvados-git-sync.rb, creates empty repos.
6663: Fix API config example for HTTPS, add corresponding SSH example.
6663: Formatting fixes.
6663: Update nginx config to look more like API server's.
6663: Add "restart nginx" instructions.
6663: Split "install runit" into debian and red hat variants.
6663: More explicit gitolite version advice, and link to home page.
Merge branch '6838-docker-path' closes #6838
6663: Remove unneeded nginx config, clarify where config should be added.
6663: Add non-runit-dependent daemon startup script.
6663: Finish editing sentence.
6663: Fix config/application.yml -> application.yml.
6157: Explain how choice of hostnames relates to slurm and arvados.
Merge branch '6157-worker-hostnames' closes #6157
6663: Add back nginx configs to match tested values.
Merge branch '6663-git-server' refs #6663
Do not split partition config across lines. No issue #
6844: Return valid locator (hash+size) if 0 copies requested.
6844: Skip uselessly writing unsigned manifest to Keep.
6844: Always write a valid \n-terminated manifest. Confine exception-catching to expected exceptions.
6844: Do not use inspect.getcallargs: it leaks memory. Passing num_retries as a positional argument is no longer supported.
6844: Clear inserted data/logs after websocket tests.
6844: Clean up websocket tests, fix possible deadlock, reduce unnecessary sleep().
Merge branch '6844-py-mem-leak' closes #6844
6934: Add arvados_pam package.
6934: Replace custom config with yaml. Simplify code path.
6934: Improve and test logging
6934: Load config from pam conf instead of yaml. Add docker/integration tests and example configs.
6260: Fix races in keepstore tests. Expose WorkQueue in-progress/queued stats.
6260: gofmt
6260: Expose queue sizes in /status.json. Fix sleep/race in trash_worker_test.
6260: Pull entire status object out of WorkQueue atomically.
6827: Test logging handler with short token.
5824: Move request auth code into an SDK package. Support more ways of passing tokens.
5824: Move HTTP server code to SDK.
5824: Move spying ResponseWriter to SDK.
5824: gofmt
5824: Move quoted-logging function to SDK.
5824: Move client pool to SDK.
Shut down API server after suite (noticed during 5824, otherwise no issue #)
5824: Add Get() method to arvadosclient.
5824: Use fmt "%+q" instead of custom escaping.
5824: Do not JSON-encode string params. Fixup comments for godoc.
5824: Add UUID and PDH matchers.
5824: Fix up comments for godoc.
5824: Doc/comment fixes, add test for "Get empty UUID"
5824: Document ClientPool.
5824: Add test for nonexistent resource type.
6260: Improve Gate explanation.
6260: Rename ReportDone to DoneItem.
6260: Add explanatory comment about testing /status.json.
Merge branch '6260-work-queue-status' refs #6260
5824: Return InvalidArgument (not 404) for poor argument choices that do not involve real HTTP failures.
5824: Add tests for UUIDMatch and PDHMatch.
5824: Test invalid argument vs. 404.
6934: Package libpam-arvados using Python recipe. Add sdk/pam to run-tests.
6934: Add packaging instructions
6934: Put wrapper in /lib/security/ instead of writing dist-packages path in config file. Make integration test work.
6934: Split module
6934: Add pam_sm tests.
5824: Merge branch 'master' into 5824-go-sdk
Merge branch '5824-go-sdk' refs #5824
6934: Merge branch 'master' into 6934-pam
6934: Fix Dockerfile instructions. Use default test port. Use arvados/foo tag convention.
6934: Add libpam-python dependency on all targets.
6934: Move integration tests into separate dir.
6934: Build libpam-arvados only for targets that have libpam-python.
6934: Recommend system-log-daemon metapackage, not rsyslog specifically.
Merge branch '6934-pam' refs #6934
Merge branch '6934-pam' refs #6934
6638: Dry up debug printfs.
6638: Add python sdk package dependencies.
6638: Add package deps for llfuse and pycurl backports
6638: 6934: Fix up shellinabox example and add to package.
6638: Read package dependencies from fpm-info.sh in source tree.
6638: Build current version of arvados-src, not github master.
6638: Add postgres libraries to build containers, so bundler can build the pg gem.
6638: Fix rvm usage error. Straighten out "need new gem" and "need to build new package from gem" logic.
6934: Add sdk/pam to run_upload_packages.py refs #6934
6638: Use (git clone; git checkout) -- git clone --branch doesn't work with commit hashes.
Merge branch '6638-backport-deps' refs #6638
Merge branch '6638-backport-deps' refs #6638
Use relative data_files paths so package is installable in a virtualenv. refs #6934
Accept libcurl4-openssl-dev as an alternative to libcurl4-gnutls-dev dependency. refs #6638
6640: dup all_projects when reusing it for find_objects_for_index, to
6640: Replace paging test with all_projects bugfix test.
6640: Fix whitespace, remove excess end tag.
6263: Pass through environment variables needed by gitolite-shell.
6263: Set flag to skip gitolite permission checks, and set SERVER_ADDR so gitolite doesn't complain.
6263: Split git-specific stuff into git_handler.go, add tests for new env vars.
6781: Add arvados-login-sync.
Merge branch '6781-arvados-login-sync' refs #6781
6781: Use env vars instead of yaml config file.
6781: Add test cases and packaging support.
6781: Add arvados-login-sync section to shell server install page.
6781: Do not try /get_all_logins (the new /logins route is now 2 years old).
6781: Clarify "logins" vs. "get_all_logins" APIs by moving common code into render_logins_for.
Merge branch '6781-arvados-login-sync' refs #6781
6781: Pass tests regardless of whether an admin group exists on the test host. refs #6871
6263: Update doc.go: mention gitolite-shell.
6263: gofmt
Merge branch '6263-arv-gitolite' refs #6263
6859: Fix up module usage, mock backtick to test arv-put error handling.
6859: Find locator-like strings even if not space-delimited.
Merge branch '6859-fix-invalid-manifests' refs #6859
6263: Generate Go coverage reports in $WORKSPACE/tmp/coverage-*
Write "all interfaces, any port" as ":0" (not ":") for compatibility with Go 1.5. refs #6997
6997: Avoid unnecessarily inefficient dirent sorting (and a mismatch between index and store/fetch logic) by doing a custom traversal.
6997: Add test case for (*UnixVolume)IndexTo().
6997: Fix fencepost error causing race condition in test.
Merge branch '6997-keepstore-mem' refs #6997
6953: Add arvados-docker-cleaner instructions to compute node setup page.
Merge branch '6953-docker-image-cleaner' closes #6953
6263: Check for gitolite in sanity checks.
6263: Skip re-installing virtualenv if it is already set up. Upgrade pip along with setuptools.
6263: Add gitolite integration test. Fix existing tests to use bare repos on server side.
6263: Rearrange test suites to share code better.
5824: Enable blob signing in integration tests, and send keepstore logs to files.
Merge branch 'wtsi-hgi-fix/7205-arv-edit-tty-check'
Merge branch '6263-gitolite-test' refs #6263
Merge branch '6263-gitolite-test' refs #6263
Merge branch '6263-go-coverage' refs #6263
7121: Add test case to demonstrate deadlock.
7121: Replace Get(loc,true) with CompareAndTouch(). Add Compare method to Volume, UnixVolume, MockVolume.
7121: Return DiskHashError instead of CollisionError from Compare() where appropriate.
7179: Add TestableVolume interface, make UnixVolume tests a bit more reusable.
7179: Start a set of generic volume tests.
7121: Test mutex usage with a mock instead of time.Sleep.
7121: 7218: Get the serialize lock before opening a file.
7121: Update comment to match new argument name.
7121: Rename mutex_test.go -> mock_mutex_for_test.go
7121: Clarify expected behavior of Compare() in Volume interface.
7121: Log all errors (except the everyday "not found") encountered during CompareAndTouch().
7121: Fixup log messages (remove excess \n, show which volume had a Get() error).
7121: Add comment about why buffers must be returned.
7121: Add collisionOrCorrupt->DiskHashError test cases where the mismatched data is in buf1.
7179: Add Volume interface specs.
Merge branch '7121-fix-deadlock' closes #7121
7179: Improve description of how races must be handled.
6260: Just use the token loaded by the SDK (ARVADOS_API_TOKEN),
6260: Remove extra hyphen from command line args.
6260: Fix up error messages.
6260: Remove test for impossible condition.
6260: Simplify verifyBlocks logic.
6260: Fix up /status.json checks.
7179: Improve comments.
Merge branch 'master' into 7179-test-mocks
Merge branch '7179-test-mocks' refs #7179
6260: Remove unnecessary Sleep()
6260: Reduced-replication test requires replication>=desired, not replication==len(keepServers).
7311: Make recognizable temp dirs under a single base; allow override with --temp DIR
Merge branch 'wtsi-hgi-fix/7310-git-v2-http-500-error-matches' fixes #7310
7179: Tighten Put requirements when overwriting existing data.
7311: Do not forget to make temp dirs.
Merge branch '7311-temp-base' closes #7311
7329: Fix infinite loop in Compare when reading an empty file (whether
7329: Add generic tests for empty block handling.
7309: Push branches explicitly in tests instead of relying on `git config push.default`
Merge branch '7309-git-push-default' closes #7309
7329: Fix infinite loop in Compare when reading an empty file.
7241: Add AzureBlobVolume
7241: Accept command line flags for Azure blob volumes.
7241: Add Put() and Check()
7241: Add Delete, IndexTo, Touch, and Mtime.
7241: Stub Azure API calls
7241: Add -azure-storage-replication flag.
7241: Do not bother trying to add a volume if credential args are missing/empty.
7241: Use new CreateBlockBlobFromReader and SetBlobMetadata APIs for Put and Touch.
7241: Use If-Match header to address Delete/Put races.
Merge branch 'master' into 7241-azure-blob-volume
Merge branch '7241-azure-blob-volume' closes #7241
7214: Fix "X-Keep-Replicas-Stored: 0" header when block is already present. refs #7214
Fix incorrect line numbers on compile errors reported by "go test" in coverage mode. No issue #
7254: Do not forget -replication arg when failing to load resume state.
7254: Test that replication arg is passed through to KeepClient.put()
Merge branch '7254-dont-lose-replication-arg' closes #7254
6967: Treat blob_signing_key like a secret in `rake config:check`.
6967: Move source_version detection code from config yaml to lib/app_version.rb.
6967: Move source_version detection code from config yaml to lib/app_version.rb.
6967: Use git status --porcelain to isolate from user config
6967: More helpful comment & assertion failure message
Merge branch '6967-yaml-format' closes #6967
7491: Fix error handling/reporting in keepclient/GET
7491: Ensure status channel stays open until all upload workers finish.
6967: Update test to match improved code.
7167: Tidy up errors. Remove extra comment copy.
7167: Replace duplicate tests with PermissionSecret tests
7167: Fix up comments
7167: Update tests and comments to new error vars.
7167: Deobfuscate variable names
Merge branch '7167-blob-sign-sdk' refs #7167
Merge branch '7491-keepclient-bugs' refs #7491
7159: Work around CreateBlob race by polling for updates when a brand new blob is found empty.
7159: Test race deadline
7159: Exclude new empty blocks from index.
7159: Log when waiting for get/put races
7159: Shorten race waits during generic tests
Merge branch '7159-empty-blob-race' refs #7159
Warn about unhandled case if broken node has no ping time. refs #7286
7167: Propagate read errors to caller. Fixes failing TestTransferShortBuffer.
7159: Fix error handling when reading full size block. refs #7159
7159: Return benign os.ErrNotExist error from Compare to avoid excessive logs. refs #7159
7159: Omit non-Keep blobs from index
7159: Address golint complaints
Merge branch '7159-clean-index' refs #7159 refs #7168
Merge branch '7167-propagate-error' refs #7167
Merge branch 'master' into wtsi-hgi-feature/arv-view
7167: Fix up progress/ETA logging.
7167: Rename conf flags to -src and -dst to match arv-copy. Always add .conf for short form.
7167: Always pass --num-keep-servers to python wrapper. Update comments.
7173: Add jessie to supported distro list.
Merge branch '7173-jessie' refs #7173
7173: Update bundle.
7204: Tweak usage message, add a .shellescape, fix error-reporting error
Merge branch 'wtsi-hgi-feature/arv-view'
7159: Add keep-exercise
Merge branch '7159-keepexercise' refs #7159
5824: Add keepdl.
5824: Assign MIME type by file extension. closes #6327
5824: Add install doc
5824: Add doc.go
5824: add (*KeepClient)CollectionFileReader()
6358: Fix rendezvous probe order on Put.
6358: Test partial ordering with multiple writer threads.
5824: Support vhost-based collection lookups.
5824: Use vhosts in curl integration tests. Add large file test.
5824: Add Content-Length header.
5824: Fix up error checking and early-close behavior inCollectionFileReader.
5824: Update keepproxy usage.
5824: Clarify difference between keepproxy and keepstore (bandwidth and convenience -- not security).
5824: Modernize install page, cf. other services.
5824: Fix up support for PDH in vhostname.
5824: Comment to explain "authorizer" test helpers.
5824: Handle various combinations of c= and t= more consistently. Use vhosts in integration tests.
5824: Accept anonymous tokens on command line.
5824: Fail at startup if ARVADOS_API_HOST is not set.
5824: Log X-Forwarded-For header value if provided.
5824: Fix up DNS docs.
5824: Add read-error and lots-of-blocks tests.
5824: Implement "trust all content" mode.
5824: Add -attachment-only-host feature.
5824: gofmt
5824: Add option to redirect Workbench downloads to a keep-web service
5824: Use keep-web in Workbench integration tests
5824: Update bundle
Merge branch '7173-jessie'
5824: Refresh keepproxy services list on SIGHUP. Update Workbench upload test to expect success.
5824: Simplify keepproxy test setup.
5824: Log actual client IP address (along with X-Forwarded-For header, if any).
5824: Fixup nginx and workbench configs
5824: Leave "install keep-web" page out of install guide TOC for now.
Merge branch '6358-put-rendezvous' closes #6358
6358: Fix race opportunity in ThreadLimiter.
6358: Fix probe order test logic.
7160: Add Azure Storage config page, update keepstore help text, add run script.
7160: Clarify exampleAccountName -> exampleStorageAccountName
Merge branch '7160-azure-blob-doc' closes #7160
Fix typo in example config file
5824: Fix wrong title.
5824: Clarify docs.
5824: Rename keepdl to keep-web.
5824: Rename cookie to arvados_api_token.
5824: Rename conventional dl.* to collections.*
5824: Clarify docs.
5824: Add test for file in subdir.
5824: Add tests.
5824: Add comments and fix variable names, cf. golint.
5824: Add some clarifying comments and golint/vet/fmt fixes.
5824: Merge branch 'master' into 5824-keep-web
5824: Update arvadostest usage.
5824: Add -anonymous-token flag.
Sync Gemfile.lock to current Gemfile.
5824: Merge branch 'master' into 5824-keep-web
Merge branch '5824-keep-web' into 5824-keep-web-workbench
5824: Enable anonymous use of keep-web in integration tests.
7668: Move node stats from info to properties in fixtures.
5824: Send keepproxy and keepstore logs to stderr via fifo, just like nginx_access_log.
5824: Give an anonymous token to keepproxy, not an admin token.
5824: Mention -anonymous-token in godoc. Sync usage messages.
7444: Delete containers as soon as they stop.
7444: Note automatic removal of stopped containers, and how to disable.
7444: Test deletion error handling.
7444: Clean stopped containers at startup.
7444: Do not remove docker containers with docker --rm; let dockercleaner do it.
7444: Set docker container name to {taskUUID}-{attemptNum}.
7444: Rename kwarg remove_stopped_containers -> remove_containers_onexit
5824: Clarity edits in usage docs.
5824: Use session cookie instead of persistent.
5824: Turn off debug printfs unless enabled by calling program.
5824: Avoid sending empty slices through toRead chan. Fixes race in test case.
5824: Merge branch 'master' into 5824-keep-web
Merge branch '5824-keep-web' into 5824-keep-web-workbench
5824: Add services/keep-web
5824: Start keep-web service for Workbench integration tests.
Merge branch '5824-keep-web'
Merge branch '5824-keep-web'
Merge branch '7444-dockercleaner-containers' closes #7444
7722: Build keep-rsync package. Deduplicate go->fpm recipe.
5824: Rename -address to -listen
5824: Use ARVADOS_API_TOKEN=foo + -allow-anonymous instead of -anonymous-token=foo.
7724: Use a scoped token in data manager tests.
Merge branch '5824-keep-web' refs #5824
Merge branch '7724-scoped-token' closes #7724
7724: Use datamanager token in keepproxy index test. refs #7724
7724: Use datamanager token in keep-rsync tests. refs #7724
Add libwww-perl and python mock dependencies, so tests can run on ubuntu 14.04
5824: Merge branch 'master' into 5824-keep-web-workbench
5824: Merge branch 'master' into 5824-keep-web-workbench
5824: Sync test suite to new keep-web argument names.
5824: Use fifo2stderr for arv-git-httpd and keep-web logs, too.
5824: Fix Keep server shutdown, check errors, simplify stderr redirection.
5824: Fix server shutdown code.
5824: Move "periodically refresh Keep services" func from keepproxy to SDK.
5824: Fixup new keepproxy tests to use simplified test setup.
5824: Fix disposition=attachment handling.
5824: Support partial content with Range header (only if start==0).
5824: Propagate non-token parts of query string (notably ?attachment=disposition) when redirecting.
5824: Add anonymous-404 and download-by-pdh tests.
5824: Preserve query in keep_web_url template. Warn when redirecting preview to a single-origin keep_web_url.
Merge branch '7722-build-keep-rsync' closes #7722
5824: Support configuration with a download-only host.
Use a different port number for each test case. No issue #
5538: Test that POST method is not retried.
5538: Update comments to match new tests.
Merge branch '5538-test-post-retry' refs #5538
5824: Update/clarify docs and comments.
5824: Merge branch 'master' into 5824-keep-web-workbench
Merge branch '5824-keep-web-workbench' refs #5824
Merge branch '5824-keep-web-workbench' refs #5824
5824: Fix -attachment-only-host test config. Test more preview/download variants.
5824: Fix path and query escapes.
5824: Fix clear-download-dir helper.
Merge branch '5824-keep-web-workbench' closes #5824
7711: Store cloud node size id and price in properties of Arvados node record.
Merge branch '7711-record-node-price' closes #7711
7490: Quote strings in error messages, fixup error matching in tests.
7751: 7654: Rename bool to _closed to avoid conflicting with superclass closed() method.
7676: Prefer cheaper nodes when allocating nodes to jobs.
7676: Move crunch dispatch code into lib.
7676: Test choosing cheaper nodes, and basic crunch-dispatch startup/locking.
7676: Improve comments.
Merge branch '7676-dispatch-cheaper-nodes' closes #7676
7751: Add --mount-tmp option.
7751: Move code from arv-mount executable to module.
7751: Add tests for --mount-tmp option.
7751: Test mount arguments.
7751: Return an empty array from InodeCache.find() instead of None.
7751: Update explanation of special .arvados#collection file. Fix wayward use of "Keep locator".
7751: Suppress help messages when testing usage errors.
7751: Lowercase method name for PEP-0008.
7751: Fix shell wrapper code path.
7751: Fix stack trace on shutdown caused by race between destroy() and llfuse unmount.
7751: Refactor TmpCollectionDirectory: generate .arvados#collection less often.
7751: Fix race by telling fuse not to cache the .arvados#collection dirent.
Merge branch '7751-mount-tmp' refs #7751
5824: Move run_test_server diag messages from stdout to stderr.
7751: Clean up KEEP_LOCAL_STORE env var after tests.
Move remaining arv-run-pipeline-instance usage info from comments to --help message.
Force pbr<1.7.0 when installing mock.
Update capybara in workbench bundle.
Warn, but do not error out, if a collection has conflicting block sizes.
7851: Choose available ports for selenium and poltergeist.
7851: Use a random display (:100-:499 = port 6100-6499) for Xvfb, instead of always :99.
Fix crash on removing nonexistent directory.
Merge branch '7851-available-port' refs #7851
Clear job queue before testing crunch-dispatch start/stop.
Mock dispatch instead of relying on rollback in child proc.
7697: Avoid reusing long-idle HTTP connections. Avoid retrying non-idempotent operations.
Merge branch '7697-socket-retry' closes #7697
7710: Add keep_services#accessible test.
7868: Allow admin to add arguments to "docker run" commands.
Merge branch '7868-docker-run-args' closes #7868
7751: Set up an arv-mount scratch directory for each task, and put its path in TASK_KEEPMOUNT_TMP.
7751: Add convenience class for staging task output in $TASK_KEEPMOUNT_TMP.
Merge branch '7751-crunch-fuse-output' closes #7751
7888: Log how much time was spent before/after sending response headers.
Merge branch '7888-log-req-phase-times' refs #7888
Fix manifest stats leaking between runs via globals. No issue #
7937: Fix ignored error from GetKeepServersAndSummarize.
Merge branch '7937-ignored-error' refs #7937
7937: Update test case.
7901: Add crunchstat-summary
7393: Add S3 volume type.
7393: Quiet excessive debug printfs.
7393: Add -uuid and -url options, fix memory sharing in -vary-request.
Merge branch '7393-s3-volume' closes #7393
7901: Add crunchstat-summary program.
Merge branch '7901-crunchstat-summary' refs #7901
Merge branch '7901-crunchstat-summary' refs #7901
Increase default Python SDK timeout to 256s.
7888: Option to use multiple concurrent range requests when fetching from Azure.
7888: Fix whitespace / gofmt
Merge branch '7888-azure-read-mux' refs #7888
7942: Skip unreliable test. refs #7942
7901: Add job stats, elapsed time, summed user+sys and tx+rx, and some human-readable highlights.
Merge branch '7901-human-summary' closes #7901
Log request body size. No issue #
7884: Serve simple cross-origin AJAX POST requests without redirecting.
7884: Detect when a "download log" response has a JSON-encoded redirect
7884: Clarify "credentials" comment.
Merge branch '7884-ajax-log-redirect' closes #7884
7965: Fail orphaned jobs still marked "running" after reboot.
Merge branch '7965-fail-abandoned-jobs' closes #7965
Increase timeout for bringing up test server.
8082: Fix column markup (always use up "log" grid column, always use up all 12 grid columns).
Merge branch '8082-col-markup' refs #8082
7942: Skip unreliable test. refs #7942
8123: Add --pipeline-instance mode: generate a report for each finished component.
8123: Recommend more economical values for resource_constraints.
8123: Aim 5% below GiB memory size boundaries.
8123: Add option (--format html) to generate canvasjs charts.
8123: Generate multiple sets of charts when data source is a pipeline instance.
8123: Add option to include stats from child jobs.
8123: Fix crunchstat formatting error and resulting crunchstat-summary crash.
8189: Replace MAX_LIMIT const with max_items_per_response config.
8189: Use large page size only when fetching all pages.
Merge branch '8189-handle-large-collections-better' refs #8189
Fix cli "update" example. No issue #
7996: Add keep-web to install TOC and server/cert lists.
Merge branch '7996-keep-web-doc' closes #7996
8177: Expand trust_all_content comments.
8177: Offer -trust-all-content option for no-wildcard installs.
Merge branch '8177-keep-web-trust-all-content-flag' closes #8177
8123: Ignore crunchstat "caught signal:" log.
6833: Add (most of) a test case for token expiry.
8281: Fix arv-mount ignoring --retries argument when writing file data.
8281: Fix KeepClient retry bugs.
8281: Limit # write threads to #copies remaining, not #copies total.
Merge branch '8281-arv-mount-retry' closes #8281
7263: Avoid getting stuck processing stderr for one task for a long time.
8123: Include chartjs.js in package.
8284: Fix confusion between %proc and %jobstep.
8123: Update test dependencies.
8123: Explain mysterious memory constraint logic.
8123: Change --include-child-jobs to --skip-child-jobs (default False).
8123: Use -v,-vv instead of --verbose,--debug.
8123: Fix type check to accommodate unicode.
8123: Fix accidental old-style class.
8123: Explain existing_constraints and use a proper instance variable.
8123: Escape HTML chars in page title.
Merge branch '8123-crunchstat-graphs' closes #8123
8123: Install chartjs.js asset file.
8288: Give fusermount -u a chance to work before resorting to operations.destroy().
8288: Add test case for --exec mode.
8288: Do not call operations.destroy() as a last resort, just abandon the llfuse thread.
Merge branch '8288-arv-mount-deadlock' refs #8288
8288: Add timeout option to close() method of event clients.
Merge branch '8288-poll-client-close-timeout' refs #8288
Recognize another way slurm tells us about node failures.
8285: Test that arvados.events.subscribe() is called only when needed.
Merge branch '8285-fuse-subscribe-websockets' closes #8285
8289: Do not add fallback orders if client already specified an unambiguous order.
8289: Strip redundant orders, even when provided explicitly by client.
Merge branch '8289-no-extra-orders' closes #8289
8341: Get job log from logs API if the log has not been written to Keep yet.
8341: Use a worker thread to get page N+1 of logs while parsing page N.
8341: Move reader classes to reader.py.
8341: Use a Queue of lines and one thread, instead of a succession of threads and a deque of buffers.
Merge branch '8341-live-crunchstat-summary' refs #8341
Fix nodemanager test race. No issue #
7263: Avoid getting stuck processing stderr for one task for a long time.
Merge branch '7263-better-busy-behavior' refs #7263
Emit log when installing docker image.
Merge branch '8284-fix-slurm-queue-timestamp-check' closes #8284
8341: Use "time since job start", not "time since task start", as X axis.
8341: Do not round up Y axis to even numbers, just use max series value.
8341: Fix up debug labels. Avoid deadlock after exceptions in thread.
8341: Include Keep network activity in net stats.
8341: In pipeline mode, process all jobs concurrently.
8341: Retrieve only the log attributes that actually get used.
8341: Update test results.
8341: Fall back to live logs if log collection is saved but missing.
Merge branch '8341-crunchstat-job-time-axis'
Process live logs for unfinished jobs in pipeline mode, too.
8178: Stop accepting zeroed data, now that the s3test bug is fixed.
Merge branch '8178-trash-interface-generic-volume-test' closes #8178
8409: Use 80% utilization as keep_cache_mb_per_task reporting threshold. refs #8409
Merge branch '8400-additional-gitignore' of https://github.com/wtsi-hgi/arvados closes #8400
Merge branch '8485-datamanager-identical-modifiedat' of https://github.com/wtsi-hgi/arvados closes #8485
8484: Fix fixtures with modified_at=NULL
Merge branch '8484-sanity-check-collection-count' closes #8484
8345: Add libselinux-dev build-dep missed by awk script, skip "recommended" packages
8099: When invoking setup tasks via srun, check slurm queue and propagate stderr to logs.
Fix dependencies after installing built package. refs #8345
Install bundler gem in all build containers.
Merge branch 'master' into 8345-llfuse-unpin
Workaround for https://bugs.python.org/issue7980. No issue #
8099: 7263: Merge branch 'hgi/7263-even-better-busy-behavior' of github.com:wtsi-hgi/arvados into 8099-babysit-all-srun
Merge branch 'master' into 8345-llfuse-unpin
8345: Use a separate vendor/cache for each OS target. See dev.arvados.org/issues/8345#note-11
8343: Allow overriding -cgroup-root=/sys/fs/cgroup via crunch-dispatch run script.
Merge branch '8343-cgroup-root' closes #8343
8099: Remove duplicated line
Merge branch 'master' into 8099-babysit-all-srun
Merge branch '8099-babysit-all-srun' closes #8099
7263: Check new tasks after a task completes, even if others failed in the same reapchildren().
7263: Let arvados library do the json encoding.
Merge branch '7263-batch-task-lookup' closes #7263
8345: Build and install new fuse libs before building llfuse.
8345: Add build-essential to ubuntu1204 package-building image
8345: Build contextlib2 backport, needed by python-llfuse on debian7.
8345: Fix up log messages to show which distro/package is being built/tested.
8345: Merge branch 'master' into 8345-llfuse-unpin
8345: Uninstall old llfuse from reused virtualenv, if necessary.
Remove duplicate environment variable.
8345: 8491: Downgrade to llfuse 0.41 until #8345 gets sorted out (but keep the fuse backports).
8345: Need "yum install -y" to reliably install rpm packages.
Merge branch '8345-llfuse-unpin' refs #8345
Add 'build/' from commit '555b039609a3c8700c27767c255fdfe00eb42063'
8491: move files from arvados-dev into their new places
8491: --retry applies to do_install too
8491: Remove git-commit.version before running api server tests
8491: 8222: Prohibit activesupport 4.2.6 so our gem can be installed with ruby <2.2.2
8491: Merge branch 'master' into 8491-import-build-tools
8079: Prevent users from looking up other users' tokens by UUID.
8079: Prevent users from changing their own token UUIDs.
8079: Tidy up and document current_api_client_is_trusted.
8079: Allow where(api_token: foo) and disallow where(api_token: ["contains", "f"])
Merge branch '8079-lookup-token-uuid' closes #8079
Fix ineffective test case
8491: Add default target "make help"
Merge branch '8491-import-build-tools' refs #8491
Fix backports dir search for PACKAGE="llfuse==0.41.1".
8554: Dedup Get() checks, add comments, fix up regexp.
8554: Un-pyramid code
8554: Report Walk errors and continue, instead of stopping silently.
gofmt. refs #8754
8345: Do not crash when update() called without an obj.
Merge branch '8747-fix-crash-on-update'
8767: Make offset work properly in ApiClientAuthorizationsController#index.
Merge branch '8767-items-available'
8556: Update method signature for current SDK version.
8800: Drop queue_position support.
8904: Avoid installing pip >= 8 in a Python 3.2 virtualenv.
8697: Relax version constraints so gem can be used in ruby187/ree projects.
8697: Move error messages from stdout to stderr.
8697: ruby 1.8.7 compatibility in arvados/collection.
8697: Fix Locator.parse() (was failing on locators with hints).
8936: Warn about disruptive effect of modifying blob_signature_ttl and blob_signing_key.
9005: Workaround: Close idle connections aggressively.
Change Check to Assert to avoid crash after failure. No issue #
Merge branch '9004-close-keep-connections'
9009: Fix missing Close() in collectionreader.
Merge branch '9009-keep-web-close-conns'
6833: Fix excessive debug logging in TokenExpiryTest and subsequent tests.
Merge branch '8697-ruby187-compat'
8831: Add crunchrunner to compute node dependencies.
8831: Add crunchrunner to shell node dependencies.
Merge branch '8831-crunchrunner-doc'
9017: Add run-tests.sh --short flag to skip (some) slow tests.
Merge branch '9017-skip-slow-tests'
9066: Add keepstore -max-requests argument.
Merge branch '9066-max-requests'
Log a banner at the top of each test.
Read resource object from a file, e.g., arv collection create --collection /tmp/foo.json
8653: Fix whitespace.
8653: Turn off debug messages / verbose logging in test suite.
8653: DRY testing code.
8653: Add arvados-cwl-runner --create-template flag
Merge branch '8653-cwl-to-template'
Use "grep -xF ... >/dev/null" instead of "grep -qxF ..."
Merge branch '9119-oj-load-strict'
Relax arvados-cli gem dependency version constraints in order to be
9068: Drop PUT requests if the client disconnects before we get a buffer.
9068: Move buffer allocation from volumes to GetBlockHandler.
9068: Do not use coverage tools when using non-default test flags ({gostuff}_test=...)
9068: Fix inconsistent receiver names.
Merge branch '9068-drop-abandoned-conns'
Fix compatibility with latest azure-sdk-for-go.
Update API server and Workbench bundles to latest arvados gems.
9017: Skip some slow API server tests in --short mode.
Merge branch '9017-apiserver-short-tests'
8128: De-dup container unit tests
8128: Add Locked state to Container model.
8128: Update crunch-dispatch-slurm to use new Locked state.
8128: Update crunch-dispatch-local to use new Locked state.
8128: Add runtime tokens for containers, and locks for multiple dispatchers
8128: Add arvados.v1.api_client_authorizations.current
8128: Use row lock during Container update, add comments.
8128: Fix flaky test: pipe the "echo UUID" script to sh, not to "echo UUID".
8128: Fix test race.
Merge branch '8128-crunch2-auth-api'
9188: Update SetBlobMetadata func signature.
Accept auth tokens with uppercase letters.
Merge branch 'wtsi-hgi-9231-rename-redunancy-to-replication-desired'
9272: Fix some race conditions in flaky tests.
Merge branch '9272-test-races'
9162: Add keep-balance
9162: Add replication level histogram
Merge branch '9162-keep-balance'
9343: Do not check env vars when setting up Keep client for pull requests.
Merge branch '9343-no-env-vars'
9272: Get container auth instead of passing the dispatcher token into the container.
9272: Pass container auth info into container if requested.
9272: Fix up state transitions:
9272: Simplify json decoding with Unmarshal.
9272: Skip slow test when running -short tests.
Merge branch '9272-use-container-auth'
9278: Set expires_at=now if a client sets it to a time in the past.
9278: Expose expires_at in API response.
9278: Ensure locator signatures expire no later than expires_at.
Merge branch '9278-expiring-collections'
9374: Consolidate various Container structs as arvados.Container.
9374: Add arvados.APIClientAuthorization.
9374: Propagate API transaction error details from response body to caller.
9374: Use arvados.Collection instead of own CollectionRecord.
9374: Fix uses of "this" as receiver name.
9374: Remove unused receivers.
Merge branch '9374-go-sdk'
9395: Add CollectionBatchSize config to keep-balance.
9395: Add CollectionBuffers config to keep-balance.
9395: Explain CollectionBatchSize and CollectionBuffers in -help message.
9395: Fix encoding of integers in query params.
Merge branch '9395-keep-balance-page-size'
9486: Use API to filter nodes by last_ping_at.
Merge branch '9486-filter-nodes'
8470: Resolve runtime_constraints ranges to numbers when satisfying a Container Request.
8470: Resolve mounts to PDH.
8470: Clean up ContainerRequest tests.
8470: Resolve docker image hash or tag to collection PDH when creating a Container.
Merge branch '8470-resolve-container-req'
9491: Put*() return number of replicas actually stored, even if we ended up storing more than requested.
9491: Fix keepproxy support for X-Keep-Desired-Replicas header.
Merge branch '9491-desired-replicas'
9518: Fix username conflict resolution.
Merge branch '9518-username-conflict'
9513: Do not set response Content-Length to -1 when returning 411 Length Required.
Merge branch '9513-keepproxy-content-length'
8016: Split crunchstat into a module and a commmand line tool.
8016: Reduce logging noise by waiting for cgroup files to appear before polling.
8016: Fix up docs and trim API.
8016: Fix timestamp format (add missing nanoseconds), and use it in tests.
8016: Fix comment grammar and weird variable name.
8016: Fix some error checking in ThrottledLogger.
8016: Report container resource usage in "crunchstat" log.
4753: Select an available port for the passenger server when testing websockets.
Merge branch '4753-websocket-test-port'
9437: Report timestamps as nanoseconds since epoch in keepstore index.
9437: gofmt
9437: Accept 1-second timestamps from old keepstore servers.
9437: Clarify units for -blob-signature-ttl. Update keepstore -h message in install docs.
Merge branch '9437-keep-index-ns'
8016: Fix race in test case.
8016: Rename Poll to PollPeriod.
8016: Clarify comments/docs and cpuSample initialization.
Merge branch '8016-crunchrun-crunchstat'
8016: Rename Poll to PollPeriod. Amends c63c699aa9948f6a672536ba08e664471fb0d654.
Respect running_job_log_records_to_fetch in JobsController#logs.
9542: Avoid retrieving huge result sets from postgres.
Merge branch '9542-websocket-queries'
Bump cwltool dependency. Previously pinned version was broken.
Revert "Bump cwltool dependency. Previously pinned version was broken."
9528: Drop sbatch --parsable flag. It fails in older versions of slurm,
Merge branch '9528-slurm-parsable'
9456: Do not clear trash lists between runs when the list of keep services has not changed.
Merge branch '9456-less-clear-trash'
9567: Avoid ruamel.yaml >0.11.11. refs #9567
9567: Print log messages from keep et al. as they appear, even in jenkins where stdout is not a tty.
9567: Use longer timeouts for keep-web download tests.
Merge branch '9567-keep-web-session-test'
Merge branch 'fix-python-sdk-mac' of https://github.com/adambom/arvados
9613: Stop prepending ":" to profile keys.
Merge branch '9613-user-profile-string-keys'
9624: Fix format string.
9639: Avoid sharing rvm gemsets between concurrent builds.
9639: Delete all cached discovery docs, but do not delete the cache directory.
9639: Relax unnecessarily strict time limit.
9613: Dry up user profile tests.
9613: Ensure submitted values are saved, and displayed on the edit form after it reloads.
9613: Touch up "find required field" loop.
Merge branch '9613-test-profile-save'
9550: Allow overriding keep services discovery with ARVADOS_KEEP_SERVICES env var.
9550: Add service discovery override field to new SDK.
9550: Update comments.
9550: Ignore extra spaces in ARVADOS_KEEP_SERVICES, and ensure each entry is an absolute URI.
Merge branch '9550-keep-services-env'
8555: Implement trash with race-recovery for S3 volumes.
8555: Add log messages for time-parsing errors.
8555: Move checkRaceWindow out to a func, tweak comments
8555: Test various backend states. Update recent/X timestamp during Untrash.
8555: gofmt
8555: golint
8555: Fix EmptyTrash so it can clean up old races.
8555: De-obfuscate test case.
8555: Reformat test cases.
8555: Improve variable names and comments.
8555: Log statistics in EmptyTrash.
8555: Untrash to repair inconsistent state (trash/X without recentX).
8555: Test Get() after successful Untrash. Test Put+Mtime in all scenarios.
8555: gofmt
8555: Fix up comments.
8555: golint
8555: Fail Go tests if not gofmted.
Merge branch '8555-s3-trash'
9406: Support passing additional arguments from crunch-dispatch-slurm to crunch-run.
9406: Add -cgroup-parent-subsystem argument.
9595: Support "json" mount type.
9595: Fix hardcoded temp dir in test. Add checks for CleanupDirs().
Merge branch '9595-mount-json'
9406: Explain why there are two separate CgroupParent fields.
Merge branch '9406-cgroup-parent'
Fix potential race in ThrottledLogger flusher.
Simplify write flusher using a time.Ticker.
Fix race in crunchstat.
gofmt.
9706: Add example config file.
9745: Load API host and token from crunch-dispatch-slurm config file.
9745: Update example config.
9684: Style nitpicks.
Merge branch '9684-workflows'
9826: Deserialize symbols when loading server config YAML.
Merge branch '9826-config-symbols'
9818: Use lograge to log API requests in logstash format.
9818: Use lograge/logstash for Workbench request logs.
9818: Log params when small, first 1K of json-encoded params when big.
Merge branch '9818-logstash'
9773: Fix up find-or-create-job code.
9773: Add script_parameters_digest column to support faster job reuse.
9773: Use script_parameters_digest in reusable job query.
9773: Exclude script_parameters_digest from searchable columns -- it is not even visible to clients.
9773: If previous jobs disagree about outputs, but a new job is
Merge branch '9773-faster-find-or-create'
9706: If "foo.service" exists in the source dir for a Go program, package it so it installs to /lib/systemd/system/.
9706: Add postinst script to Go packages that have systemd unit files, so (on Debian) the service starts up without manual intervention.
9706: Stop systemd services when their packages are uninstalled.
9706: Add systemd unit file for crunch-dispatch-slurm.
9706: Notify systemd when crunch-dispatch-slurm is ready.
9706: 9745: Fix config struct to match example.
9706: Use ${} interpolation syntax consistently.
Merge branch '9706-package-systemd-files'
9709: Restore logging of manifest_text by default, add config option to omit it.
Merge branch '9705-crunch2-install-guide-wip'
9709: Test empty unlogged_attributes config.
9709: Tweak config comment.
Merge branch '9709-restore-manifest-logging'
9700: Remove "trashLifetime != 0 is not supported in S3" check, now that it is supported.
9799: Use regular (non-admin) user in websocket integration tests.
9799: Use regular (non-admin) user in Workbench websocket tests.
9799: Fix container fixture owner_uuid attributes to look like real-life containers.
9799: Add tests for container log permissions.
9799: Add tests for container permissions.
9799: Permit read access to containers via container requests. Move Log- and Container-specific permission queries to log.rb and container.rb.
9799: Update test: "spectator" user is now allowed to read log entries about itself.
9799: Dry up SQL statements using named bind parameters.
9799: Restore permission on Log entries via owner_uuid.
9799: Remove redundant test.
9799: Fix test order dependency.
9799: Clean up permission code and comments.
9799: Add missing container request, so container fixture is visible to test user.
9799: Use more realistic log entries in Workbench live-logging tests. Remove redundant test.
9799: Add forgotten serialized column.
9799: Fix show/hide "cancel container req" button: check ArvadosBase#editable?, and use CR priority instead of container priority.
9799: Move work unit tests from application_layout_test to work_units_test.
9799: Remove bogus bootstrap classes and fix html syntax.
9799: Fix up live_log_lines so it works for all work units.
9799: Fix bootstrap markup.
9799: Update tests: non-admin user can see container assigned to committed CR.
Make flaky test fail less often.
Add max_request_log_params_size config (was hardcoded to 1000).
9888: Move record-filtering code into model classes.
Merge branch '9888-move-job-reuse-code'
9875: Fix script_parameters_digest to sort hashes hiding inside arrays, too.
Merge branch '9875-fix-params-digest'
9931: Fix missing module. Remove global "include" in tests, so bugs like this can be caught in tests.
9799: Fix method def style
9799: Remove duplicate uuids from db queries.
9799: Merge branch 'master' into 9799-nonadmin-logs
9931: Fix missing include in workbench test support.
Merge branch '9799-nonadmin-logs'
9679: Extricate container/job log code from pipeline-specific code.
9679: Mention container state changes in container [request] log messages.
9679: Add trailing newline to log text if necessary.
Merge branch '9679-dispatch-event-logs'
9898: Remove unused code
9898: Change state transition error responses from 403 to 422.
9898: Check previous state after obtaining row lock.
9898: Code cleanup.
9898: Fix broken test.
9888: Add find_reusable unit test.
9996: Stop retrieving collections if a fatal error makes any further work futile.
Merge branch '9888-unit-tests'
Merge branch '9996-stop-on-error'
9898: Update test for new response code.
Merge branch '9898-container-lock-api'
Fix crash when pipeline template is not readable. refs #10003
9986: Send SIGKILL if llfuse thread does not join.
9986: Share one multiprocessing pool with all IntegrationTests.
9986: Kill test suite when danger of deadlock is detected.
9986: Report actual thread shutdown wait if >0.1s (instead of just saying it took >1s)
9986: Move llfuse thread-shutdown check into a "finally" block so it actually runs.
Merge branch '9986-fuse-test-deadlock' closes #9986
9986: Retry (up to 2 times) any python test suite killed by SIGKILL.
Merge branch '9986-fuse-retry-if-killed' closes #9986
9954: Add systemd unit keep-balance.service.
10008: Move token exp test back to IntegrationTest
10008: Use mocks to ensure collection is re-fetched.
10014: Add details to various error messages.
10014: Add error details. Use filesystem language ("directory") instead of "subcollection".
Merge branch '10014-collection-error-detail' closes #10014
Merge branch '9954-keep-balance-systemd' closes #9954
9957: Refactor keep-web to load config from a file, with legacy support for command line flags.
9957: Clarify anonymous token explanation.
9957: Clarify AuthToken is not used.
Merge branch '9957-keep-web-config' closes #9957
9955: 9950: Add shim for using an arvados.Client to configure an arvadosclient.ArvadosClient.
9955: Add keepproxy config file and systemd unit file.
Merge branch '9955-keepproxy-config' closes #9955
gofmt. refs #9955
Use longer timeout for flaky test.
10124: Replace "sleep 1, then assert" with "retry assertion for up to 10 seconds".
Merge branch '10124-fix-sleepy-tests' closes #10124
9950: Load config from /etc/arvados/arv-git-httpd/config.json.
9950: Rename Root config to RepoRoot.
9950: Add systemd unit file arv-git-httpd.service.
9950: Propagate Client.Insecure config to arvadosclient library via ARVADOS_API_HOST_INSECURE env var.
9950: Remove uses of environment variables.
Merge branch '9950-arv-git-httpd-config' closes #9950
9912: Support YAML config files. Change default config file paths to /etc/arvados/foo/foo.yml.
Merge branch '9912-config-yaml' closes #9912
9990: Add configs for Workbench API client timeouts.
Merge branch '9990-workbench-client-timeout' closes #9990
9953: Ignore non-container events (volume, network) and events with no status, instead of crashing.
9953: Load config from file.
9953: PEP-8
9953: Install python data files to /usr/share/... not /usr/data/share/...
9953: Add systemd unit file for arvados-docker-cleaner.
9953: De-duplicate Go and Python postinst/prerm scripts.
9953: Update build scripts to Go 1.7.1.
9953: Install/remove systemd unit files wherever systemd exists, even if it is not running.
9953: If config cannot be read/parsed, show just the error without the stack trace.
9953: Update default config path.
9950: Fix arvados-git-httpd executable path. Update config path to /etc/arvados/git-httpd/.
9950: 9953: 9954: 9955: 9957: Add AssertPathExists= to unit file to avoid startup until config file exists.
9953: Add arvados-docker-cleaner.service to MANIFEST.in.
Merge branch '9953-dockercleaner-config' closes #9953
9848: group some assertions into bigger test cases.
9848: Copy the output and log collections (if any) when finalizing a container request.
9848: Use pointer receiver for all arvadosclient.ArvadosClient methods.
9848: Set expiry time on container output and log collections.
Merge branch '9848-copy-container-output' refs #9848
Increase timeout and allow early success on download test. No issue #
9956: Load volume config from YAML file
9956: Add systemd unit file keepstore.service
10041: Set ARVADOS_KEEP_SERVICES from Client.KeepServicesURIs field.
5737: Update gems so tests run in Ruby >2.1
5737: Swap comparison so we get non-equal strings instead of a time parse error.
5737: Avoid redefining recursive_hash_search() on each use
5737: Refactor websocket test suite setup to behave well under test-unit
5737: Rename "skip" to "omit"
5737: Update passenger gem
5737: Fix some ruby warnings
5737: Use test-unit-activesupport gem. Fixes "omit".
5737: Update httpclient gem
5737: Fix some ruby warnings
5737: Fix test so call counts are as expected
5737: Tweak websocket tests
5737: Fix accidental use of local variable instead of intended instance method
5737: Send passenger logs to console
5737: Fix passenger 502: wait for handshake to finish before closing websocket
5737: Tidy up Gemfile and update bundle
5737: Cleanup test and add comment about collation sensitivity.
5737: Return of Minitest
9848: Finalize container request immediately if resolving to an already-finished container.
Merge branch '9848-finalize-on-reuse' closes #9848
10008: Disable flaky test. refs #10008
10234: Change default repo root from CWD to recommended path.
10234: Clean up program name (arvados-git-httpd) and config paths.
10234: Use YAML for example config file.
10234: Avoid trying to read real config file in test suite.
10234: 10263: Use a GitoliteHome configuration entry instead of passing through gitolite env vars from the caller.
10234: 10263: Update arvados-git-httpd docs to use systemd + config file.
10234: Pass through gitolite env vars so old env-var-configured sites continue working.
10234: Remove doc.go, already superseded by usage.go.
Merge branch '10234-git-httpd-config' refs #10234 refs #10263
10272: Fix nil pointer dereference in help message. refs #10272
10211: Return an error instead of crashing if stream is closed twice.
9956: Remove "listen on address with no port" in help message.
9956: Move deprecated options out to a separate file.
9956: Remove obsolete TODO comment.
9956: Show list of supported volume types.
Merge branch 'master' into 9956-keepstore-config
9956: Fix run_test_server: default is now -enforce-permissions=true.
10211: Clarify expected result of double-close.
9956: gofmt
10280: Fix ignored error from cmd.Start().
Merge branch '9956-keepstore-config' refs #9956
Merge branch '10280-squeue-error' closes #10280
10281: Add DirectoryReplication config.
10281: Update comment.
10281: Test default replication=1 for directory volumes.
Merge branch '10281-directory-replication' closes #10281
Merge branch '10211-double-close-crash' closes #10211
9956: Fix startup log message. refs #9956
Remove obsolete comment. No issue #
3486: Accept false and nil/null as parameter values, and avoid translating nil to false.
10232: Own just a section of authorized_keys instead of clobbering the whole thing, unless --xclusive flag given
10232: Increase ascii art.
Always log sbatch command, not just on failure. No issue #
Merge branch '10232-non-exclusive-keys' refs #10232
Fix wrong environment variable ARVADOS_API_INSECURE -> ARVADOS_API_HOST_INSECURE. No issue #
10297: Accept docker images named sha256:{...}.tar when specified by PDH.
Merge branch '10297-container-image-file-sha256' refs #10297
10291: Add config to disable API methods.
10291: Fix catching all exceptions in test case.
10291: Check discovery document before trying to choose/use a work API.
10287: Perform blacklist and duplicate checks on usernames received from SSO.
Merge branch '10291-discovery-blacklist' refs #10291
Merge branch '10287-sso-username' refs #10287
10291: Respond 404 to requests for disabled APIs.
10291: Test multiple disabled APIs.
Merge branch '10291-really-disable-apis' closes #10291
Merge branch 'crunch-job_finds_newer_docker_hashes' of https://github.com/tmooney/arvados
10232: Call getpwnam() and getgrnam() for every name instead of relying on Etc.to_enum().
10008: Call checkupdate() on ancestors when opening a file.
10008: Merge branch 'master' into 10008-flaky-token-test
10008: Move test setup from __init__() to setUp()
Merge branch '10232-non-exclusive-keys' refs #10232
10008: Comment to explain ancestor update.
10291: Update --help message. refs #10291
10008: Comment to explain known deficiencies.
Merge branch '10008-check-token-exp-on-open' refs #10008
10343: Allow running without config file, if -config arg is not given.
10343: Update arvados-docker-cleaner install docs to use packaged systemd unit file.
10343: Improve log message.
10343: Update postinst script to run on Red Hat systems, and accommodate Python33 Software Collections.
10343: Add comment about unwieldy ExecStart command.
10343: Merge branch 'master' into 10343-dockercleaner-config
Merge branch '10343-dockercleaner-config' refs #10343
10224: Change event_type to "delete" to match HTTP verb and Python SDK code.
10224: Choose a recent-event threshold without querying the entire event history.
10224: Update tests to expect event_type="delete".
10224: Workaround Firefox failure to parse localized dates in metric locales.
Merge branch '10224-efficient-event-poll-startup' refs #10224
10459: Remove old IP from DNS when a node changes its IP or hostname.
10459: Tidy up some verbose code.
Merge branch '10459-cleanup-node-dns' refs #10459
10468: Add ConnectTimeout and ReadTimeout configs for S3 volumes.
10468: Use long S3 timeouts by default, instead of waiting forever.
10468: Add RequestTimeout to Azure volume config.
10467: Abort S3 request and release buffer if caller disconnects while server is waiting for S3.
10467: Add Debug flag to config.
10467: Return "context done" error instead of 404 if client hangs up during GET.
10467: Move http request context setup out to func.
10467: Abort S3 and release buffer if caller disconnects during S3 PUT request.
10467: Fix panic: cannot call CloseNotify() after ServeHTTP finishes.
10467: Use context instead of http.CloseNotifier to interrupt buffer waits.
10467: Tidy up s3 early-cancel.
10467: Fix context usage: ensure cancel always gets called.
10467: Fix ctx not propagated.
10467: Fix context error not propagated.
10467: Use ErrClientDisconnect. Convert type assertion panic to 500 error.
10467: Interrupt Compare operation if caller disconnects.
10467: Update keep-exercise to new SDK.
10467: Check for Go 1.7.
Update keep-exercise to new SDK. No issue #
10468: Use consts for defaults.
keep-exercise: limit PRNG usage to 512KiB per write. No issue #
Run keep-exercise tests. No issue #
Merge branch '10468-blob-storage-timeouts' closes #10468
10467: Use context.Background() for tests and background processes.
10467: Merge branch 'master' into 10467-client-disconnect
Merge branch '10467-client-disconnect' refs #10467
10310: Check discovery doc when deciding ArvadosBase.creatable?.
10499: Warn about user guide sections that rely on crunch1.
3486: Fix ignored "required: false" in pipeline template. refs #3486 refs #10470
10517: Test edge cases in "deselect manifest_text by default" feature.
Merge branch '10517-consistent-select' closes #10517
8460: Scaffold for websocket server.
8460: Add default config and -dump-config flag.
8460: Receive events and broadcast to clients.
8460: Send events.
8460: Hide *websocket.Conn behind interface.
8460: Fix connection cleanup and db connection handling.
8460: Ping clients only when read times out and outgoing queue is empty.
8460: Test and package services/ws as arvados-ws.
8460: Check permissions.
8460: Add Log type.
8460: Combine ping and notify goroutines.
8460: Pass datasource in Go style.
8460: Drop unnecessary read timeout handling.
8460: Refactor session logic (subscription protocol) out of handler (queueing and delivery).
8460: Cache permissions.
8460: Check permissions on event target instead of log entry.
8460: Avoid log.Fatal once started.
8460: Obey event_type filters if given in all subscription requests.
8460: Accept nulls in some columns.
8460: Print listening address at startup.
8460: Skip non-log events if filtering by event_type.
8460: JSON request log.
8460: Use websocket server for integration tests.
Merge branch '10534-go-systemd-sdnotify-v14' of https://github.com/wtsi-hgi/arvados
8460: Merge branch 'master' into 8460-websocket-go
8460: Send selected old/new attributes with v0 events.
8460: Cache negative permission.
8460: Log connection stats.
8460: Send {"status":200} messages. Bring up ws server for Python SDK tests.
8460: Support created_at filters.
8460: Allow session Receive handler to queue multiple messages.
8460: Retrieve recent logs and send old matching events if last_log_id given.
8460: Pass tests even if websocket server sends events that do not match our filters.
8460: Return recent events if last_log_id given.
10498: --create-workflow and --create-template do the same thing, depending on --api choice.
8460: Merge branch 'master' into 8460-websocket-go
8460: Structured logging.
8460: Refactor "old events / other messages" mechanism to use the outgoing message queue.
8460: Stop existing nginx server, if any, before starting new.
8460: Reply to unparsable messages with status:400.
8460: Fix up logging.
10498: Tweak var names and help messages.
10498: Check error message in "incompatible APIs" test.
Merge branch '10498-register-with-chosen-api' closes #10498
10470: Recognize more slurm error messages.
Merge branch '10470-slurm-error'
8460: Accept 403 response from API as "not permitted", instead of "unexpected error".
8460: Fix "send to closed channel" race by using context lib to release goroutines.
8460: Inject permChecker from main.
8460: Add /status.json
8460: Report status of outgoing queues.
8460: Move logging setup to log.go, use fixed-width timestamps.
Merge branch '10499-cwl-crunch2-docs' refs #10499
10484: Serve MemStats at /debug.json instead of /status.json.
10484: Report volume IO stats for S3 volumes.
10484: Track s3 errors by response code.
10484: Track non-s3 errors by Go type.
10484: Tidy up stats-tracking code into a bucket proxy type.
10484: Test s3 bucket stats.
10484: Remove unused volumeStats.
Merge branch '10484-keepstore-stats' refs #10484
10473: Log requests/responses in JSON format by default; add config option for text format.
10473: Fixed-width timestamps.
10473: Use fixed-point timestamps and more human-readable field ordering, hide "incoming request" if level<debug.
10473: Tune up text-format logging.
Merge branch '10473-keepstore-logrus' closes #10473
10484: Report current client count.
Merge branch '10473-keepstore-logrus' closes #10473
Merge branch '9897-log-block-prefetch-worker-exceptions' closes #9897
9998: Tidy up, add test.
Merge branch 'master' into 9998-unsigned_manifest
10535: Fix race-to-deadlock bugs in tests.test_events: use threading.Event, not threading.Condition. refs #10535
Merge branch 'master' into 9998-unsigned_manifest
Merge branch '10484-count-clients' closes #10484
8460: More statistics in status.json: events in/out, reqs received, lowest client queue delay.
10586: Fix deadlock in KeepWriterThreadPool.
10586: Suppress stack traces for expected http errors.
10586: Do not pile on more errors in the service.finished() case.
10586: Fix misuse of "!=" to mean "is not".
10586: Add extra notify_all() as deadlock insurance.
10467: Add "client disconnect" test.
Merge branch '10586-writer-pool-deadlock' refs #10586
10467: Add "context canceled" tests for PUT and HEAD.
10467: Update var names in parameterized test func.
8460: Merge branch 'master' into 8460-websocket-go
8460: Move loggedDuration from keepstore to sdk pkg as stats.Duration.
8460: Add systemd unit file.
8460: Add godoc page.
8460: fixup 15aca78 stats.Duration
8460: Rename pg -> event_source.go
8460: status.json -> debug.json
8460: Add simple /status.json
8460: Merge branch 'master' into 8460-websocket-go
8460: Cancel context before returning from handler.
8460: Log entry at startup.
8460: golint fixes.
8460: Fix binary path.
8460: Add json mime type.
8460: Add install doc page.
8460: Tweak logging keys, more similar to keepstore.
10684: Use read-only bind mounts for certs and crunchrunner.
Merge branch '10684-ssl-ro' refs #10684
Merge branch '10467-client-disconnect' refs #10467
10276: Remove data manager (superseded by keep-balance).
Merge branch '10276-remove-data-manager' closes #10276
8460: Fix copy-paste errors; use html entity instead of literal utf8 bullet.
8460: Merge branch 'master' into 8460-websocket-go
8460: Add object_kind field.
8460: Add event_at and props.{new,old}.{pdh,owner} to websocket v0. Fix fuse crash when event_at or props.*.* is missing.
8460: Fix deadlock at shutdown by closing event stream before unmounting.
8460: Add some docs/comments.
8460: Move context-aware-logger code to ctxlog pkg.
8460: Use new ws server for integration tests only if ARVADOS_TEST_EXPERIMENTAL_WS is set.
8460: Rearrange erb so it is also syntactically valid YAML.
8460: Tolerate None where dicts are expected in events.
Tidy up queue usage. No issue #
8460: Merge branch 'master' into 8460-websocket-go
10690: Add "rake config:dump" task to Workbench and API server.
10467: Abandon Azure requests if client hangs up.
Merge branch '8460-websocket-go' refs #8460
Merge branch '10690-config-dump' refs #10690
10585: Add crunchstat -signal-on-dead-ppid option.
10467: Merge branch 'master' into 10467-client-disconnect
Merge branch '10467-client-disconnect' refs #10467
10585: Clean up defaults and error checks; release ticker when finished.
Merge branch '10585-crunchstat-lost-parent' refs #10585
Ensure column information is up-to-date before creating full text indexes. refs #10028
10273: Check scope=[all] when looking for an existing superuser token.
Merge branch '10273-check-superuser-token-scope' refs #10273
Close database connections when ending threads.
10538: Exchange expires_at for new trash_at and delete_at columns.
10538: Use include_trash in keep-balance to avoid premature block deletes.
10538: Add /arvados/v1/collections/{uuid}/trash endpoint.
Fix use of unquoted user input in regexp.
Load keep-balance config from default location if -config arg not given.
10467: local directory driver: skip disk IO if client disconnects before lock is acquired.
8345: fix whitespace
8345: Pin llfuse to <1.0
8345: Fix dirent timestamps.
8345: Revert change to getattr() @catch_exceptions setup.
8345: Upgrade to llfuse 1.0
8345: Build python27-python-contextlib2 for centos6.
Update package-test images to ruby 2.3
Obey --only-build arg in "test packages" mode.
Merge branch 'master' into 8345-llfuse-1.0
Merge branch '8345-llfuse-1.0' closes #8345
Bring back ca-certificates to package-testing images.
Fix version# of centos7 llfuse backport.
8345: Build python27-python-contextlib2 for centos7.
10538: Fix misspelled IsTrash -> IsTrashed
10538: Clear up default_trash_interval logic
10538: Clarify error message.
10467: Remove unneeded errorReadCloser, make getter/putter interfaces more like WriterTo/ReaderFrom.
10467: Remove debug printfs.
10467: Use BlockReader and BlockWriter interfaces instead of passing methods to get/putWithPipe.
10538: Mention fixture names in comments when not using fixture finders.
Merge branch '10538-trash-delete' closes #10538
Merge branch '10467-client-disconnect' closes #10467
10682: Avoid invoking special "test mode" behavior in Azure SDK.
10682: Add backend stats for Azure volumes.
10682: Track Azure backend errors by type.
10682: Add comments to statsTicker.
10682: Add GetMetadataOps and GetPropertiesOps to Azure backend stats.
Merge branch '10682-azure-stats'
10795: Remove test-selector code (use -check.f instead).
10795: For md5(""), skip the request and just return an empty reader.
Merge branch '10795-get-zero-block'
8345: Add run-tests.sh --repeat option.
8345: Compatibility with llfuse 0.41 and 1.x.
8345: Revert llfuse to 0.41.1.
Merge branch '8345-revert-llfuse-to-0.41.1'
Fix run-tests.sh --skip option.
10816: Use a recursive postgres query instead of building the permission graph in Ruby.
10682: Report IO stats for filesystem-backed volumes.
10682: Track readdir ops.
Merge branch '10682-localfs-stats'
10808: Fix crash in cache-ownership-mismatch warning message.
10816: Release savepoint after rollback. Use exec_query instead of execute. Move perms_for_val to a class constant.
10816: Merge branch 'master' into 10816-postgres-permissions
10816: Add comments to poorly-documented name/binds args to exec_query().
Merge branch '10816-postgres-permissions'
Merge branch 'master' into 9998-unsigned_manifest
Merge branch '9998-unsigned_manifest'
9831: Use a timestamp instead of a serial number to come up with a unique name after a name collision.
9831: Avoid attempting the same name twice, even if an attempt takes less than 1ms.
Merge branch '9831-faster-unique-name'
10797: Update arvbox to Ruby 2.3.1, Go 1.7.4.
10797: Fix ruby warnings.
10797: Update install docs to Ruby 2.3.
10797: Update package-building images to Ruby 2.3.1, Go 1.7.4.
10808: Avoid using the disk cache if a different user owns it (e.g., running a rake task or crunch-dispatch as root).
Merge branch '10808-file-cache-ownership'
10808: Exempt "change state to Cancelled" from "locked by uuid" protection.
10808: Move comment.
Merge branch '10808-admin-cancel-job'
Fix run-tests.sh --only-install option.
10877: Configurable base URL for Azure storage, e.g., StorageBaseURL: core.usgovcloudapi.net
10877: Add example using core.chinacloudapi.cn.
Merge branch '10877-azure-base-url'
10880: Fix races in event-polling tests.
Merge branch '10880-poll-test-races'
10883: scancel slurm job when erroring out early.
Merge branch 'master' into 9998-no-count-items-available
10868: Log job runtime constraints and node properties when making a dispatch decision.
10868: Add fake uuid to test case for more realistic log messages.
Merge branch '10868-log-dispatch-decisions'
9998: Change count=true/false to exact/none. Add tests.
Merge branch 'master' into 9998-no-count-items-available
Merge branch 'master' into 9998-no-count-items-available
9998: Handle timestamp collisions in collection index.
10868: Include node UUIDs and properties in job logs.
9998: Update API docs.
9998: Use links instead of specimens to test count param.
Merge branch '10868-job-log-node-info'
9998: Remove superfluous word.
9998: Update CLI usage.
Merge branch '9998-no-count-items-available'
10703: Do not catch signals in crunch-dispatch-slurm. Simplify "stop dispatcher loop" API.
10701: Remove unneeded complexity in squeue invocation.
10703: Un-pyramid run().
10704: Rate-limit startup attempts per container.
10700: Rename PollInterval to PollPeriod in library to match commands and config files.
10990: Add tests for byte range requests that start at byte >0.
10990: Support Range headers with non-zero start offset.
10990: Fix collectionreader tests.
10990: Adjust readahead to work better on small segments. Improve comments.
10990: Remove superfluous code for headers now handled by http.ServeContent.
Merge branch '10990-keep-web-ranges'
10690: Add -dump-config to arv-git-httpd, crunch-dispatch-slurm, keep-balance, keep-web, keepproxy.
10690: De-duplicate dumpConfig.
10690: Fix missing return after exit().
10690: Fix conflicting uses of "config" name.
Merge branch '10690-config-dump'
10700: Rephrase "should cancel" condition to be less unclear.
10868: Remove old DNS entry immediately when a new node re-uses an old node's IP address.
10700: Simplify squeue checker.
10700: Rename squeue identifiers (sqCheck = SqueueChecker{})
Merge branch '10868-ip-addr-conflict'
Merge branch '10700-dispatch'
10705: Fix double-logged error.
Merge branch '10705-dedup-log-message'
10990: Add Seek tests.
Merge branch '10990-keep-web-ranges'
10969: Add docker_image_formats server config, and corresponding check in `arv keep docker`.
Merge branch '10969-check-docker-format'
Merge branch 'master' into 10797-ruby-2.3
10700: Fix error message.
Merge branch '10700-dispatch'
11070: Add some logs, ensure shutdown after a pq connection problem.
11070: Fix startup race: clients can safely connect before/during pq setup phase.
10701: Fix edge case: scheduling_parameters.partitions=[]
10701: Tidy up error handling.
10701: Refactor dispatch library.
10701: cleanup
10701: Add back MinRetryPeriod throttle. Update comments/identifiers.
10701: Fix chan type.
10701: Get multiple pages of queue if necessary.
11088: Fix keep_cache_ram units.
Merge branch '11070-ws-listener-problem'
Merge branch '11088-keep-cache-units'
11017: Add arv-migrate-docker19
10701: Improve comments and identifiers.
10701: Update error text in test.
Merge branch '10701-refactor-dispatch'
11017: When compute nodes use image format v2, prefer migrated docker images.
11017: Add arv-migrate-docker19 to package.
11097: Drop check for conflicting outputs when reusing completed containers.
10797: Ruby 2.3.3
10797: Merge branch 'master' into 10797-ruby-2.3
Merge branch '10797-ruby-2.3'
Merge branch '11017-docker-migration'
11127: Do not crash server if SweepTrashedCollections thread has an exception.
11127: Delete dependent links too when emptying trash.
Merge branch '11127-delete-trash-with-links'
7995: Add keep-balance to install guide.
11097: Merge branch 'master' into 11097-reuse-impure
11097: Update test to match new behavior.
7995: Add "dry run" note.
11097: Update docs to reflect new container reuse behavior.
11097: Clarify reuse query.
Fix dispatch panic when processing an update after tracker has been closed/deleted.
7995: Copy edits.
7995: Fix up inconsistent "e.g." vs. "e.g.,".
7995: Fix up dry-run instructions.
7995: Add note about one keep-balance process at a time.
Merge branch '11097-reuse-impure'
Remove pidfiles after shutting down test servers.
Merge branch '7995-keep-balance-docs'
11156: Fix infinite loop condition.
6347: Use RFC3339Nano to render timestamps loaded from serialized fields.
Merge branch '6347-log-timestamps'
10979: Check for orphans only once at startup. Add missing Lock() in
run-tests.sh exit non-zero if gofmt fails
10777: Stop container if arv-mount dies before container exits.
10777: Close and flush logs right away instead of waiting for next tick.
10979: Add missing SqueueChecker initialization.
10764: Add unit tests
10764: Permission tests. Support PDH permission check.
Merge branch '10777-die-if-arv-mount-dies'
10764: Test v0 session.
10764: Simplify test server shutdown.
10764: De-duplicate real/test server startup. Add test for broken config.
Merge branch '10764-ws-tests'
11168: Change db serialize from YAML to JSON.
11168: Prohibit down-migration to YAML-only codebase.
11168: Add missing require.
11168: Revert serialization change in order to avoid breaking job reuse.
11168: Remove unused import.
11168: Always deep-sort before comparing in where_serialized.
11138: Test for docker image after loading, in case docker-load erroneously proclaimed success.
11168: Double-decode serialized fields if database was mangled by downgraded API server.
11138: Show actual image ID when checking whether docker image is loaded.
Merge branch '11168-serialize-json'
Remove obsolete GOMAXPROCS advice.
Add missing install step: add keep-balance token to keepstore configs.
Merge branch '11138-docker-load-fail'
11166: Use ensure_unique_name to avoid collisions when saving logs and output.
Merge branch '11166-log-name-collision'
3115: Create keep_disks implicitly only in #ping action, not #show.
Merge branch '3115-keep-disk-create-surprise'
11221: Always restart systemd services, even after a few startup failures.
5036: Add "--subtype foo" flag to set mounted filesystem type to "fuse.foo".
Merge branch '5036-arv-mount-type'
10218: Wait for container to be started (not just created) before trying to cancel it.
11065: Delete old audit logs.
11065: Defer audit log cleanup until after_commit.
7709: arvados-cli allows activesupport>=4
Do not let golang.org/x/tools/cmd/bundle get ahead of Ruby bundler.
Fix broken link from crunch2 to crunch1 docs.
Merge branch '7709-sdk-cli-active_support'
11065: Merge branch 'master' into 11065-rotate-logs
10669: Robust cache for httplib2
10669: Bust old (possibly broken) cache entries by naming our files *.tmp
10699: Delete >2-day old cache entries.
10669: Only delete *.tmp when cleaning old cache files.
10669: Test that SafeHTTPCache is actually used.
10669: pep8
10669: Improve error reporting in test case.
Merge branch '10669-safe-http-cache'
10766: Migrate install guide from puma to arvados-ws.
10766: Fix copy-paste error in godoc link.
Merge branch '11221-always-restart-services'
11065: Update comments.
11065: Avoid scientific notation in PostgreSQL literal.
Merge branch '11065-rotate-logs'
Merge branch '10766-install-ws'
11209: Add arv-mount --unmount and --replace flags
11209: "--unmount /path/..." unmounts /path and all fuse mounts below it.
11209: Try unmounting at least once even if timeout=0.
Remove -i flag from Rails3 pg_dump invocation.
Remove debug printf.
11209: Fix missing import. Use context handler for standalone mode.
11209: Add tests for --unmount and --replace flags.
11209: Remove unused imports.
Pin ws4py <0.4 (our websocket tests fail on 0.4.1)
7709: Fix tests (count existing links before adding new ones).
7709: Leave container_request alone when applying default keep_cache_ram value.
7709: Do not set job_readable when it won't be used.
7709: De-duplicate "ensure unique name" implementations.
7709: Update bundle
7709: Upgrade to rails4, fix some of the compatibility issues.
7709: Allow null for empty serialized fields.
7709: Remove passenger/puma websocket tests.
7709: Remove services/api/log/
7709: Serialized fields reject wrong types with useful error messages.
7709: Fix full-text index check
7709: A scope that permits GET permits HEAD too.
7709: Accommodate Rails3 or 4 error message.
7709: Fix state leaking between test suites.
7709: Fix some ruby warnings.
7709: Suppress Ruby warnings while loading gems.
7709: Fix warning while suppressing warnings.
7709: Fix Ruby warnings.
11209: Fix daemon mode.
11209: Replace "--unmount /path/..." with "--unmount-all /path".
11209: Test using ./bin/arv-mount from source dir.
7709: Avoid omniauth >= 1.5.0 to stay compatible with Ruby < 2.1.9
7709: Use assert_nil instead of assert_equal nil.
7709: Dispatch assert_equal to assert_nil when needed to accommodate MT6.
7709: Fix lazy splat.
Merge branch '11209-unmount-replace'
11361: Add pdh index to collections table.
Merge branch '11361-pdh-index'
11235: Log a message when a job is interrupted by node failure.
Merge branch '11235-log-retryable'
11355: Arvados gems accept json 2.x.
11209: Restore missing import.
7709: Fix tests (count existing links before adding new ones).
7709: Leave container_request alone when applying default keep_cache_ram value.
7709: Do not set job_readable when it won't be used.
7709: De-duplicate "ensure unique name" implementations.
Merge branch '7709-api-rails4' (partial)
11100: Add container_requests.output_ttl field. Fix validation of output/log changes on finalized CRs.
11100: Clean up permission checks.
11100: Add comment about delete_at validation race.
11100: If caller sets collection.delete_at too early, set it to the earliest allowed time instead of failing.
11398: Fix missing items_available in nodes#index response.
Merge branch '11398-nodes-items-available'
11402: Disable caching if $HOME/ is not writable, instead of returning a broken cache.
11100: Specify units in output_ttl docs.
11100: Adjust delete_at in before_validation hook instead of validation. Permit changing trash_at without extending delete_at.
11308: fmt
11308: Test sdk/python with Python 3.
11308: Futurize.
11308: Fix futurize.
11308: Futurize stage2.
11308: UserDict to Python 3.
11308: md5 to hashlib.
11308: Update python-gflags.
11308: Import future module.
11308: Eliminate old_div().
11308: Fix variable name conflict.
11308: Fix bytes vs. str problems.
11308: md5 -> hashlib.
11308: Fix deprecated logger.warn -> logger.warning.
11308: run-tests --skip python2
11308: Fix string/bytes confusion for Python 3
11308: Fix traceback.format_exc() usage.
11308: assertRegexpMatches -> assertRegex
11308: Expect full class name in traceback.
11308: Close socket after pycurl.
11308: pep8
11308: Fix keepstub bugs and Python 3 compatibility.
11308: Add stderr assertion for easier debugging.
11308: Update assertion to accommodate different exception in py3.
11308: Fix key order sensitivity.
Merge branch '11100-cr-output-ttl'
11308: Remove obsolete test setup steps.
11308: Create missing temp dir. Unset proxy env during tests.
11308: Raise exception on invalid/unsupported open() mode.
11308: Fix exception handling in test case.
11308: Fix useless assertion.
11308: Fix misleading argument name.
11308: Move stub server setup to keepstub.StubKeepServers.
11308: Remove superfluous hashbangs
11308: Make imports epydoc-parseable.
11308: Close socket after pycurl.
8104: OPENSOCKETFUNCTION accepts calls from pycurl 7.21.
8104: Remove pycurl<7.21.5 dependency.
8104: Stop building pycurl package for ubuntu1604.
8104: Drop unnecessary lambda.
8104: Do not close sockets after giving them to pycurl. Reverts 7b31dd40b31bdbf7473b50fd407cd6f636657c69.
Merge branch '8104-pycurl-721'
11209: Restrict --unmount* operations to given --subtype.
Merge branch '11209-unmount-subtype'
11349: Add management server with /status.json
11349: Report wishlist size in status["nodes_wish"].
7709: Merge branch 'master' into 7709-api-rails4
7709: Fix ActiveRecord::ImmutableRelation.
Secure link to google search.
11308: Put back exception handling
11308: Fix futurize misinterpretation.
11308: Avoid Python2-inefficient list() operations.
11308: Remove superfluous parens.
11308: Fix bytes vs. strings in test case.
7709: Remove Ruby websocket server.
7709: Clear deliveries before each test. Fixes flaky test.
11349: Warn if management/status server is not enabled.
11349: Move assertion outside loop for clarity.
Merge branch '11349-nodemanager-status-api'
Build packages for python "future" module.
11349: Fix section name in example configs.
11308: Remove unused import.
11308: Merge branch 'master' into 11308-python3
11209: Use arv-mount --unmount-all instead of mount|awk|xargs script to clean up stale mounts from previous jobs.
11308: Restore dependency lost in merge.
11308: Fix whitespace
11308: Futurize test_arv_get.py and commands/get.py
11308: Remove unused import.
11308: Fix bytes vs. strings in arv-get command and tests.
11308: Add sanity check for python3-dev.
11308: Stop if fuse.h or gnutls.h sanity check fails.
Merge branch '11209-crunch-unmount-all'
Fix crunch-run tests.
11509: Allow cross-origin requests with Range headers.
Fix ignored symlink failure.
11509: Add comments.
Merge branch '11509-keep-web-cors-range'
11459: Move RAILS_PACKAGE_ITERATION to run-library.
11517: Add indexes to help slow queries.
Merge branch '11517-indexes'
11470: Fix accidentally cleared fields during job_task update.
11470: Fix identifiers to conventional Go.
Merge branch '11470-update-task-fields'
11308: Merge branch 'master' into 11308-python3
11308: Fix bytes vs. strings from recent merge.
11308: Fix arvfile append mode: write() changes the file pointer.
11517: Use count=none when retrieving logs.
11517: Comment with_count() method
11517: Skip database reset in some workbench unit tests.
11517: Add test for count=none.
Merge branch '11517-no-count-logs'
Fix panic in test case (overwriting a locked sync.Mutex with an unlocked one).
7709: Merge branch 'master' into 7709-api-rails4
11544: Return 404 instead of 500 for malformed collection IDs.
Merge branch '11544-bad-collection-id-500'
7709: Merge branch 'master' into 7709-api-rails4
Merge branch '7709-api-rails4'
7709: Add rails-observers gem so plugins can use it.
Merge branch '7709-rails-observers'
11537: Add Via header to proxied keepstore requests.
11537: Add Via header to get/head/post/put responses.
Merge branch '11537-keepproxy-loop'
10988: Remove apt.arvados.org repo priority: we no longer publish/require the older packages.
Merge branch '10988-document-ubuntu1604-support'
11606: Disable deep_munge behavior. Rename test so it runs in test suite.
11308: Merge branch 'master' into 11308-python3
11308: Stop test suite if api etc. cannot be started.
Merge branch '11606-rails4-no-munge'
11308: Fix bytes vs. strings from recent merge.
11308: Merge branch 'master' into 11308-python3
8019: Use tl.stopping channel instead of tl.flush to signal Close. Use unbuffered channel to signal flush.
11629: Limit database reads for all list responses, not just index.
11634: Fix whitespace.
11634: Prevent bundler from trying to use/update system gem cache.
11308: Merge branch 'master' into 11308-python3
Merge branch '11308-python3'
11629: Fix items_available in groups#contents response.
11629: Apply memory limits to job/pipeline "components" fields.
11638: Add PostgresPool config option, and default application_name
11638: Log a warning if Postgres pool size is unlimited.
Merge branch '11638-limit-conns'
11308: collection.keys() supports indexing for python2 callers.
11308: Fix modes not tested in test case.
Merge branch '11308-collection-keys'
Merge branch '11629-groups-contents-memory'
Merge branch '11634-bundler-cache'
11629: Fix ambiguous column reference.
Merge branch '11629-groups-contents-memory'
11590: Add "reuse jobs even if conflicting outputs exist" and "log reuse decisions" config options.
11681: Return native str type from KeepLocator.__str__() and Collection.portable_data_hash()
Merge branch '11681-str-newstr'
11590: Reword log messages.
Merge branch '11590-log-reuse'
11644: Add /mounts endpoint using random mount UUIDs assigned at runtime.
11644: Use filesystem UUID and path as DeviceID for local disk volumes.
11644: Use endpoint and bucket name as DeviceID for S3 volumes.
11644: Use storage account and bucket name as DeviceID for Azure volumes.
11644: Accept index/pull/trash requests for a specific mount.
11644: Add pull-to-mount-UUID test. Tidy up pull worker and tests.
11644: Test non-empty MountUUID in trash list.
11644: Replace linear search with map for looking up mounts by UUID.
11644: Add volume replication level to /mounts response.
11644: Ensure generated UUIDs are always 27 chars.
11644: Test fields in /mounts response.
11546: Avoid loading/saving non-essential fields in /arvados/v1/containers/lock.
11546: Wrap lock/unlock in transactions.
11720: Control dependencies with govendor.
Merge branch '11546-fast-lock'
11720: Merge branch 'master' into 11720-govendor
11720: Update Go dependencies.
11720: Add vendor/.gitignore.
11590: Clarify "job state" condition in log message.
Merge branch '11590-log-reuse'
11590: Log container reuse decisions.
11644: Unify block-index handlers. Move prefix arg to query string.
11644: Add DeviceID() to Volume interface.
Merge branch '11644-mounts-api'
11590: Add container logging tests.
11590: Dry up log_reuse_info() calls.
9005: Remove debug printf.
9005: Fix missing Close().
9005: Fix missing error checks.
Merge branch '9005-conn-leak'
9005: 11726: 11729: Disable http keepalive.
Merge branch '9005-disable-keepalive'
Merge branch '11590-log-reuse'
Merge branch '11720-govendor'
9005: Share http Transports and Clients across KeepClients.
9005: Copy default transport if possible. Move magics to consts.
9005: Remove periodic CloseIdleConnections().
9005: Modify default transport instead of building from scratch.
Merge branch '9005-keep-http-client'
11720: Fix go test recipe so *_test=-check.vv uses vendor dir, too.
9005: De-duplicate arvadosclient setup.
9005: Keep service discovery up to date automatically.
9005: Fix test cases to accommodate service discovery changes.
9005: Tidy up test suite.
9005: Rename ClearCache to RefreshServiceDiscovery
9005: Restore "refresh services on SIGHUP" to keepproxy.
9005: Replace matchTrue regexp with StringBool func.
11652: Add "recursive" flag to groups#contents controller.
11652: Update API docs.
9005: Serve keep_services/accessible regardless of auth status.
Merge branch '9005-share-discovery'
11809: Cache permission and collection lookups.
11652: Add tests for non-recursive and home-project cases.
11652: Merge branch 'master' into 11652-recursive-contents
11809: Add unit test for cache.
11809: More cache tests.
Merge branch '11652-recursive-contents'
11809: Add /status.json handler.
11809: Rename FooEntries -> MaxFooEntries in cache config.
11809: Prune collection cache asynchronously.
11809: Increase default cache sizes.
11809: Skip lookups in forceReload case. Add forceReload tests.
11809: Merge branch 'master' into 11809-keep-web-cache
11809: Add tests for /status.json.
Merge branch '11809-keep-web-cache'
11837: Fix "include_trash" scope and test case.
8784: Add keep-web directory listings via (*arvados.Collection)FileSystem().
8784: Correct wget args, tweak styles.
8784: Show file sizes.
11803: Get group permissions with 1 query instead of N queries.
10557: Tidy up some user setup code.
10557: Always run user setup procedure when is_active becomes true.
8784: Use arvados.Collection in cache.
Merge branch '11803-repo-perms'
2411: Add build/check-copyright-notices.
Merge branch '2411-check-copyright'
10557: Remove redundant hook condition.
11283: Fix "available slot number" query.
8784: Fix error returned by Readdir() when count > remain.
11283: Fix exec_query() usage for rails4.
Merge branch '10557-auto-setup'
11729: Avoid refetching manifest in forceReload case
Merge branch '11729-cache-miss'
8784: Just pass collection name to serveDirectory, not whole record.
8784: Improve comments. Fix fake error on range requests.
8784: Show "no files" note if the collection is empty.
8784: Remove useless condition.
8784: Update to Go 1.8.3.
11283: Add explanatory comment to SQL query.
Merge branch '11283-available-slot-query'
8784: Merge branch 'master' into 8784-dir-listings
8784: Replace remaining uses of keepclient.Reader with arvados.File.
Merge branch '8784-dir-listings'
10557: Tidy up users#setup controller.
8784: Update arvbox to go1.8.
Merge branch '10557-setup-cleanup'
8784: Use keep-web to serve dir listings for shared collections.
8784: Skip redundant arv-put invocations.
8784: Fix service discovery race in tests (send SIGHUP to keep-web).
8784: Link directly to keep-web when sharing collections.
11901: Fix unclosed db.Rows object.
11901: Add /_health/ping and /_health/db health checks.
11901: Require management token for health checks.
11901: Report db stats (notably OpenConnections) in /debug.json
11901: Test 404 when health checks disabled.
11901: Try more tokens in 404 test.
Merge branch '11901-ws-db-conns'
2411: Fix liquid comment template and skip top matter in docs.
2411: Fix up whitespace.
2411: Add exceptions.
2411: Fix tests to accommodate copyright notices.
2411: Set template for more nodemanager test files.
2411: Add copyright after <?xml declaration in svg files.
11807: Migrate old records in jobs table from YAML to JSON.
2411: Add copyright notices to everything.
2411: Remove golang<1.4 shim.
2411: Add exceptions.
2411: Eliminate whitespace to placate test.
Merge branch '2411-check-copyright'
2411: Maintain copyright header in structure.sql.
Merge branch '2411-rake-copyright'
11917: Do not clear rails cache at boot time.
11910: Fix racy tests: ignore non-matching logs from previous tests.
11910: Ensure test passes despite race between old and new events.
Merge branch '11910-ws-test-race'
Merge branch '11807-yaml-to-json'
8784: Add token to path (not query string) in download links.
8784: Fix test for latest firefox.
Fix regex usage.
Update .licenseignore.
Add missing copyright headers.
11843: Improve formatting.
11843: Do not fail pipelines on transient API failures.
Merge branch '8784-dir-listings'
11927: Remove errant type attr from anchor element.
11927: Eliminate unnecessary use of selenium.
Merge branch '11927-tags-test'
2411: Fix crunch-job license statement.
Merge branch '11843-arpi-transient-error'
11945: Disable collection cache.
Merge branch '11945-disable-cache'
11945: Key collection cache on pdh+token.
Merge branch '11917-dont-clear-cache'
Merge branch '11945-collection-cache-key'
11950: Recognize new (2.11.0, debian9) git error messages.
Merge branch '11950-stretch'
11944: Remove invalid assertRaises(None, ...)
11944: Turn off warnings while testing command line tool stderr.
11944: Ignore arv-put info on stderr.
11960: Include is_trashed in old/new attributes sent to client.
11960: Fix permission checking for events on trashed collections.
11960: Fix permission checking for "delete" events.
11965: Update help message to match new --api default.
11968: Wait in DB() for a database connection if necessary.
Revert "11965: Update help message to match new --api default."
Revert "Merge branch '11857-acr-default-crunch2'" (edf37e92f885ed4d5bcf587317cc9c6d90d8ece1)
11960: Test permission on "delete" event.
11960: Fix events leaking between tests.
11960: Test trashed collection.
11960: Test that "delete" permissions are not too permissive.
Merge branch '11948-cwl-conformance-fix'
Merge branch '11968-ws-startup-crash'
11960: Fix races in tests.
11960: Get the full list of old IDs up front in sendOldEvents.
Merge branch '11960-trash-events'
11973: Upgrade setuptools in package-builder images.
11973: Set DEBIAN_FRONTEND=noninteractive in pkg-build/test images.
11973: Build avro 1.8.1 for ubuntu1604 and add to schema-salad deps.
11973: Tweak fpm dependency spec.
11842: Split generic webchart code from ChartJS to WebChart.
11842: Generate charts with dygraphs.
11842: Use CDNJS for dygraphs js and css.
11842: Remove chartjs.
Merge branch '11842-crunchstat-summary-dygraphs'
10805: Fix crash during shutdown when running with llfuse 1.x.
Merge branch '10805-llfuse-1.2'
10805: Skip llfuse.close() to avoid llfuse deadlocks in test suite.
Merge branch '10805-avoid-llfuse-deadlock'
10805: Build llfuse-1.2 for ubuntu1604.
10805: Bump debian version in fuse install doc.
Style fixes.
11906: Refactor health-check handler into SDK.
11906: Use sdk/go/health for keepstore health checks.
11906: Add tests for sdk/go/health.
11906: Remove keepstore's health-check unit tests.
1906: Fix up doc comment.
11906: Test 404 cases.
11906: Clarify "zero value" test.
Merge branch '11906-health-check-lib'
12025: Change permission cache key to avoid upgrade bugs.
12027: Exit "retry unlocked" if slurm fails during setup phases.
Merge branch '12025-perm-cache-version'
Merge branch '12027-slurmfail-during-setup'
11906: Remove intermediary servemux and handler func.
12024: Use goamz fork to fix Go1.8-vs.-AWS transfer encoding bug.
Merge branch '12024-zero-content-length'
11898: Remove superfluous uniq(&:id) call
12024: Build new Go packages when the vendor bundle changes.
Merge branch '12024-vendor-bundle-version'
12024: Fix syntax error.
12024: Fix checksums in vendor.json.
Merge branch '12024-vendor-bundle-version'
12024: Use vendor bundle when building Go packages.
12024: Update help message.
12078: Fix crash encoding booleans with libjson-perl 2.90.
Merge branch '12078-perl-bool-params'
12027: Recognize a new "node failed" error message.
Merge branch '12027-slurm-error-text'
12033: Add mithril via npm.
12033: Add session-manager page.
12087: Fix up context cancel usage.
12087: Fixup some problems found by go vet.
12087: Don't report read/write errors on terminated sessions.
12087: Remove useless continue statement.
12087: Exit "wait for low water mark" loop if session terminates.
12033: Add /collections/multisite search page.
12033: Fade out stale results.
12033: Link collections to remote workbench sites.
Merge branch '12087-ws-busy-loop'
12033: Use browser locale to render collection timestamps.
12033: Merge results from all sites into one table.
12033: Add "load more" button.
12033: Extract multisite loader to its own class.
12033: Style uuid column.
12033: Load more results automatically on scroll.
12033: Fix auto scroll after search; acknowledge end of results.
12033: Ensure current Rails login is always in sessions list.
12033: Fix logging in from location with non-empty query string.
12033: Log out and back in to a site without forgetting it.
12033: Fix double slash in href.
12033: Add top nav link to multisite search.
11309: Recognize containers and container requests by UUID infix.
12033: Restore search term and scroll position after navigation.
12033: Fix typo in selector.
12033: MultipageLoader and MultisiteLoader offer the same interface.
12033: Generalize MultisiteLoader to MergingLoader.
12033: Move classes to global namespace.
12033: Fix site status indicators.
11309: Accommodate crunch2 crunchstat log format.
11309: 9001: Fix catch-all-exceptions antipattern.
11309: Fix parsing and labels. Add --threads option.
11309: Prefix errors with "notice:" or "warning:" to aid parsing.
11309: Drop parent container name from labels.
11309: Fix runtime_constraints suggestions for containers.
11309: Sort containers by request time.
11898: Remove superfluous uniq(&:id) call
Merge branch '11898-no-distinct'
12123: Don't expect properties.new_attributes.components in events.
12081: Retry "update" API when called via ResourceProxy->save.
12134: Revert "10805: Skip llfuse.close() to avoid llfuse deadlocks in test suite."
Merge branch '12081-crunch-job-retry'
Merge branch 'master' into 12033-multisite-search
12033: Restore "rake npm:install" step to arvbox.
Merge branch '12123-arv-ws-keyerror'
12123: Implement "unsubscribe" method.
12123: Subscribe before unsubscribing to avoid losing messages.
11309: Fix mocked API data to pass test.
11309: Fix NewSummarizer()'s ability to accept a loaded API record.
11283: Update api server's arvados gems.
Merge branch '11309-crunchstat-summary-containers'
12033: Combine loading/done flags into a ready/loading/done state.
12033: Simplify loader interface.
12033: Improve confusing comment.
12033: Remove unused code.
Merge branch '12123-ws-unsubscribe'
Merge branch 'master' into 12033-multisite-search
Update test to match new log message.
12033: Canary query before initiating login procedure.
12033: Improve multi-site search navigation a bit.
Merge branch 'master' into 11898-no-distinct
12134: Use fixed/unreleased llfuse in test suite.
12134: Minimize multiprocessing worker startups.
12033: Add config to link multi-site search to a remote site.
Merge branch '11898-no-distinct'
12033: Add Object.assign polyfill for test suite.
12033: Disable/hide multi-site search by default.
12033: Avoid startsWith(), unsupported in test suite.
4878: Fix whitespace and missing </p> tag.
10472: Ignore "Running [command]" log message from crunchstat.
Merge branch '12134-llfuse-patch'
12190: Don't test block existence before first write.
Merge branch '12190-s3-read-after-write'
12033: Merge branch 'master' into 12033-multisite-search
12033: Skip "show" btn if guessed workbench link is obviously wrong.
12033: Document SaveUIState.
12033: Document one-SaveUIState-per-page limitation.
Merge branch '12033-multisite-search'
10472: Conserve FDs by not opening collections until ready to read.
10472: Follow job trees made by arvados-cwl-runner jobs.
10472: Un-bork text reports.
12033: Fix tests.
12033: Fix tests.
12033: browserify is needed in production env, for package-building.
12033: Fix tests.
10472: Add test for arvados-cwl-runner job tree.
10472: Merge branch 'master' into 10472-csummary-cwl-pipeline
12194: Construct @@ operands more carefully to avoid bad API munge.
12194: Fix accidentally mutated argument.
12194: Fix empty search case.
12107: Use git-fetch instead of pack|unpack, and only when needed.
12194: Handle to_tsquery(null).
12194: Update comment.
12221: Merge branch '12221-crunchstat-summary-valueerror'
10472: Fix logging format.
10472: Use itervalues() instead of iteritems() if key is not used.
12033: Fix npm sanity-check recipe.
Merge branch '10472-csummary-cwl-pipeline'
12107: Fix unreliable use of git-fetch.
Merge branch '12194-search-always-valid'
12197: Add missing end tags.
12197: Show UUID and PDH of trashed collections.
Merge branch '12194-search-always-valid'
12197: Remove trash/delete labels on content rows.
12197: Test PDH search and display on trash page.
12197: Update PDH column label for consistency.
Merge branch 'master' into 12197-show-pdh-in-trash
Merge branch '12197-show-pdh-in-trash'
Merge branch 'master' into 12107-faster-git
12107: Use a fresh internal.git repo for each test run.
12107: Test fetch-and-tag on unreferenced and non-tip commits.
12107: Fix broken "git branch --contains" check.
12107: Fix broken commit-exists-in-internal-repo check.
12247: Propagate write errors, don't hide them with "bad checksum".
12247: Simplify (Hash)Sum() usage.
12107: Fix test suite dependence on outer git config.
12084: Reap children whose threads exit abnormally (status == nil).
12084: Remove dead code.
12084: Fix dispatcher getting bogged down on "too many open files".
Merge branch '12084-dispatch-stuck'
12107: Add protective comment.
Merge branch '12107-faster-git'
Merge branch '12247-no-failed-checksum'
Fix dashboard crash on uncommitted container request.
12298: Allow non-null log when cancelling an unrunnable container.
Merge branch '12298-cancel-fail'
12276: Ensure dirent cache is disabled.
12287: Preserve JSON number types/formats in container records.
12298: Allow non-null log when cancelling an unrunnable container.
12276: Ensure dirent cache is disabled.
12287: Preserve JSON number types/formats in log file.
12032: Fix compatibility with PostgreSQL 9.4.
Suppress superfluous error message.
12246: Add debugging clues when command is not found in container.
Merge branch '12246-command-not-found'
12347: Fix --disable-reuse for containers API.
12347: Fix formatting.
12347: Fix --disable-reuse for jobs API.
12273: Skip non-regular files when uploading outputs to Keep.
11583: Stop signal handlers on shutdown.
11583: Fix hung goroutines when stubbed container never runs.
Merge branch '12347-disable-reuse'
12347: Don't reuse runner job/container if arv:enableReuse is false.
Merge branch '12347-obey-wf-reuse-hint'
8333: Support "arv keep docker repo:tag" syntax.
Merge branch '12273-skip-special-files'
12260: Health check aggregator.
12260: Test case for one unhealthy service.
12260: Improve data structures.
8333: Support "myrepo.example:8888/repo/image:tag"
8333: Support "myrepo.example:8888/repo/image"
12260: Fix remote ping auth. Make server work.
Merge branch '8333-docker-repo-tag'
12260: Add -config arg. Drop non-resolvable hostname support.
12260: Add more services. Use existing package names.
Update BytesAllocated -> BytesAllocatedCumulative in status report.
12416: Merge branch 'fix/keepstore-s3-radosgw-empty-object' of https://github.com/wtsi-hgi/arvados into 12416-s3-empty-object
12416: Merge Put() into PutReader().
Merge branch '12416-s3-empty-object'
12260: Update comments to match real func/type names.
12260: Merge branch 'master' into 12260-system-health
12260: Build arvados-health package.
Merge branch '12260-system-health'
11260: Fix tests.
12461: Avoid race bug in Rails file cache.
Fix wrong HTTP status code for early "impossible path" check.
Merge branch '12461-cache-race'
Tom Morris (15):
Fix and simplify wait_for_ajax
Restore Capybara.default_max_wait_time side effect
Fix row selection logic to not wait
Fix long wait for Capybara test
Fix not present test for Capybara
Fix negative Capybara assertion/matcher
Add explanatory comment about FIXME
Merge branch '10077-workbench-integration-tests' closes #10077
Switch to 2 space indent - no functional changes
Fix pipeline instance expand - fixes #10778
Ignore editor backup files & .DS_Store
Merge branch '10778-expand-job-bug' - Fixes #10778
Document epydoc dependency
A few copy edits
Add Running to excluded non-categories. Improve error
Ward Vandewege (788):
Initial commit
Make sure to use the right git repo when deploying.
Deploy script fixes
Only add rvm-capistrano in the development group
Deploy script fixes.
Initial commit.
Add rvm-capistrano
Add therubyracer gem
Add pg gem
Add passenger to Gemfile (we run it standalone)
And the matching Gemfile.lock change.
Capistrano fixes
More capistrano fixes
Update to latest Rails 3.1 release
Moved to orvos-app-0
Ignore local .rvmrc file
A few fixture fixes
Very basic tests for /collections, /nodes, /pipeline_invocations
More test env tweaks + first collections api test
Fix for
Authenticate against auth.clinicalfuture.com
Drop superfluous comment.
Fix comment.
Make josh_id propagate return_to as a GET parameter, and use that GET
Remove duplicate configuration value in lib/josh_id.rb.
Rename leftover Metadatum references to Link.
Add deploy scripts for the Orvos instance on Ant.
Remove some unneeded stuff from the ant deploy file.
Do not assume uuid_or_hash is a string when using it in string
Throw an error when the uuid used for find() is invalid.
Fix test for :orvos_api_token in Thread.current, and log errors if it is
Do not show a 'Log out' link when the user is not logged in.
Be careful with the shovel (<<) operator. It modifies the object on the
Switch to Oj for JSON parsing.
Merge branch 'master' of git.clinicalfuture.com:orvos-server
Switch to Oj for faster JSON handling.
Switch to MultiJson which uses Oj under the hood (since it is available).
Switch to MultiJson which uses Oj under the hood (since it is available).
Back out upgrade to MultiJson for now until we upgrade to rails 3.2. See
When no limit is supplied, limit number of index results to 100.
Support
Weed out (most of) the unnecessary leftover bits in our rest discovery file.
Remove superfluous fields from the Link object.
maxResults is really called limit, update the discovery document accordingly
Initial version of the cli client.
Move auto-generated REST discovery document to the right path.
Merge branch 'master' of git.clinicalfuture.com:orvos-server
Now make sure we are really using the new rest file. refs #1406
Do not blow up when a default value for an argument is a boolean.
Handle errors more gracefully.
Do not assume
Make sure to pluralized object names in the List method as listed in the
Fix up the discovery document - make sure we define 'items' in the the
Make subcommands singular. Slight layout improvement, too.
Merge branch 'master' of git.clinicalfuture.com:clients
moved api server code into new directory structure
reorganized repo structure
Merge branch 'clients'
moved explorer into new directory structure
Merge branch 'explorer'
Change title on main page.
More title fixing on index.html
More title tweaks.
Add (rudimentary) install instructions for the API server.
More install instructions.
Ruby SDK: make suppressing SSL warnings a configurable option rather
Ruby SDK: add sufficient files to make an Arvados gem.
Ruby SDK: add gem dependencies
Expose Node slot_number via api.
Remove unique from hostname index in nodes table.
Delete capistrano files; they have been moved to a separate repository.
Remove reference to orvosapi.com, replaced with arvadosapi.com
More API server installation instructions.
Formatting fix
Be more explicit.
Whitespace.
Delete apps/workbench capistrano files; they have been moved to a
Add workbench install instructions.
Merge branch 'master' of git.clinicalfuture.com:arvados
Fix title for workbench install instructions.
Layout fixes.
More layout fixes.
Improve 'Setting up SSH access' instructions.
More tweaks for the ssh page.
Fix ssh ProxyCommand lines.
Add some more detail about arv's dependencies and how to install them.
Fix versoining number for arvados gem.
Merge branch 'master' of git.clinicalfuture.com:arvados
Changes for ruby sdk packaging as a gem.
Merge branch 'master' of git.clinicalfuture.com:arvados
Remove 'pre' from the gem version; otherwise we can't pull them in easily from puppet.
Add whls wrapper. This is a temporary hack. Not proud of it :/
Add piwik support to workbench.
Tweak arv keep wrapper a bit more. No more output rewriting, instead use
Fix numbering of tutorials on workbench landing page.
Typo in the 'create virtual machine' command.
Add docker arvados/docserver build.
We don't use the bootstrap-rails gem, and it does not exist anymore.
Fix deprecation warning:
We don't use the bootstrap-rails gem, and it does not exist anymore.
Fix deprecation warning:
We don't need the pg gem in workbench.
Allow up to 10 levels of nesting for variable substitution in config.rb
Style improvements -- rubocop
Add sass explicitly
We want to be able to handle naked
Add docker image with workbench server.
Add docker image with workbench server.
Fix Dockerfile comments.
docserver container updates:
Add workbench server to build.sh and run.sh
Merge branch 'master' of git.clinicalfuture.com:arvados
Improve error handling for login operations.
Also auto-generate _SECRET config variables when they are unset, like we
Merge branch 'master' of git.clinicalfuture.com:arvados
Bring the name of the workbench secret token config variable in line.
Add SSO docker container
Merge branch 'master' of git.clinicalfuture.com:arvados
Generated files shouldn't have been committed.
Fixes for SSO rails app.
Let API server use SSO container.
un-comment ADD line for api.tar
* OMNIAUTH_APP_SECRET is no longer used
Move UTF-8 locale generation and the DEBIAN_FRONTEND environment setting
Merge branch 'master' of git.clinicalfuture.com:arvados
Update the dockserver Dockerfile to take advantage of the doc/Gemfile,
* Make arvados/passenger container
Optimize docserver/Dockerfile
* Mention SSO server to documentation in README
Make sure the API rails app tmp directory is writable by www-data
Make workbench work with the API server.
Add a config setting: auto_admin_user
* Docker API container: add support for the auto_admin_user feature (API server).
Merge branch 'master' of git.clinicalfuture.com:arvados
Only apply the auto_admin_user configuration rule when no admin users
Only apply the auto_admin_user configuration rule when no admin users
Add an SSH server in each docker container.
Rename docserver to doc for consistency.
build.sh has been superseded by Makefile.
Refactored and updated run.sh:
Only require keep volume when a keep container is being started.
Allow overriding of default ports for API, doc, SSO, and workbench
Remove hardcoded references to Arvados Workbench, replace with
Merge branch 'master' of git.curoverse.com:arvados
More clinicalfuture.com -> curoverse.com
Fix error message when the all-users group can not be found.
All git access should happen via git@ urls, not git://
Make the config_uuid field match what we expect in our tests.
Lock to a version of the arvados-cli gem that's been pushed to the world.
Fix Gemfile(.lock) in sdk/cli
Fix comment - location of arvados configuration file has changed.
Fix up incorrect dependency on google-api-client
Handle the situation where config.workbench_address is unset more
Merge branch 'master' of git.curoverse.com:arvados
The list-inactive-users.rb script has been moved to the puppet tree for
Fix regular expression for collection UUIDs to it can also handle with UUIDs like
Make sure to abort nicely when run with a version of ruby older than 1.9.3
Rename clinicalfuture -> curoverse everywhere.
Add missing curly bracket in assets/stylesheets/application.css.scss
In the pipeline instance comparison, do not blow up when a row component
Update url field in python sdk setup.py file.
Improve python sdk package building to allow for easy distribution via
Update Python SDK documentation to reflect availability of PyPI package,
Improve headings in Python SDK installation section.
Do not blow up on node create when the ec2 configuration values are not
Be consistent in referring to the 'All users' group.
We can't have dependencies on unpublished gems checked in; they break
Fix links to docs in workbench
URL change for the auth server.
When running with WEBrick, auto-generate a new self-signed SSL
Strip all arvados-specific settings from the .example environment files.
Merge branch 'master' of git.curoverse.com:arvados
Typo.
Fix link, this fixes bug 2403
API server: strip all arvados-specific settings from the .example
Do not print secrets in the config:check rake task.
Allow disabling of output for rake config:check via a QUIET environment
doc: Add rake linkchecker task, and document it in the README file
* README: remove numbers in headings
Merge branch '2509-rake-linkchecker'
Do not blow up when HOME is not set.
Create a can_login link for a new user object, even if there are already
Set default value for
Fix test that was expecting a (now) incorrect number of links.
Properties should not have symbol keys. Our API runs on JSON, which does
User.full_name should not contain just a space when first_name and
Improve wording of the welcome e-mail a bit.
Documentation tweaks:
Merge branch 'master' of git.curoverse.com:arvados
Add optional notification for new users that are in the inactive state.
Add tests for 'new inactive user notification' feature.
Fix AdminNotifier, bitrot after upgrade to Ruby 2.x and Rails 3.x.
Remove reference to non-existant users_url
Refactor AdminNotifier.
Remove now-superfluous build.sh step from the documentation to build the
Do not blindly overwrite GOPATH if it exists.
Fix whitespace.
In daemon mode, arv-mount should set up the fuse connection before
CollectionReader can throw exceptions (for example, when passing an
Merge branch '0000-arv-mount-bugfixes'
Add some basic request logging to Keep.
Do not use symbol keys in the properties of permission links.
Coding standards hook, initial commit.
Only check master, because bad things happen otherwise.
Merge branch 'master' into 2681-new-inactive-user-notification
Change default for
Provide link to users page in notification e-mails.
Switch to using auth helpers instead of messing with Thread.current in the user tests.
Refactor user notification tests
Merge branch 'master' into 2681-new-inactive-user-notification
DTRT with system_user in our user test case. Partial revocation of
Merge branch 'master' into 2681-new-inactive-user-notification
Merge branch 'master' into 2681-new-inactive-user-notification
Merge branch '2681-new-inactive-user-notification'
Retry the fusermount call up to 10 times with a small pause between
Merge branch '2951-more-robust-fuse-test-teardown'
Fix workbench URL in inactive user notification e-mail.
Initial commit of the scripts used in our Jenkins pipelines.
Build crunchstat package.
Merge branch 'new-git-hooks' of git.curoverse.com:arvados-dev into new-git-hooks
A few fixes:
Merge branch 'new-git-hooks'
Add puma to the Gemfile for the API server; we use it for the websockets
Fix inconsistent JSON error return behavior on job create.
arv-run-pipeline-instance should not keep trying to create jobs when the
Bugfix: the discovery doc should not clobber defaults values with the
Build a new arvados-src package.
Remove superfluous directory from arvados-src path.
docker: build_tools/build.rb: exit with non-zero exit code when something goes
docker: build_tools/build.rb: support new Debian docker.io packages
docker: build_tools/build.rb: improve language around Google account.
docker: more changes to support new Debian docker.io packages
docker: arvdock: support new Debian docker.io packages
docker: a bunch of tweaks to bring the docker setup up to snuff with the latest
docker: fix up the link between workbench and the API server, work around the
docker: bump up passenger version. This is highly unsatisfactory, and needs
docker: remove warehouse container; add new keep container.
Make sure the arvados-src package actually contains the latest source. Duh.
docker: usability improvements for arvdock.
docker: arvdock: whitespace cleanup
docker: arvdoc: start up doc and workbench last so it's less likely the
Merge branch 'master' into 3118-docker-fixes
Merge branch '3118-docker-fixes'
Add docker test script.
Make sure to exit with $EXITCODE
Add documentation to doc.arvados.org for the local Docker install.
Merge branch 'master' into 3118-docker-fixes
Merge branch '3118-docker-fixes'
Add documentation to doc.arvados.org for the local Docker install.
UnixVolume has no business being loud about files that can not be found.
Move error logging for block-not-found from GetBlock to GetBlockHandler.
Do not print an error when hitting 'not found' with a the MockVolume
Merge branch 'master' into 3141-keep-logging-improvements
Add state of the docker installation alert to the top of the docker
Merge branch '3118-docker-fixes'
The linkchecker rake task should return with the exit code of linkchecker.
Fix up link in the docker alert box.
Remove superfluous cd commands
Merge branch '3118-docker-fixes'
docker: mkimage-debootstrap.sh: wrap which calls that are expected to
Add some more detail to the docker installation page.
Do not blow up trying to run a job if runtime_constraints is set to nil
Merge branch 'master' into 3141-keep-logging-improvements
Revert "Do not print an error when hitting 'not found' with a the MockVolume"
keep: MockVolume should return os.ErrNotExist instead of errors.New("not
Merge branch '3141-keep-logging-improvements'
Doc: fix link to pipeline instances page in workbench.
crunch-dispatch: make sure to update the info['slurm_state'] for the
Make sure to bring the arvados debian packages up to the latest version
Improve arv-run-pipeline-instance's logging when it encounters an error.
Be more explicit that hello-world.py is a script which is expected to fail.
Another tweak to tutorial-job-debug.html.textile.liquid
Whitespace fix.
Be consistent about the version of ruby that is installed by rvm and
Merge branch '3219-further-docker-improvements'
Sanitize how we handle the --ssh option in the api docker container.
Be smarter about building and the --realclean option:
Update the documentation to reflect the new, safer build.sh --realclean
Merge branch '3219-further-docker-improvements'
First set of changes to add crunch-dispatch and compute nodes. This is not complete yet.
Two compute nodes work at the slurm level. Crunch-dispatch runs.
The --pretty option was removed from arv some time ago. Do not refer to
Compute nodes can run jobs - almost. The final arv-put --raw in the
Compute nodes can now run jobs. Also, small fixes all over the place.
3220: fix tests
Add local git repository support.
Merge branch 'master' into 3219-further-docker-improvements
Do not set action_mailer config settings in production.rb.example, we
Further changes to make pipeline running work reliably.
Whitespace cleanup.
Python egg build fix for the sdist rotate command
* Add shell node to docker installation.
Merge branch 'master' into 3219-further-docker-improvements
arvdock starts 2 compute nodes, always.
* Update docs to reflect that crunch is now supported.
Merge branch 'master' into 3219-further-docker-improvements
* move the downloading of skydns and skydock from arvdock to
Whitespace fixes
Make the build script prompt for a shell login name.
Make --deepclean also remove the crosbymichael/skydns and crosbymichael/skydock images.
Suppress warnings and spurious error messages for the clean, realclean
* add argument help to build.sh
Merge branch 'master' into 3219-further-docker-improvements
Add Keep install instructions (manual).
Fix the auto_admin_user feature; it was broken by accident in
Merge branch 'master' into 3219-further-docker-improvements
Merge branch 'master' into 3219-further-docker-improvements
Merge branch '3219-further-docker-improvements'
Whitespace fix in docker install documentation.
Documentation fix: add a link to the Keep installation page in the left
Workbench: make the Setup User screen less confusing by labelling the
Do not blow up on the 'Manage account' page if the db contains an ssh
Add user e-mail address to the list of users an object can be shared with
3261: simplify Makefile
Merge branch 'master' into 3261-keep-docker-config
Merge branch '3261-keep-docker-config'
Be less presumptuous about the GNU/Linux distribution we are being run
* Default to using the auth.curoverse.com SSO server. Add instructions
Update Docker install documentation:
Only the deepclean Make action should remove the skydns-image and skydock-image files.
Provide a bit more detail about build.sh in the documentation.
Merge branch '3363-docker-add-option-to-use-auth.curoverse.com'
Make use of the realclean build.sh feature to build the docker images.
Merge branch 'master' into 3551-go-layout
Merge branch '3551-go-layout'
Merge branch '3551-go-layout'
Merge branch '3551-go-layout'
keep is now called keepstore
docker: be a bit smarter in the make clean target, so that we also
Remove duplicated cleanup commands; the realclean target takes care of
Merge branch 'skip-args'
Be smarter about cleanup in crunch-job after a job finishes. This should
Do not blow up in crunch-dispatch: fix syntax error. Ouch.
Do not blow up in crunch-dispatch.rb: do not pass --jobs and --pipelines
Do not blow up when arv is called with an unrecognized argument.
crunchstat: fix typos in comments and help/error text.
Do not blow up when running jobs in local mode.
Revert 2 hunks from a5819ec1e48fba90658fcf676ffc50c1f216d484, thus
Replace all remaining references to Clinical Future with Curoverse.
Save a bit of time: do not install RI and RDoc for the gems we install
Fallout from #3036: the test for the checkbox value on acceptance of the
Fixes for the deploy stage of the build pipeline: actually upgrade our
Make sure to run apt-get update before trying to install
Also build our java-bwa-samtools docker image.
Only install the latest gem (i.e. the one we just built).
Add some robustness in the postgresql department.
Undo two hunks that I accidentally committed in 565c31196f3ccd4346c8a1ac1e19b1deda22eea6.
Copy only the debs that were generated to the remote server.
Switch from PRM to freight for publishing our packages. It's way, way
Make regexp to match fpm's output slightly more robust.
Bugfix: do not try to overwrite docker_image_locator if it is already set.
* Do not blow up when arv-get requests a collection that gets a 404 from the api server.
Make keepstore log normal operation with more detail, and follow the
Fix typos in workbench language.
Merge branch '3967-improve-keepstore-logging'
Protect the one place in the api server code where we assume the
API server: setting workbench_address is not required. This unbreaks the
In our docker base image: do not rely on pgp.mit.edu, use
In our docker jobs image: use our debian packages for the python sdk and
Improvements to our package building script: fill in more of the
Add new run-diagnostics-suite.sh jenkins script.
Make sure our generated packages are world-readable, otherwise they
Deal with Jenkins' new, tight umask in a more thorough way.
Make run-deploy.sh do a bit more work to update the rest of the staging
Add README, COPYING and agpl-3.0.txt.
Remove spurious newline
Parameterize the instance to run the diagnostics suite against.
Fix stupid bug in run-diagnostics-suite.sh - destination config file
Allow specification of revision to test with diagnostics suite.
Allow for run-diagnostics-suite.sh to pull the git revision to test from
Make run-build-packages.sh much more quiet by default. Also make
More updates for run-build-packages.sh:
Improve output when fpm call fails.
Switch to human readable timestamps (YYYYMMDDhhmmss in UTC) for all our
Upload our arvados/jobs image to the Docker Hub.
Make sure to catch the merge commit when doing git log in the directory
A few improvements for run-build-packages.sh:
Make sure to use httpclient 2.5.0 or higher for Workbench. Older
Build a few more dependency packages for node manager.
run-build-packages: Test that fpm is callable before trying to do
* exclude /usr/local/lib/python2.7/dist-packages/tests from our fpm packages
Arvados python SDK: do not blow up when HOME is not set.
Make run-deploy.sh a bit smarter about which compute nodes are actually
First set of improvements:
Explain the Keepstore's -permission-key-file argument and API server's
Add install dependencies for the SSO server.
Merge branch 'master' into 4186-install-doc-improvements
Merge branch '4186-install-doc-improvements'
Clean up *.pyc files between tests.
Improve config/application.yml.example for the API server and workbench:
Batch of improvements for the manual installation documentation:
Keepproxy just needs an anonymous token.
Fix typo.
Workbench: Change from yui to uglifier to compress javascript in
Add keepstore installation page.
Merge branch 'master' into 4186-install-doc-improvements-2
Clean up the installation overview pages a bit more.
Merge branch '4186-install-doc-improvements-2'
Fix broken link in docs. Duh.
Upload our docker images to the docker.io registry.
Make sure all executables we package are executable.
Make run-docker-tests.sh a bit smarter about detecting common failures.
Add first version of the easy-evaluation script.
Rename easy installation script. Update comments at the top of the script.
Add instructions for 'binary' docker installation.
Enable auto_setup_new_users in the docker API server image.
Enable auto_setup_new_users in the docker API server image.
system_user and anonymous_user should be excluded from
system_user and anonymous_user should be excluded from
The sso server docker image is not used by default at the moment, so
The sso server docker image is not used by default at the moment, so
The sso server docker image is not used by default at the moment, so
The sso server docker image is not used by default at the moment, so
When auto_admin_first_user is set to true (default false), the first
Small tweaks: fix example for running one API test in run-tests.sh;
Add unit tests for new auto_admin_first_user configuration setting.
Merge branch 'master' into 4383-easy-install
Update the behavior of the auto_admin_user configuration field a bit to
Add more tests so that we cover all possible combinations of the
Only try to install .config/arvados/settings.conf if we have a token to
Merge branch 'master' into 4383-easy-install
Install instructions: be a bit more explicit that arvdock is downloaded
Another documentation tweak based on review feedback.
Merge branch 'master' into 4383-easy-install
Merge branch '4383-easy-install'
Explicitly list the URL that get.arvados.org redirects to, for those
Add some basic documentation for the arv subcommands.
Add TCPKeepAlive and ServerAliveInterval ssh options to the .ssh/config
Merge branch 'master' into 4486-document-arv
Remove trailing whitespace.
Merge branch '4486-document-arv'
Fix broken doc link.
Remove some redundant lines from sdk/bin/arv
Make arv-tag print usage when --help is given. Also, fix typo in usage lines.
Update arv-tag --help text in arv subcommands documentation.
Fix sdk/go/keepclient test race condition
Make our dynamic dns update system for compute nodes more generic.
Also build a package for python-six.
Retry docker push a few times if it fails.
Be a bit smarter about handling the dynamic dns resolver records for
Package cwltool.
Also build a deb package for python-pyexecjs, which is a dependency for
Build a python-jsonschema package, because cwltool requires version 2.4
Do not blow up when another puppet process is already running during deploy.
Refactor deploy script a bit; make it less sensitive to transient errors
A few wording tweaks.
Remove spurious empty line
Reorganize the arv documentation a bit more, putting it all in one place.
run-tests: make the error message displayed when WORKSPACE isn't set a
Add third installation option for the python SDK: our debian/ubuntu packages.
Fix typo.
Wording improvements on the sdk cli installation page.
Make the 'check-environment' blurb stand out more from the regular text.
Link directly to the CLI SDK install instructions.
Add basic documentation for arv keep put and arv keep get.
Refactor the 'Uploading data' page.
Merge branch '4336-improve-upload-data-to-keep-doc'
Improve the installation instructions for keepproxy and keepstore - take sudo into account.
Merge branch '4186-install-doc-improvements'
Make sure that our go packages get generated with a version number that
Document arv keep ls in the cli sdk reference.
Pull the common-workflow-language code from the new
Fix typos in documentation.
Fix syntax error in workbench diagnostics_test_helper.rb
Docker: passenger only needs to support ruby, so specify that explicity
Add timing information to the docker tests.
Download bwa and samtools from a self-hosted mirror, sf.net downloads
Minor run-tests.sh refactoring:
Remove superfluous output while running sdk/cli tests through run-tests.sh
Skip two more CLI tests that need a running API server.
Merge branch 'master' into 4844-stricter-min-nodes-wip
Merge branch '4844-stricter-min-nodes-wip'
Add some more sanity checks in run-tests.sh, and refactor that part of the script
Cleanups:
Detect stale compute node records with the same IP address as the new
Address review comments:
Merge branch 'master' into 4887-invalidate-duplicate-ip-on-old-compute-nodes
Merge branch '4887-invalidate-duplicate-ip-on-old-compute-nodes'
Wording tweaks to the workbench profile page.
Make sure to build a new package for keepproxy if the go SDK has been updated.
Python version numbering is obscure. For our debianized Python packages,
Merge branch 'master' into 0000-workbench-user-profile-wording-changes
Automatically rotate and compress the test.log files for the api server
Fix filename when compressing old test logs.
Merge branch 'master' into 0000-workbench-user-profile-wording-changes
Merge branch 'master' into 0000-workbench-user-profile-wording-changes
Merge branch '0000-workbench-user-profile-wording-changes'
arv-ws should prefer system's CA certificates (if available).
A few tweaks based on review feedback.
Merge branch 'master' into 4340-arv-ws-fix-ssl-warning
Merge branch '4340-arv-ws-fix-ssl-warning'
Wording tweaks for doc.arvados.org.
Documentation cleanup: CLI SDK:
* Only source /etc/profile.d/rvm.sh if it exists and is a regular file
Make sure the performance testing gems are available when workbench is
We need tmp/cache to exist before we can run performance tests
Add a script to run our performance tests.
Revert "We need tmp/cache to exist before we can run performance tests"
Also add minitest and friends to the performance group.
Run the performance tests the documented way. This is possible after
Make sure tmp/cache exists before we run our performance test suite.
Fix typo.
Merge branch 'master' into 5200-anonymous-download
Merge branch '5200-anonymous-download'
Make pip install quiet, no point in printing loads of useless warnings
When run-tests.sh is invoked to only run the api server tests, do not
Fix typo in COPYING.
5220: whitespace cleanup.
Reflect deploy changes in run-deploy.sh
Fix typo on doc homepage.
Merge branch 'master' into 5310-arv-copy-by-pdh
Merge branch '5310-arv-copy-by-pdh'
Move the licensing info out of the second column and towards the footer of the page on the doc site.
Build arvados-data-manager deb.
Be consistent in the naming of the arvados-data-manager package and the binary in the package.
Follow the naming conventions for hostnames; add SSO server as a
A series of changes to the top nav bar to make it more compact.
Remove reference to as yet non-existant compute node setup page.
Merge branch 'master' into 5211-improved-top-navbar
Merge branch '5211-improved-top-navbar'
Add python-backports.ssl-match-hostname to the list of python packages
Add git.uuid.local.domain to the manual install prerequisites page on
More tweaks for the manual install prerequisites doc page.
Go SDK improvements:
Add a keepproxy docker image
Merge branch 'master' into 5529-add-keepproxy-docker-image
Merge branch '5529-add-keepproxy-docker-image'
Make sure to also push the keepproxy and sso images to the Docker registry.
Merge branch '5529-build-and-test-keepproxy-docker-image'
The easy-docker-install method should also get the keepproxy and sso
Documentation: installation pages improvements. Be clear about the
Make arvdock abort with a nice error message if docker or curl can not
Be consistent in our package and binary naming:
run-tests.sh can now run the arv-git-httpd tests too.
Docker: fix the build of the sso docker container.
Make MAX_SLOTS a configurable parameter, and rename it to max_compute_nodes.
Improve comment in application.default.yml based on review feedback.
Merge branch '5926-parameterize-max-slots'
Fix docker image build for SSO server after refactoring work in #5829.
Fix install instructions: create arvados repository.
Update SSO server documentation to match current reality.
Even more SSO server install instructions.
Reorder SSO install documentation so that the rake db:setup call does
Add instructions to generate assets to SSO installation page.
Fix sso docker container build, and enable local account creation for
Check in docker/sso/database.yml.in, forgotten in commit ea7da293b0a10baef18cfc6e107c87a9c3b09546.
Documentation fix (quoting) for the 'Add an Arvados repository' page.
Be consistent in our package and binary naming:
run-tests.sh can now run the arv-git-httpd tests too.
Add installation instructions for compute nodes; update the installation
Add GPG key for RVM installation in the doc.
Build API server and Workbench debian packages.
Fix license names in API and workbench packages.
Default BUILD_BUNDLE_PACKAGES to false.
Simplified run-deploy.sh now that we have moved away from Capistrano.
Name change for the arvados-api-server and arvados-workbench packages. The slim versions are now just
Name change for the arvados-api-server and arvados-workbench packages - part 2.
Be sure to include the pre-generated assets (tmp/cache/assets) in the
Merge branch 'master' into 6096-package-rails-apps
Merge branch '6096-package-rails-apps'
Fix links to workbench on run-tests.sh. Also shut up the PyYAML build warnings.
Add instructions to the 'Create standard objects' page to create a cluster-wide readable project for standard Arvados Docker images.
Make the run-deploy.sh script much more general. It can now be used to
Store the full git hash of the checkout in the arvados-src package, in a file called git-commit.version
Add rpm generation support to run-build-packages.sh
Add very rudimentary support for pushing the rpm packages into a yum
Do not build and upload gems that already exist on rubygems.
Fix asset generation in the workbench package. Make sure the build step will error out if asset generation fails for whatever reason.
Regex match fixes in the character class match for $FPM_RESULTS:
Add ubuntu1204 target to run-build-packages.sh
Drop built packages into packages/$TARGET instead of the old 'debs' directory.
Update comment references from 'debs' to 'packages'.
Make sure to also build a backported package for python-llfuse. We need
Add rsync to the repo update command in run-build-packages.
Add arvados_pam.py and shellinabox pam configuration file.
Make run-deploy.rb install the correct version of the arvados/jobs
Small fixes for run-deploy.sh
Fix up git log statements:
Add dockerfiles for Debian7 and Ubuntu1204
Fix typo.
Fix path for arvados_pam.py
Rename file
Merge branch 'master' into 6384-libpam-arvados
Whitespace fixes
Add libpam-arvados package build to run-build-packages.
Add dependency on libpam-python to the libarvados-pam package.
Build arvados/jobs docker image separately from the others.
Make the arvados/jobs image leaner.
Make sure that mkimage-debootstrap.sh does not try to do an interactive
Make sure that mkimage-debootstrap.sh uses -f when tagging docker
Merge branch 'master' into 6569-smarter-jobs-image
Make sure to add python-virtualenv to the Docker arvados/jobs image.
Add debian8 as a build target.
Merge branch 'master' into 6569-smarter-jobs-image
Merge branch '6569-smarter-jobs-image'
run_deploy.sh improvements:
Make sure we have a conference/environments/production.rb file so that
Build python3-websocket-client packages, which is now a dependency for
arvdock realclean/deepclean should not delete images tagged as arvados/build.
Add dockerfiles/build-all-build-containers.sh script
freight is already smart enough to do atomic updates.
We need to force-tag the arvados/jobs docker image.
Make sure jessie packages end up in the jessie repository.
run-deploy.sh improvements: remove the need for a .ssh/config entry for
Fix logic bug in run-deploy: deploy api server first before gathering
Make sure that we build arvados-src package with the correct version of
Performance optimization for the run_upload_packages.py script: only
Make sure to also deploy switchyard.
Improve compute node setup script.
Some more improvements based on review feedback.
Merge branch 'master' into 6942-improve-compute-node-setup-script
Merge branch '6942-improve-compute-node-setup-script'
Stop building broken debs for our gems. We don't use them and they have
6781: Add arvados-login-sync (was update-shell-accounts.rb)
6781: Fix scope for user-sync script in documentation.
Update/clarify package assumptions for api server and workbench.
Doc fix: protect against accidentally pulling in all versions of the
run-build-packages-all-targets.sh should abort of WORKSPACE is not set.
run-build-packages-all-targets.sh should abort of WORKSPACE is not set.
Optimization: do not run bundle install for the API server package when
Optimization: do not run bundle install for the API server package when
Add build script for the new Arvados SSO server package.
First batch of improvements based on review feedback.
Split out shared functions into a new run-library.sh.
Merge branch 'master' into 6939-package-sso-server
Merge branch '6939-package-sso-server'
Fix documented scope needed for run-deploy.sh
arv-copy: Make the help text for the --src and --dst options more easily
Another tweak to the scopes required for run-deploy.sh to do its thing.
Make sure the library can always be found.
Update docs to accomodate URL change for the Arvados redmine
Make the build step for our libcloud package a lot less noisy by default.
Fix the test that avoids running puppet multiple times on installations
First commit.
Add prerm and postrm scripts, various cleanups.
Remove bash-ism from arvados-sso-server.postinst.
Initial version of run-test-packages-sso.sh
Add support for CentOS6 to the improved SSO packages.
Ruby 2.1.7 is out, update documentation accordingly.
A number of additional tweaks.
Update the SSO installation documentation to reflect the new and
More improvements for the SSO server installation guide.
Another tweak for the SSO server installation guide: the PostgreSQL
More fixes for the SSO server packages.
More language tweaks.
Merge branch '7330-postinst-for-sso-server'
Merge branch 'master' into 7330-improved-sso-package
Merge branch '7330-improved-sso-package'
Ensure proper permissions on /usr/local/arvados/src in the arvados-src package.
Make it possible to pass --debug through to the script run inside the Docker
Make sure to run the arvados-api-server-upgrade.sh and
Shut up dpkg-scanpackages warnings.
SSO installation doc fix: to run rails console, you need to be in the
Make sure bundle is installed in arvados-api-server-upgrade.sh,
Fix path issue when running run-build-packages-all-targets.sh from
Add python-pip to the Arvados build images.
Update redmine links in README.
Make sure that the CentOS 6 test script actually does the yum update call.
arvados-api-server-upgrade.sh fix: do not try to chmod the cache
Default run-deploy.sh to ssh port 22, and provide a command line option
Make sure not to silence package installation during the test-packages
8189: Fix too-small page size in "fetch all pages" code.
Doc improvement: explain in more detail what the -serialize argument to
Merge branch '8248-explain-serialize'
Fix typo.
Make sure not to accidentally include config/application.yml and
Merge branch 'master' into 8066-git-init-from-postinst
Some review fixes.
Merge branch '8066-git-init-from-postinst'
Make sure not to accidentally include config/application.yml and
Make our API server packages for debian-based distributions depend on
Pin package builds of the google-api-python-client and oauth2client
Build arvbox instead of the old docker images.
Build package for arvados-cwl-runner.
Get the go binary from a http url instead of https, so that our
Build dependencies for arvados-cwl-runner, automatically.
Make the build of the schema_salad packages more quiet.
Build script improvements:
More small build script improvements.
Clean up build/run-build-docker-jobs-image.sh a bit.
Fix sdk/cwl tests after commit 9df4d7da24ef8be639af1ab806cf833ee544fb45
Build script fix: make sure that run-build-docker-jobs-image.sh always
Fix a few more dependencies for the python-arvados-cwl-runner package.
Bump up the iteration for the python-arvados-cwl-runner package so that
Fix package build and test of the arvados-cwl-runner package for
Build a package for crunchrunner.
Build newer cwltool version.
Add build/run-build-packages-python-and-ruby.sh script to handle upload
A few more fixes for run-build-packages-python-and-ruby.sh, and a small
Fix package building by pinning docker-py to version 1.7.2
Package crunchstat-summary.
Fix a bunch of misspellings in our Go code (all in comments).
Fix building packages: work around pip having a mind of its own with
Fix centos6 packages build.
Fix inverted test for pypi/gem upload logic. Make upload more verbose.
When running run-build-packages-python-and-ruby.sh with --debug, pass
Remove hardcoded -v in call to run_upload_packages.py
Build distribution packages for the version of python-cwltool that
Build distribution packages for the version of python-schema-salad that
Package ruamel.yaml, which is a new dependency of schema-salad.
Fix bug in run-build-packages-one-target.sh: make sure to escape the *
Add dependency for ruamel.yaml to the build list.
Fix centos6 package build (ruamel.yaml package building arguments for fpm).
Pin the html5lib package we build for centos6 and centos7 to version
9639: Make it possible to run unit, functional and integration tests for
Merge branch '9639-split-workbench-tests'
We have a new build job at ci.curoverse.com. Update the build status
Document how to download and tag the arvados/jobs Docker image.
Merge branch '9677-update-install-docs'
Use our cached copy of phantomjs-1.9.7 instead of pulling it from https
Small naming fix for the config variables.
Small naming fix for the config variables.
Make sure that we can set ARVADOS_KEEP_SERVICES via the
Merge branch '10041-test-arvados-keep-services'
Add the new KeepServiceURIs option in the crunch-dispatch-slurm
Do not try to chown_R the gid of .ssh, because we're making an
Pin the python-daemon backport for centos7, newer versions have
Explicitly list all root-level directories in this repository in the
Package keep-exercise.
Fix typos.
Fix typo in comment.
README.md: add link to the Go Report Card for Arvados.
We should not hardcode dependencies in our packages for secondary
Add a mechanism to the arvados-api Gemfile to load 'plugin' gems.
Remove debian7 from the build directory.
Merge branch '10785-remove-debian7'
Merge branch 'master' into 10786-remove-centos6-from-build-directory
Remove CentOS6 support from the build directory.
Remove additional CentOS6 files.
Merge branch '10786-remove-centos6-from-build-directory'
run-build-packages-all-targets.sh: Make sure to build all packages once not twice.
In our rails postinstall package script, preserve newlines when looking
Merge branch '10838-fix-rails-package-postinst'
Fix run-tests.sh --only option for the workbench suite.
Bump up rails package iteration.
Pin the python-pathlib2 dependency to 2.1.0, because 2.2 introduces more
run-build-packages-one-target.sh should respect the --debug flag.
Build the python-futures package, version 3.0.5, for our supported
Refactor the build-packages scripts a bit to make them much more
Merge branch 'master' into 10858-build-packages-improvements
Handle rpm-based backports correctly.
Fix the shortcut that tests the need of rebuilding the arvados-src
If the fpm run fails, always print out the failed command so that we
Documentation updates to reflect the end of support for Debian Wheezy and CentOS 6.
More documentation updates to reflect the end of support for Debian
Merge branch 'master' into 10800-update-doc
Merge branch '10800-update-doc'
Fix rpm architecture in the test_package_presence function. Expand on
10858: Equivalent commit for 10881: Add python-futures as explicit
Merge branch '10858-build-packages-improvements'
Remove references to debian7 and centos6. Also remove some hardcoded
Remove deprecated --force-yes flag during package testing, replace with
Remove old debian7 package testing file.
Add support for Ubuntu 1604 ("xenial") to our build scripts.
Merge branch '10980-add-ubuntu1604-support'
Doc update: add trademark symbol next to the word 'Arvados' in 2 places.
Merge branch 'master' into 10797-ruby-2.3
Update golang version to 1.7.5. Also make the equivalent changes to the
build improvement: really include apps/workbench_functionals when
Fix package test for ubuntu1204.
10988: Document our Ubuntu 16.04 support.
We stopped building packages for Ubuntu 12.04, it is now officially EOL.
Our API server and Workbench packages depend on 2 gems that compile
Fix Workbench package dependency.
The Workbench package has a build-time dependency on tzdata for
Be smarter about how we handle packages built in previous runs.
Fix typo.
Add AUTHORS, update COPYING and license file names.
Add Chen Chen to AUTHORS.
Fix typo.
Make it a little more obvious which arvbox docker images are being built.
Workbench shouldn't error out when config files are missing if
For now remove the call to rake npm:install from arvbox, it depends on
Merge branch '12122-rails-groups-assets-fix'
Workbench: when showing the detail for script_parameters in the pipeline
Apply a few gofmt -s fixes.
Fix typo in python documentation.
Do not blow up the provenance graph if a PDH used in the workflow no
Merge branch '12316-fix-provenance-graph'
Update documentation to reflect that we are packaging Debian 9
Run cwl conformance tests with the containers api.
arvbox: bump ruby version up to 2.3.5
arvbox: jobs api: crunch-dispatch: stop excessive stat of /etc/localtime
arvbox: containers api: make crunch-dispatch-local poll more agressively
arvbox: run the rails code in production mode when running in localdemo or
arvbox: use overlay2 inside arvbox.
Merge branch 'master' into 12355-make-cwl-conformance-tests-faster
Merge branch '12355-make-cwl-conformance-tests-faster'
bryan (16):
Change _config.yaml to point to arv-copy.html.textile.liquid
Added documentation for copying collections and pipeline templates and instances
changed formatting and added words
changed arv-copy expectations and moved around words in the html file
changed formatting of cluster1 and cluster2
Ready for review
added _arv_copy_expectations
added png and need to switch to 6014 to grab the url
Fixed some changes but need to add copy git repos
Fixed more comments
Fixed syntax
Fixed more syntax errors
added note #13
Note #15
Note #16
Merge branch '5288-arv-copy-documentation' refs #5288
mishaz (104):
Added to Go SDK documentation:
Moved code to seperate file for syntax coloring.
Included included file.
Moved file.
Combined all examples into one big, runable source file with syntax coloring.
Merge branch 'update-go-sdk-documentation'
Committing just to get some input on my Go Style.
Updated code in response to Tim's Review.
Responded to Tim's comments.
Moved manifest package from services/datamanager to sdk/go
Added params struct for GetCollections
Fixed bug where error was dropped on the floor!
Created helper method to deal with iterating through list returned by SDK.
Added some code to deal with keep disks.
Created sdk util package and moved helpful methods there.
Moved Keep code to its own package.
Merge branch 'master' of git.curoverse.com:arvados into 3408-production-datamanager
Switched to using structs rather than maps to hold SDK responses and code is much easier, much less noisy parsing code.
Reported scanner errors, which I forgot to do before (and was hiding real problems).
Added test to show that our code fails on long manifests.
Added tests to check that we're iterating on manifest lines correctly and handling blank lines in manifests.
Deleted unused type.
Added flags to read data manager token from a file.
Modified IndexHandler to match TrashHandler and PullHandler
Merge branch '4197-remove-enforce-permissions-check-from-index-handler'
Started reading response from keep server.
Started reading index from keep servers.
Added reporting of disk usage. This is the Collection Storage of each user as described here: https://arvados.org/projects/arvados/wiki/Data_Manager_Design_Doc#Reports-Produced
Long overdue checkin of data manager. Current code runs, but uses way too much memory and eventually crashes. This checkin includes heap profiling to track down memory usage.
Added blockdigest class to store digests more efficiently. This has the nice side effect of reducing how many string slices we use from the SDK, so the large string can get garbage collected once we remove other usages.
Changes to manifest that I forgot to add to previous checking.
Switched from strings to BlockDigests to hold block digests more efficiently. Started clearing out manifest text once we finished with it. Made profiling conitional on flag (before it crashed if not provided). Added final heap profile once collections were finished.
Started parsing modification date as a timestamp instead of leaving it as a string.
Added string copying to try to reduce memory usage, didn't seem to work. Cleaned up logging (and logging logic) so that we only see one line per batch.
Fixed heap profile writing so that we overwrite previous heap profiles rather than adding to them. Minor cleanup too.
Started focusing on Keep Server responses again. Switched to using blockdigest instead of strings. Added per block info so that we can track block replication across servers.
Added logger to write log messages that grow over time. Not working yet.
Fixed bugs in logger, changed interface some, added documentation.
Added support for MinimumWriteInterval.
Started using Logger in data manager and collections.
Added ability to turn off logging by passing an empty string as the event type.
Added structure to data manager log entries, grouping similar fields.
Added memory alloc in use to stats exported to log. Also added EditHooks to Logger, enabling users to add functions to get called on each Edit() call.
Switched batch size to 50. Added logging of batch size.
Switched Logger edit hooks to write hooks so they'll trigger less often.
Added ForceRecord() method to enable writing remaining log changes before exiting.
Added recording of fatal errors to logger.
Merge branch 'master' of git.curoverse.com:arvados into 3408-production-datamanager
Started reading collections and keep data in parallel. Moved some logic from datamanager.go to collections.go. Added logging to end of run.
Moved some logic from datamanager.go to keep.go.
Created loggerutil to hold common datamanager logger code. Moved FatalWithMessage to it.
Started logging in keep.go. More work to be done.
gofmt'd all my source code. No other changes.
Removed channel awareness from keep.GetServerContents().
Broke keep.GetServerContents() into smaller functions.
ran gofmt
Finished adding logging to keep.GetServerContents but have not tested fully yet.
Added Logger.MutateLog() on Tom's suggestion. Tried it out in one instance to make sure it works.
Added comment, ran gofmt.
A bunch of changes, most in response to Peter's review.
Renamed BlockDigest's ToString() to String() to implement fmt.Stringer() interface so that we get more readable error messages when structs contain BlockDigests.
Improved erorr message to make it clear what's a size and what's a timestamp.
Now fetch Keep Server Status and record it to the log. Renamed some fields and added a comment for a potential improvement to decrease lock contention.
Renamed timestamp fields to begin with "time_"
Updated logger to do all work in a dedicated goroutine, so we don't need to worry about locking. Small changes to calling code.
Moved some logging code from datamananager to loggerutil.
Added different event types for started, partially complete and final log entries.
More changes in response to Peter's review.
Merge branch '3408-production-datamanager' refs #3408
Changes to allow datamanager to run indefinitely:
Merge branch 'master' into 3408-production-datamanager refs #3408
Merge branch '3408-production-datamanager' refs #3408
Added BlockToReplication field to collection.ReadCollections.
Added flags to write network data and then read it back. This is useful to speed up development, but should not be used in production since data will be stale.
Fixed bug where I wasn't copied saved data.
Added block to collection index map. Started using collection index to save memory over using long uuid strings to identify collections.
Added lots of unit tests.
Fixed bug in BlockSet.ToCollectionIndexSet.
Cleaned up test.
Added some tests.
Added a couple helper methods to remove boilerplate from unittests, to make them more readable.
gofmt cleanup.
Added logger util GetOrCreateMap() and started using it everywhere.
Started logging memory reserved in addition to memory used.
Rewrote ArvadosApiError as RemoteApiServerError:
Added tests to confirm that error messages indicate that error is remote.
Renamed RemoteApiServerError to APIServerError and NewRemoteApiServerError to newAPIServerError.
Switched collection_test.go to use the excellent gocheck framework after seeing it in arvadosclient_test.go.
Removed a comment that was accidentally copied in.
Made HostPort() an explicit method to print the HostPort, so that String() can change in the future.
Added code for generating pull lists.
Merge branch 'master' into 3408-production-datamanager
Fixed compilation errors after merging master.
Modified pull list code to change json output to match the specification at https://arvados.org/projects/arvados/wiki/Keep_Design_Doc#Pull-List
More reformatting of unittests to improve readability.
Changes in response to code review.
Added string to error message to help with debugging.
Added size to block locators, touching most of the code.
Renamed BlockToReplication BlockToDesiredReplication.
Merge branch 'master' into 3408-production-datamanager
Fixed outdated comments in arvadosclient.Discovery().
Added unit test for protocols in From field.
Now reading default replication level from api server.
Added comments about zero-replica blocks in underReplicated.
peter (9):
Notes on new user documentation
Fuse driver works for mounting collections and reading files. Tested with jlake collection, spot checked m5dsum values of several files which matched the md5sum catalog in the jlake collection.
Robustness improvements
2883: Adjust timestamp display based on timezone and locale.
2883: Adjusted task type column width. Commented out code for working with
2883: Added filtering for crunchstat, slurm log lines.
2883: Fixed variable being used before it was declared, and fixed assignment of
2883: Changed "task-output" category to "task-print" and "output" to
2883: Fixed select all/select none messing with the "show task" radio buttons.
radhika (2308):
2525 - Arvados Java SDK. API discovery and call methods working. Need logging, further cleanup, and more testing.
2525 - Arvados Java SDK. Eclipse project file. Since we are using maven, eclipse users may find it hard to setup eclipse project without this file.
2525: Cleanup and additional tests.
2525: Add sdk/java/target to gitignore list
2525: Address multithreading concerns in the sdk.
2525: Add pipeline template create testing. Use maven resources for input files.
2525: Use log4j for logging from java SDK
2525: What was that typo!!
2525: Add ArvadosSDKJavaUser.java to serve as an example implementation of SDK usage example.
2525: Minor updates to log4j configuration of log line format.
2525: README file to serve as getting started guide.
2525: when using main method, print output to console.
Merge branch 'master' into 2525-java-sdk
2525: Java SDK does not need com.google.oauth-client library
525: Update the .classpath to reflect the latest trimmed down version of pom.xml
Merge branch 'master' into 2525-java-sdk
2525: add apiVersion to constructor args
2352: Update arv-run-pipeline-instance to handle RunningOnClient and Paused states
Merge branch 'master' into 2352-use-state
2352: Getting null pointer exception after merging in master!!
Merge branch 'master' into 2525-java-sdk
2525: token, host name, and host insecure can also be passed to constructor similar to other SDKs. If so, the environment variables are ignored and passed in values are used.
2525: call method invocation should match arvados api documentation, instead of discover document format.
2525: call method invocation should match arvados api documentation, instead of discover document format.
2525: call method invocation should match arvados api documentation, instead of discover document format.
2525: a little cleanup of variable names
2525: delete object created during test.
2525: Rename ArvadosSDKJavaUser.java as ArvadosSDKJavaExample.java
2525: override the default name for the generated package
2525: More tests
2525: updated README
2525: readme and test updates
Merge branch 'master' into 2525-java-sdk
2525: Add support for array request parameter type
2525: add float handling to request parameters
2525: add map parameter type handling to request processing
2525: small refactoring
2525: return map instead of json string for the call method.
Merge branch 'master' into 2525-java-sdk
2525: Better handling of unsupported api name and version
2525: Remove unused imports
2525: Add java SDK to documentation site.
2525: Now that we have the doc page updated with Java SDK details, updated the README file to
Merge branch 'master' into 2352-use-state
2352: when using arv-run-pipeline-instance, set state to RunningOnClient.
Merge branch 'master' into 2352-use-state
2352: initial tests for arv-run-pipeline-instance
Merge branch 'master' into 2525-java-sdk
2525: use -Dmaven.test.skip=true when packaging the SDK.
2525: correct indentation for pom.xml
Merge branch 'master' into 2525-java-sdk
2525: Address review feedback.
Merge branch 'master' into 2525-java-sdk
Merge branch 'master' into 2525-java-sdk
2525: Add ArvadosSDKJavaExampleWithPrompt.java to serve as an easier tool to make quick calls to API server.
2525: improve response printing in the examples for readability.
Merge branch 'master' into 2525-java-sdk
2525: example
2525: example param parsing
2525: more testing. also "=" works in place of "is_a" for a filter.
2525: update java sdk documentation to include ArvadosSDKJavaExampleWithPrompt
2525: refactor response printing
2525: Expand prompt example to list the available resources and methods. This can help enhance user experience.
2525: list resources, methods, and parameters in prompt example to improve user experience.
2525: prompt example
Merge branch 'master' into 2525-java-sdk
Merge branch 'master' into 2352-use-state
2352: when --submit is used, set the state to Ready, not RunningOnClient.
2352: set active to nil when Ready
2352: look for only RunningOnServer in queue method in pipeline_instance
2352: fix issue with setting active during save
Merge branch 'master' into 2352-use-state
2352: when a running pipeline_instance is stopped in workbench, put it in Paused state.
Merge branch '2352-use-state'
2525: getAvailableParametersForMethod includes properties from request also.
2525: update available params method
Merge branch '2525-java-sdk'
2767: Fix subtasks 2395, 2396, 2397, 2398, 2399.
Merge branch 'master' into 2767-doc-updates
2352: Update the workbench and arv-run-pipeline-instance script to use pipeline_instance status instead of active and success flags.
2352: add a info level log statement in pipeline_instance.rb if active or success flags are updated. The expection is that this should no longer happen.
Merge branch 'master' into 2352-remove-attrs
2352: Address review feedback: (1) input can be edited even in Ready state, (2) pipeline cleanup to set state to Paused if RunningOnClient.
2352: log statement to include uuid if either active or success found in changed attributes.
Merge branch 'master' into 2352-remove-attrs
Merge branch 'master' into 2767-doc-updates
2767: update tutorial page to add a cd statement before git cloning
Merge branch 'master' into 2352-remove-attrs
2756: Display event log on pipeline_instance event log udpates.
Merge branch 'master' into 2756-eventbus-in-workbench
2756: Use event_log.js javascript asset to subscribe to websockets
Merge branch 'master' into 2756-eventbus-in-workbench
2756: check if browser supports websockets before creating the websockets connection.
2756: improve the data displayed in event log display area.
2756: store websocket in window to facilitate reuse within page. also, remove elementId from map when leaving page.
Merge branch 'master' into 2756-eventbus-in-workbench
2756: subscribe for event log messages on ajax:complete or ready.
Merge branch 'master' into 2756-eventbus-in-workbench
2756: trigger event when object is updated. process the messages on trigger.
2756: ws url storage does not have to be managed by each page subscribing for events. also, html class names to include arv in them.
2756: event log div placement corrected.
2756: much cleaner and leaner
2756: use meta tag to provide websocket url to event_log js file
Merge branch 'master' into 2756-eventbus-in-workbench
Merge branch 'master' into 2756-eventbus-in-workbench
2525: handling of array objects was not quite correct. it would not have worked if a pure array object that is not filters is passed in.
2756: write log info from redis into log table. this will help us read this log info via the event log websockets.
Merge branch 'master' into 2756-eventbus-in-workbench
2756: trigger event for any elements that contain this log event's object_uuid in it's data-object-uuids.
Merge branch 'master' into 2756-eventbus-in-workbench
2756: listen for event logs on job uuids.
Merge branch 'master' into 2756-eventbus-in-workbench
2756: show log for this pipelines jobs.
2756: display log history for a running pipeline.
Merge branch 'master' into 2756-eventbus-in-workbench
2756: display log file for a finished job.
2657: when displaying log in window, handle new lines
2756: need a new line between log file links in display
2756: component name should be plain, not styled with label, not a link to the job.
Merge branch 'master' into 2767-doc-updates
2756: use stderr for event_type instead of transient-log-entry
2756: rescue any exceptions. do not write to redis. use log properties, instead of summary to store the potentially big strings.
Merge branch 'master' into 2756-eventbus-in-workbench
Merge branch 'master' into 2767-doc-updates
Merge branch 'master' into 2767-doc-updates
Merge branch 'master' into 2756-eventbus-in-workbench
2756: fix issue from merge conflict
2756: log write
2756: added a link to job when a job is chosen
Merge branch 'master' into 2756-eventbus-in-workbench
2678: update a-r-p-i to set the owner_uuid on the newly created job when the pi belongs to a group. also, update crunch-dspatch to create the log lines with the group's owner_uuid.
Merge branch 'master' into 2678-owned_by
Merge branch 'master' into 2756-eventbus-in-workbench
Merge branch '2756-eventbus-in-workbench' into 2678-owned_by
Merge branch 'master' into 2525-java-sdk
Merge branch 'master' into 2767-doc-updates
Merge branch 'master' into 2525-java-sdk
Merge branch 'master' into 2756-eventbus-in-workbench
Merge branch 'master' into 2678-owned_by
Merge branch 'master' into 2525-java-sdk
2678: upon Tom's feedback, setting owner_uuid of the pi whether or not the pi belongs to a group.
2525: add some javadoc comments
2756: update the ui to show job progress, and links in one column
2756: fix typo in helper method signature
2756: log span title
Merge branch 'master' into 2756-eventbus-in-workbench
2756: fix error where partial lines were potentially logged
Merge branch '2756-eventbus-in-workbench' into 2678-owned_by
Merge branch 'master' of git.curoverse.com:arvados
An equivalent of this script is now at services/api/script. This was done as part of Feature #2272.
Merge branch 'master' of git.curoverse.com:arvados
2903: Remove active and success flags from pipeline_instance. Update all other files as necessary.
2903: active to state usage
Merge branch 'master' into 2903-remove-pi-active-and-success
2903: during validation methods, set state instead of success
Merge branch 'master' into 2903-remove-pi-active-and-success
2929: provenance graph is cutoff on the left. Removed the absolute positioning on the div to resolve the issue.
2919: update integration test to assert provenace graph div
Merge branch 'master' into 2919-provenance-graph-cutoff
2895: remove users.event_stream api
2895: remove jobs.log_tail_follow api since we are now using websockets instead of redis
Merge branch 'master' into 2895-no-more-redis
2895: remove redis gem
Merge branch 'master' into 2895-no-more-redis
Merge branch 'master' into 2903-remove-pi-active-and-success
2903: merge master
2919: set the left postitioning for the provenance graph calculated based on the left-nav size
Merge branch 'master' into 2919-provenance-graph-cutoff
Merge branch 'master' into 2903-remove-pi-active-and-success
2903: rename db migrate script with clear description
Merge branch 'master' into 2919-provenance-graph-cutoff
2871: preload links helper method is added to workbench application_controller.
Merge branch 'master' into 2871-preload-objects
2871: add helper method to get "n" number of objects of specific dataclass
Merge branch 'master' into 2895-no-more-redis
2895: remove arvados.jobs.log_stream from arv
2871: add preload methods for collections and logs in application_controller. Use the preload methods in dashboard.
Merge branch 'master' into 2871-preload-objects
Merge branch 'master' into 2895-no-more-redis
Merge branch 'master' into 2871-preload-objects
2871: get collections methods return all collections for the given uuid.
2871: rename get collections and log collections method names to match the format of links method.
2871: metadata page updated to use some preloaded collections and links; not fully done yet.
Merge branch 'master' into 2871-preload-objects
Merge branch 'master' into 2919-provenance-graph-cutoff
Merge branch 'master' into 2871-preload-objects
2871: add link head_uuids to the list of to be preloaded uuids
2871: metadata tab using preloaded objects. wow, i can't believe how many api calls are cut down with this :)
2871: improved link_name retrieval
2871: add preload_objects_for_dataclass method to application_controller
2871: preload_objects_for_dataclass for pipeline instances in dashboard
Merge branch 'master' into 2871-preload-objects
2871: preload objects for dashboard
2871: some more comments
2871: remove trailing white space
2871: handle if there is no friendly_link_name in link
2872: fix test
2872: update tests to match the new ui
2872: all tests are updated to reflect the new ui.
Merge branch '2872-folder-nav' into 2872-folder-nav-tests
2872: update test to meet the current ui updates. the pipeline instances test still failing; possible error in ui where the Run button is enabled even when input is not provided.
2872: updated test to work with the new ui.
Merge branch '2872-folder-nav' into 2872-folder-nav-tests
2872: when a pipeline instance still needs input, the Run button should be disabled.
Merge branch '2872-folder-nav' into 2872-folder-nav-tests
2872: added graph and run button checks back. the test does not pass right now due to bug in graph tab display in pipeline instance display.
Merge branch 'master' into 2871-preload-objects
Merge branch '2872-folder-nav' into 2872-folder-nav-tests
2872: test now passing since the UI issue with "Resume" button is addressed.
2872: add test for a pipeline instance with job associated so that the provenance graph can be tested.
2872: added a test that creates a pipeline instance from within a folder and runs it.
2872: updates to the new test that creates and runs pipeline instance from within a folder.
Merge branch 'master' into 2871-preload-objects
2871: add input checks for helper methods. start adding functional tests for the helper methods.
2871: add tests for preload_collections_for_objects helper method
2871: add tests for get_n_objects_of_class helper method
2871: test invoking methods a second time to retrieve preloaded data
2871: more tests
2871: more testing
2871: preload_log_collections tests
2871: test preload_objects_for_dataclass helper method
Merge branch 'master' into 2871-preload-objects
2871: trailing whitespace
Merge branch 'master' into 2871-preload-objects
Merge branch 'master' into 2871-preload-objects
2871: add andand to prevent error
closes #2871
2871: fix the typo caught by integration test
refs #2871
2896: add arv-ws cli tool to arv.
Merge branch 'master' into 2896-websocket-cli
2896: Add test file. Currently, the test only checks that arv-ws command is available and the options.
2896: incorporate review comments: add logging, enhance creating filter object.
Merge branch 'master' into 2896-websocket-cli
2896: add logging to events.py
Merge branch 'master' into 2896-websocket-cli
Merge branch 'master' into 2896-websocket-cli
closes #2896
3076: add Help dropdown to topnav.
3076: help menu class name
3088: Show created at timestamp instead of Age
Merge branch 'master' into 3088-project-chooser-on-run-pipeline-template
3088: Run this pipeline on a template shows project chooser
3088: set the owner uuid on the newly created pipeline instance to the selected project.
3088: show project chooser for run this pipeline on a template
3088: the other right
3088: so sweet to have a test AND one that fails when code is updated.
2659: split server side implementation into a separate branch to facilitate better testing.
Merge branch 'master' into 2659-anonymous-server-side
Merge branch 'master' into 3088-project-chooser-on-run-pipeline-template
Merge branch 'master' into 3076-topnav-help
Merge branch 'master' into 2659-anonymous-server-side
2659: remove the share_the_aproject_with_all_groups fixture in links.
2659: look for scopes and expires_at when getting anonymous token. also, update create anonymous user to check for group link even when the user exists.
Merge branch 'master' into 2659-anonymous-server-side
Merge branch 'master' into 3088-project-chooser-on-run-pipeline-template
refs #2659
Merge branch 'master' into 3088-project-chooser-on-run-pipeline-template
3088: address review feedback -- change the icon positioning on the Run button.
closes #3088
Merge branch 'master' into 3076-topnav-help
closes #3076
3106: display spinner when modal dialogs are communicating with server.
Merge branch 'master' into 3106-modal-loading-indicator
2380: move ssh instructions for windows env into a separate page.
2380: rearrange intro
2380: separate windows env instructions into a separate page.
2380: windows instuctions in a separate page
2380: correct the section for setting the windoes path environment variable.
Merge branch 'master' into 2380-ssh-doc
Merge branch 'master' into 2380-ssh-doc
3055: add explanation to the example commands in the advanced tab.
3140: Move tab line buttons, remove show filtering widget
3140: add additional content tabs to pane list
3140: add Other_objects to pane list
Merge branch 'master' into 3055-advanced-tab-explanations
Merge branch 'master' into 3055-advanced-tab-explanations
refs #3055
Merge branch 'master' into 3140-project-content-tabs
oops
3140: add partials for all the content tabs
Merge branch 'master' into 3140-project-content-tabs
Merge branch 'master' into 3106-modal-loading-indicator
3106: spinner to the left to avoid shifting down elements when it is visible
Merge branch 'master' into 3106-modal-loading-indicator
Merge branch 'master' into 2380-ssh-doc
2380: ssh to SSH where applicable
2380: minor doc updates
2380: minor doc updates
2380: more doc updates
Merge branch 'master' into 2380-ssh-doc
Merge branch 'master' into 3140-project-content-tabs
Merge branch 'master' into 3140-project-content-tabs
closes #3140
Merge branch 'master' into 2380-ssh-doc
2380: more updates and also correct the link to unix page from within the windows page
2380: minor updates
2380: rename ssh-access.html.textile.liquid as ssh-access-unix.html.textile.liquid
2380: one more update
3140: revert multiple content tabs work
refs #3140
2380: Extract common documentation items from Unix and Windows environments into include files.
2380: more updates
2380: update the add key image and text
Merge branch 'master' into 2380-ssh-doc
Merge branch 'master' into 2380-ssh-doc
closes #2380
2380: ssh doc references are updated to point to the right env.
refs #2380 Needed to updated all docs referencing to SSH documentation.
3203: Update "Accessing Arvados Workbench" page.
3203: Update the pipeline tutorial page to run pipeline from a project.
Merge branch 'master' into 3203-workbench-tutorial
3203: fix doc issues with compute references.
3203: fix doc issues with old dashboard references.
Merge branch 'master' into 3203-workbench-tutorial
Merge branch 'master' into 3203-workbench-tutorial
closes #3203
Merge branch 'master' into 3106-modal-loading-indicator
3106: show the spinner in the middle of the popup header
3140: address tab content mangling issue when infinte scrolling is applied to tabs due to large data.
3140: infinite scrolling in tab content display
Merge branch 'master' into 3140-project-content-tabs
Merge branch 'master' into 3140-project-content-tabs
3140: use correct id for scroller
3140: search in a tab should update results for that tab
Merge branch 'master' into 3106-modal-loading-indicator
closes #3106
Merge branch 'master' into 3140-project-content-tabs
3140: infinite-scroller
3140: apply infinite scrolling if the container is visible, when working with multiple containers in window.
Merge branch 'master' into 3140-project-content-tabs
3140: this file is not longer used. instead, other files are introduced to replace it.
3138: add wiselinks gem
3160: use blank name when copying an unnamed pipeline instance.
3160: use blank name when copying any object with no name, not just pipeline instances.
Merge branch 'master' into 3160-copy-unnamed-pipeline
closes #3160
3140: fix tests to match the project content display related ui updates
Merge branch 'master' into 3140-project-content-tabs
Merge branch 'master' into 3138-wiselinks
3138: wiselinks data-push still not working.
3138: update page-wrapper for each refresh.
Merge branch 'master' into 3138-wiselinks
3138: Wrap breadcrumps and page-wrapper in a body-wrapper div. Update this target for wiselinks for each request.
Merge branch 'master' into 3140-project-content-tabs
Merge branch 'master' into 3140-project-content-tabs
closes #3140
Merge branch 'master' into 3138-wiselinks
3138: add wiselinks data-push to project link.
Merge branch 'master' into 3138-wiselinks
3193: add manage account page to topnav.
Merge branch 'master' into 3193-manage-account
3193: add manage account action to users controller.
Merge branch 'master' into 3193-manage-account
3193: updated the vm section based on Tom's suggestion: let's skip the 'last login' column for now, until we can make logs record the actual VM uuid
3193: add wrapping to td elements
3193: add ssh key using popup
Merge branch 'master' into 3193-manage-account
3193: Move "SSH keys" and "API tokens" from notifications menu into settings gear menu. These are now available only for admin users.
3193: add note if there are no ssh keys configured
3193: disable submit button in popup if either name or public key are empty.
3193: Show name column in the ssh key section.
3138: When wiselinks is initialized, selection.js does not work. Since we want to stop using selection.js in the near future, let's not initialize wiselinks object for now.
Merge branch 'master' into 3138-wiselinks
Merge branch 'master' into 3138-wiselinks
3138: test passing as is
refs #3138
Merge branch 'master' into 2985-selection-button-move-copy
Merge branch 'master' into 3193-manage-account
2985: move selection items to the chosen project
2985: move selections to chosen project
2985: fix typo
Merge branch 'master' into 2985-selection-button-move-copy
Merge branch 'master' into 3193-manage-account
Merge branch 'master' into 2985-selection-button-move-copy
2985: pass move items only when available
2985: forgot to use ruby's cool syntax
3016: schema doc updated with new search_for attribute details.
Merge branch 'master' into 3193-manage-account
3193: add head_uuid is a arvados#repository filter when looking for recpositories.
3193: minor text updates in the UI panels.
3193: show only a portion of the ssh key so that the page does not have to scroll too far down when there are multiple ssh keys configured.
3193: split ssh section into two sections
3193: improve virtual machines section to add a sample.
Merge branch 'master' into 3193-manage-account
3193: update the ssh popup to not require name.
Merge branch 'master' into 3016-pipeline-template-search-string
3016: Update per Tom's review feedback.
3016: prefill input search with value configured in the template
Merge branch 'master' into 3193-manage-account
Merge branch 'master' into 2985-selection-button-move-copy
3193: do not validate input submitted on client side
Merge branch 'master' into 3193-manage-account
Merge branch 'master' into 2985-selection-button-move-copy
2985: let errors propagate during save
2985: let error propagate during save
Merge branch '3193-manage-account' of git.curoverse.com:arvados into 3193-manage-account
closes #3065
Merge branch 'master' into 3193-manage-account
3193: minor ui updates
3193: minor ui updates
3193: add horizontal scroll to ssh key
Merge branch 'master' into 3193-manage-account
Merge branch 'master' into 3016-pipeline-template-search-string
refs #3016
3193: add plus icon to add ssh key button
3193: use sshkey gem to validate the ssh key passed in.
Merge branch 'master' into 3193-manage-account
3193: use HISTIGNORE for current token display
3193: add contact admin blurb when the user has no virtual machines and repositories configured.
3193: minor ui update
3193: update user tutorials with the manage account page details.
3193: one more doc section and some minor ui updates.
3193: some more doc updates
3193: add public key as tooltip to the fingertip
3193: add public key as tooltip to the fingerprint
Merge branch '3193-manage-account' of git.curoverse.com:arvados into 3193-manage-account
3193: one more minor doc update
closes #3193
3296: manage profile page added
Merge branch 'master' into 3296-user-profile
3296: redirect to the page that resulted in manage profile page
3296: add comments to applicaiton.defaults.yml describing the user_profile_form_fields configuration parameter.
Merge branch 'master' into 3296-user-profile
3296: tests passing and addressed trailing white spaces
3296: send notification email when the user first creates profile, provided an address is configured
3296: display read-only email, first name, last name, and identity url in the profile page.
3296: save button placement
3296: When invoking sudo operations, skip profile check. Profile check is problematic when an admin user does "login as" a different user who has no profile set because after login as that user, the profile page is displayed and needs to be filled. To address this, send an additional "skip_profile" parameter when doing a "login as" or any other sudo operations.
Merge branch 'master' into 3296-user-profile
3296: check profile if request method is get.
3296: profile path routing
3296: remove skip_profile and add params[:partial] handling
3296: some cleanup
3296: add "message" to profile config
Merge branch 'master' into 3296-user-profile
3296: add profile also in the notification email
3296: profile config edge cases
3296: improve config parameter explanation
3296: add integration tests for topnav. also, enable profile as default test setting
3296: use update method and remove unnecessary update_profile method
Merge branch 'master' into 3296-user-profile
3296: send email when profile is created by user. add tests to profile mailer.
3296: profile notification email setting is now on server
3296: update tests
3296: good tests do add value.
3296: fix test
3296: after profile is filled in by user, bring the user back to profile page with message saying all set and provide button to take to the desired target. for improved ui experience.
Merge branch '3296-user-profile' of git.curoverse.com:arvados into 3296-user-profile
3296: profile redirection works perfectly now that session caching is removed.
3296: check for prefs_changed instead of cycling through all changes.
3296: split profile config into two parameters: one for fields and one for message.
3296: minor updates to profile page display
3296: break the tests into multiple tests
Merge branch 'master' into 3296-user-profile
3296: cover the case where prefs was there but no profile in profile check.
3296: add span tag to profile message in config to serve as an example for admins.
Merge branch 'master' into 3296-user-profile
Merge branch 'master' into 3296-user-profile
3112: add source_version to discovery document.
Merge branch 'master' into 3112-report-bug
Merge branch 'master' into 3296-user-profile
3296: fix tests
3296: fix test issue
closes #3296
Merge branch 'master' into 3112-report-bug
3112: config settings, mailer, popup display
Merge branch 'master' into 3112-report-bug
3112: when you experience a git error, that is when you see it's true colors!!
3112: simplify report message send logic
3112: add "report issue" button to 404 error page
3112: add tests for the new show version info and report problem buttons in help menu. yes, the tests did uncover a couple edge cases not covered by manual testing.
3112: report issue works for no user and inactive user cases.
Merge branch 'master' into 3112-report-bug
3112: trailing white spaces
3112: log when an issue is reported
3112: minor label updates
3112: add ajax event handling
Merge branch 'master' into 3112-report-bug
Merge branch 'master' into 3112-report-bug
3112: add "report issue" button to fiddlesticks error page
3112: add test to check the existence of "report issue" button in fiddlesticks error page
Merge branch 'master' into 3112-report-bug
3112: improve errors test to exercise the report issue button.
3112: update tests to test ajax success and associated popup changes when a report is sent.
3112: trailing white space
3112: use alert-success and alert-danger when reporting ajax events after submitting a report.
Merge branch 'master' into 3112-report-bug
3112: add caret to email address dropdown in topnav.
3188: add data-toggle to link
3153: auto-setup user during creation.
Merge branch 'master' into 3153-auto-setup-user
Merge branch 'master' into 3112-report-bug. Also, break search testing into a separate project.
Merge branch 'master' into 3112-report-bug
3112: update config param names for report notification
Merge branch 'master' into 3112-report-bug
Merge branch 'master' into 3153-auto-setup-user
3153: tests with duplicate username
3153: update the new user email to include a phrase saying the user is setup when config is on.
Merge branch 'master' into 3153-auto-setup-user
Merge branch 'master' into 3112-report-bug
3112: use partial to report error
Merge branch 'master' into 3112-report-bug
3112: and it would help if that said partial is committed :)
3112: use jquery syntax for find buttons
3112: move ajax event handling into a separate js file
Merge branch 'master' into 3153-auto-setup-user
3153: update regexp, email message
Merge branch 'master' into 3153-auto-setup-user
3153: username regexp is updated
3153: update tests to check for existing repo name
3153: use loop with counter than while(true) to derive unique username
Merge branch 'master' into 3112-report-bug
3112: change submit button text
3153: return nil if no unique username can be derived from an email address. no setup will happen in that case.
3153: trailing white space
3112: limit ajax event handling only to the report form
Merge branch 'master' into 3112-report-bug
Merge branch 'master' into 3153-auto-setup-user
3153: when deriving unique name, look for all can_login links, not just the vm_uuid configured. update tests to assert the generated username ends with a number when using a preconfigured username for this link.
3153: when exsiting repository name or vm login link is used as username, assert the generated unique username starts with the given username and ends with a number.
3112: ajax event handling on form not working. reverting back to document for now.
3112: modal is shown by js
3112: reload current location when error report modal is closing. otherwise, the modal does not show up if the help menu dropdown option is used right after sending an error report from within the 404 page.
3153: fix the while loop logic.
Merge branch 'master' into 3188-remove-selected-button
closes #3183
Merge branch 'master' into 3112-report-bug
Merge branch 'master' into 3112-report-bug
3153: directly invoke setup method during auto-setup
3153: change variable name in test from valid_username to ok_to_auto_setup
Merge branch '3153-auto-setup-tests' into 3153-auto-setup-user
3153: when new_users_are_active config param is set, even blacklisted users will be "active", and hence the email notification would say the user is setup when auto_setup_new_users is set.
3153: update tests to look for new user created message also when an inactive user is being created.
3153: minor test updates: rename the variable "active_recipients" as "new_user_recipients" to ensure test readability.
Merge branch 'master' into 3153-auto-setup-user
3112: update report_issue.js to confine event handling to report issue. WIP: mostly working, except from the 404 error page.
3112: report issue success event handling working in 404 error page.
3112: trailing white space
Merge branch 'master' into 3153-auto-setup-user
closes #3153
3153: update the newly added vm login link to not point to the "active" user
closes #3153
Merge branch 'master' into 3112-report-bug
3354: add integration tests verify textile descriptions for projects
Merge branch 'master' into 3112-report-bug
3112: Move profile testing out of application_layout_test.rb into it's own file for better testability.
Merge branch 'master' into 3112-report-bug
3112: minor update in application layout test
Merge branch 'master' into 3112-report-bug
3620: display gear menu to admin users only. update the layout tests accordingly.
3642: display search box in topnav only to active users
Merge branch 'master' into 3112-report-bug
Merge branch '3112-report-bug' into 3642-search-for-active-only
Merge branch '3112-report-bug' into 3620-admin-only-gear-menu
3112: minor cleanup
3112: add some space below the submit button and alert panel
3112: fix indentation
3112: update issue report routes config and some other minor test updates.
Merge branch '3112-report-bug' into 3620-admin-only-gear-menu
Merge branch '3112-report-bug' into 3642-search-for-active-only
3112: move manage account testing into its own file
Merge branch '3112-report-bug' into 3620-admin-only-gear-menu
3620: minor test update
Merge branch '3112-report-bug' into 3642-search-for-active-only
Merge branch 'master' into 3112-report-bug
Merge branch 'master' into 3112-report-bug
3112: alert left justification
Merge branch '3112-report-bug' into 3620-admin-only-gear-menu
Merge branch '3112-report-bug' into 3642-search-for-active-only
Merge branch 'master' into 3112-report-bug
3112: no need to reload after sending report.
3112: skip filters in actions controller, instead of application controller
Merge branch 'master' into 3112-report-bug
3112: add scrollbar to the report issue popup
3112: removed rack_test driver and added selenium driver. still the test "trying to use expired token redirects to login page" fails when the entire suite is tested, but passes when this one test file is executed.
3112: still the login_tests fails when ran in the entire suite
3112: login_tests works with javascript driver
3112: 404 error page default message while reporting issue
Merge branch 'master' into 3112-report-bug
closes #3112
Merge branch 'master' into 3620-admin-only-gear-menu
Merge branch 'master' into 3642-search-for-active-only
3620: change menu item role from presentation to menuitem for clickable links. http://www.w3.org/TR/wai-aria/roles#role_definitions
closes #3642
Merge branch 'master' into 3620-admin-only-gear-menu
closes #3620
3620: correct indentation when current user is available
refs #3620
3339: render truncated textile descriptions in dashboard
3339: update test to verify textile description in my projects panel in dashboard.
3714: disable submit button until problem description is entered.
3339: some refactoring of the render textile impl. improve the test that verifies project description.
Merge branch 'master' into 3714-report-issue-issues
3714: update tests to verify that the send report button is disabled until problem description is entered.
Merge branch 'master' into 3339-truncate-project-descriptions
Merge branch 'master' into 3339-truncate-project-descriptions
3339: fix typo in test
closes #3339
3637: add copy selected items functionality to project selection dropdown.
Merge branch 'master' into 3637-copy-selections
3637: refactor code for better maintenance.
Merge branch 'master' into 3714-report-issue-issues
3714: minor updates from review feedback
Merge branch 'master' into 3637-copy-selections
3637: remove "Copy selections" and "Move selections" options from clippy in topnav.
3637: fix test now that "Copy selections" and "Move selections" options are no longer available in clippy.
Merge branch 'master' into 3714-report-issue-issues
closes #3714
refs #3637 - merge Tom's updates to chooser implementation, where selected items are passed as parameters.
3637: undo some of the changes to absorb Tom's updates to chooser implementation
Merge branch 'master' into 3637-copy-selections
3637: copy selected items
3637: add integration tests for "copy", "move", and "remove" operations from the project selection dropdown.
3637: remove unwanted tab chars
Merge branch 'master' into 3637-copy-selections
3637: add one more test that verifies that the copy action is disabled when a subproject under a project is selected.
closes #3637
3654: add "combine selections" option to project#show page.
3654: add selection dropdown to collections pages
3654: remove "Combine selected collections and files into a new collection" option from clippy.
3710: update the combine_selected_files_into_collection method to also handle the scenarios where a collection uuid or "collection uuid/filename" is passed.
3710: handle the scenario where files from named collections are selected
Merge branch 'master' into 3654-combine-selections
Merge branch 'master' into 3654-combine-selections
3654: add integration and funtional tests for combining collections and collection contents.
3654: add selection dropdown to collection#show page
3654: add selection dropdown to collections page
3654: add selection dropdown to collections page
Merge branch 'master' into 3654-combine-selections
3654: test update
3654: add selection dropdown to collections page
3654: combine options is enabled when only data collections are selected in project view.
3654: use POST method to combine selections.
3654: little cleanup around the post method
3654: slight update to redirect to newly created collection
Merge branch 'master' into 3654-combine-selections
3654: redirection to newly created collection is now working.
Merge branch 'master' into 3654-combine-selections
2761: diagnostics testing
Merge branch 'master' into 2761-diagnostic-suite
Merge branch 'master' into 3654-combine-selections
3654: display show icon for files only when the file suffix is one of txt, gif etc. Also, fixed a bug in summary html where the collection show was failing when finished_at is not available.
Merge branch 'master' into 3654-combine-selections
Merge branch 'master' into 2761-diagnostic-suite
2761: use application.yml to configure diagnostics testing
2761: diagnostics testing config in application.yml
2761: load inputs dynamically
2761: test updates to be able to handle multiple inputs
Merge branch 'master' into 2761-diagnostic-suite
2761: wait for the pipeline run to complete or until wait time exceeds.
Merge branch 'master' into 2761-diagnostic-suite
2761: test in staging.
2761: file path needs lookup in prevew panel
Merge branch 'master' into 2761-diagnostic-suite
2761: input selection revisited
2761: update comment
Merge branch 'master' into 3654-combine-selections
3654: add a default name to newly created combined collection
3654: move the checkbox to the left of the row in collection show page. also, fixed a bug where the newly added selection dropdown shows up in the collection file chooser popup.
Merge branch 'master' into 3654-combine-selections
2875: add description to jobs and pipeline_instances.
3654: update tests to match the selection dropdown text changes.
2875: add new partial to display name and description.
Merge branch 'master' into 2875-pipeline-description
Merge branch 'master' into 3654-combine-selections
3654: disable search icon collection show page rather than hide it when it is not applicable.
2875: set initial description on the new pipeline instance created by running a template.
Merge branch 'master' into 2875-pipeline-description
2875: initial description on pipeline instance when the template has no name.
Merge branch 'master' into 3654-combine-selections
closes #3654
Merge branch 'master' into 2875-pipeline-description
2875: add initial description to pipeline instance when created using arv-run-pipeline-instance script
2875: add description option
2875: render template description as textile attribte in show recent page
2875: integration tests to verify description on pipeline templates, instances, and jobs.
no issue # fix failing test to use selenium driver
Merge branch 'master' into 2761-diagnostic-suite
2875: remove junk migration lines
2875: collection description is now textile and also the display is updated.
2875: minor text update
closes #2875
Merge branch 'master' into 2761-diagnostic-suite
2761: add diagnostics section to config
2875: remove pipeline instance fixture and use an existing one.
no issue # remove pipeline instance fixture to address failing test.
2761: update config
Merge branch 'master' into 2761-diagnostic-suite
2761: minor updates to diagnostics test config explanation.
Merge branch 'master' into 2761-diagnostic-suite
Merge branch 'master' into 2761-diagnostic-suite
2761: added some comments
no issue # : fix nil issue in pipeline_templates page
no issue # - fix couple more places where description is failing on nil template name.
Merge branch 'master' into 2761-diagnostic-suite
closes #2761
3898: db migration to add state attribute to job.
3898: add unit test for job state attribute
Merge branch 'master' into 3898-job-state-attr
3898: set finished_at if nil when success is changed.
3898: if state is changed to Running and is_locked_by_uuid is nil, return validation error.
3898: couple more test assertions
3898: look for success before running.
3898: cancelled_at takes precedence over success flag.
Merge branch 'master' into 3898-job-state-attr
Merge branch 'master' into 3898-job-state-attr
closes #3898
3898: added include statement
Merge branch 'master' into 3898-job-state-attr
refs #3898
3889: update ActionController::TestCase to monkey patch GET/POST/PUT/DELETE methods
3381: address the situation where sum percentages calculated exceeded 100%
3381: do not convert percentages into integers by using ceil. When we do this, we are sometimes ending up with percentage sum of > 100%.
3898: Update tests to match the updated job status handling.
3038: Recommended default port number related update. Now the defaults are as follows:
3038: test needed minor update to match current pipeline instance display.
Merge branch 'master' into 3038-default-port-numbers
3038: if there is a unique constraint violation when a collection is removed from a project,
Merge branch 'master' into 3038-default-port-numbers
3836: if there is a unique constraint violation when a collection is removed from a project,
3038: revert update to remove implementation that was accidentatlly pushed to this branch instead of 3836 branch!
3038: address review feeback; change the config parametter values in default yml files to avoid conflicts
Merge branch 'master' into 3889-functional-testing
Merge branch '3889-functional-testing-TC' into 3889-functional-testing
3889: update test_helper.rb with Tom's updates.
closes #3038
Merge branch 'master' into 3836-remove-collection-from-project-bug
Merge branch 'master' into 3889-functional-testing
3836: add integration test to verify name change when a collection is removed from a project,
3836: minor update to test assertion
3836: if rescued an error other than unique constraint violation, raise it back.
3836: update the fuse test to expect the newly added collections.
Merge branch 'master' into 3836-remove-collection-from-project-bug
Merge branch 'master' into 3889-functional-testing
Merge branch 'master' into 3889-functional-testing
refs #3889: Merge monkey patch code updates to detect tests that are reusing ActionController::TestCase.
3889: Fix two functional tests that were performing multiple actions on a single instance of TestCase object.
3889: fix a few tests that were performing multiple actions on a single instance of TestCase object.
3889: remove the functional test "setup and unsetup user" that is performing two actions in one test,
3889: split the test "setup user in multiple steps and verify response" into multiple tests invoking setup on active user,
3889: update the test "non-admin user can limit index" to perform the limit check in two tests rather than twice in one test.
3889: fix tests that were doing multiple actions in one test.
3889: break the "add keep disk with no filesystem_uuid" test into two tests, instead of doing two post actions in the same test.
3889: split test doing post and get requests into two different tests by using a fixture.
3889: split the test "cancel a running job" into two tests: one that cancels a running job and another that cancels an already cancelled job.
3889: split the test "update a job without failing script_version check" into two tests instead of two action in one test.
3889: split the test "normalize output and log uuids when creating job" into two tests:
3889: add the job fixture "job_with_unnormalized_output_and_log"
3889: split the test "get all pages of group-owned objects" into multiple tests that pull pages with offset of 5.
3889: update the test "use ownership and permission links to determine which logs a user can see" to account for the newly added log fixture.
3889: Since all failing API server functional tests are fixed, update test_helper to fail any future tests that execute multiple acions in one single functional test.
Merge branch 'master' into 3889-functional-testing
3889: fix typo in test name
Merge branch 'master' into 3836-remove-collection-from-project-bug
3889: add fixture "pipeline_with_tagged_collection_input"
3889: update the test "update script_parameters one at a time using merge param" to use fixture instead of create a new pipeline instance.
3889: Since all offending tests are fixed, update test_helper to generate an error if any functional test performs more than one action in on test in the future.
Merge branch 'master' into 3889-functional-testing
3889: update the fuse test to expect the newly added pipeline instances.
3889: add "#{user} can get collection using portable data hash" test
3889: some monir assertion updates.
Merge branch 'master' into 3889-functional-testing
3889: remove the job fixture "job_with_unnormalized_output_and_log" and instead use "foobar"
3889: minor test assertion updates
Merge branch 'master' into 3836-remove-collection-from-project-bug
closes #3836
Merge branch 'master' into 3889-functional-testing
3889: add a new integration tester for users controller. Refactor code so that users functional and integration tests can reuse some of the helper methods.
Merge branch 'master' into 3889-functional-testing
Merge branch 'master' into 3889-functional-testing
closes #3889
3583: replace " characters with ' characters in node value to ensure graph does not get mixed up on node labels.
Merge branch 'master' into 3583-provenance-graph-issue
4046: when combining collections in a project, set current project as owner (if writable) and show the new collection.
Merge branch 'master' of git.curoverse.com:arvados into 4036-combine-collections-in-project-issue
3583: replace " characters with \" in node value to ensure graph does not get mixed up on node labels.
Merge branch 'master' into 3583-provenance-graph-issue
4036: improve rails flash message display to check if the message is an error.
4036: update combine collection tests to expect the newly created collection in the right project.
4036: when files in a collection are combined, create the new collection in the original collection's project, provided it is writable.
Merge branch 'master' into 4036-combine-collections-in-project-issue
4036: update the test that combines files from within a collection to expect the new collection in current project, if applicable and writable.
Merge branch 'master' into 3583-provenance-graph-issue
3583: extend encode quotes logic for script parameters as well, just in case.
4036: correct div class
4025: add tooltips to some of the tab line buttons and also adjust the title of the "Move" button.
Merge branch 'master' into 4025-move-project-button-label
4205: add tooltips to pipeline instance tab line buttons
4025: add fa-fw to the newly added i tag class like all other such usages.
Merge branch 'master' into 4036-combine-collections-in-project-issue
4036: add comment for flash alert div
3882: add test to verify a cancelled job stays cancelled on futher cancel actions.
closes #4036
Merge branch 'master' into 3882-cancel-already-cancelled-job
3882: added a few more tests for cancelled_at state. Most importantly, uncomment the assertion
3882: refactor tests
Merge branch 'master' into 4025-move-project-button-label
4025: Change the button label 'Move...' to 'Move <object_class>...' and update test assertion accordingly.
closes #4025
Merge branch 'master' into 3882-cancel-already-cancelled-job
Merge branch '3882-cancel-already-cancelled-job-TC' into 3882-cancel-already-cancelled-job
3882: Change "update_timestamps_when_state_changes" from before_validation to after_validation,
Merge branch 'master' into 3882-cancel-already-cancelled-job
4046: update queue_position method to increment index and add unit test.
Merge branch 'master' into 4046-job-queue-position
Merge branch 'master' into 3583-provenance-graph-issue
closes #3583
Merge branch 'master' into 4046-job-queue-position
4046: update assert error message
Merge branch 'master' into 4046-job-queue-position
closes #4046
3990: set owner_uuid of a copied pipeline instance to that of the source, provided it is a project and writable by the current user.
3990: refactor pipeline instances integration test to reuse logic that creates and runs a pipeline.
Merge branch 'master' into 3990-owner-when-rerunning-pipeline
3990: add integration tests to rerun pipeline.
Merge branch 'master' into 3990-owner-when-rerunning-pipeline
3990: rerunning pipeline from within a project with no write permission
Merge branch 'master' into 3990-owner-when-rerunning-pipeline
Merge branch 'master' into 3882-cancel-already-cancelled-job
3882: update "after_validation :update_timestamps_when_state_changes" to "before_save :update_timestamps_when_state_changes",
closes #3882
Merge branch 'master' into 3990-owner-when-rerunning-pipeline
3990: minor test update
closes #3990
4126: when parameter is a hash, use it as value if nothing else matches.
Merge branch 'master' into 4126-preserve-parameter-hash
4126: undo the hash parameter retention logic for value
closes #4126
4062: fix greedy infinite scrolling in tabs displaying more than one kind (Jobs and pipelines tab and Other objects tab).
4062: trailing white space
Merge branch 'master' into 4062-infinite-scroll-repeat-issue
4062: test update
Merge branch 'master' into 4062-infinite-scroll-repeat-issue
4062: update python testing to handle the multiple fixture object creation, similar to workbench test_helper update.
4062: test updates
4062: Do not include offset in the next_page_href when objects of multiple kinds are being fetched; instead, we will rely on created_at filter alone.
Merge branch 'master' into 4062-infinite-scroll-repeat-issue
4054: show email address in sharing tab and update test to verify.
Merge branch 'master' into 4054-sharing-email
4054: minor test update
4026: admin user can move a project under another user's home project.
closes #4062
Merge branch 'master' into 4054-sharing-email
Merge branch 'master' into 4026-project-move-admin
4227: AM / PM indicator fix
Merge branch 'master' into 4054-sharing-email
4091: the test with 200 jobs and 10 pipelines is failing in jenkins env. reduce the number of pipelines to 2 to see if shorter runtime helps.
closes #4054
Merge branch 'master' into 4227-date-display
4227: minor test upadtes
4227: some more test and fixture updates.
Merge branch 'master' of git.curoverse.com:arvados into 4227-date-display
4227: update fixtures to use bigger finish times
4195: fix the tutorial to match the latest dashboard ui.
Merge branch 'master' into 4195-pipeline-tutorial
Merge branch 'master' into 4227-date-display
4227: much simplified date display test
refs #4091 - comment out the test that is failing intermittently in jenkins env.
Merge branch 'master' into 4227-date-display
closes #4227
4228: fix arv_normalize input when a file in a subdir of a collection is being combined.
4228: fix arv_normalize input when a file in a subdir of a collection is being combined.
4228: add test for combining files from a collection subdir.
Merge branch 'master' into 4228-collection-subdir-files
Merge branch 'master' into 4026-project-move-admin
4026: simplify writable check
closes #4026
4091: pass limit parameter from project tabs.
refs #4227 - rescue from any error in parsing pipeline start and finish time so that jenkins runs do not choke on these failures.
refs #4227 - update strptime format for parsing pipeline start and finish time.
no issue #
no issue #
refs #4227
Merge branch 'master' into 4228-collection-subdir-files
Merge branch 'master' into 4195-pipeline-tutorial
closes #4228
Merge branch 'master' into 4195-pipeline-tutorial
4195: update workbench dashboard pic
Merge branch 'master' into 4195-pipeline-tutorial
closes #4195
Merge branch 'master' into 4091-test-infinite-scrolling
4091: adjust the test fixture to use less data to match the limit size of 100 for the Jobs and pipelines tab in project.
Merge branch 'master' into 4091-test-infinite-scrolling
4091: add offset to next page url
Merge branch 'master' into 4091-test-infinite-scrolling
4285: prepend './' to file path being retrieved when it is in a subdir in a collection.
4285: use regexp to prepent ./ to file path
closes #4285
Merge branch 'master' into 4091-test-infinite-scrolling
4091: change project contents order from uuid to created_at.
Merge branch 'master' into 4091-test-infinite-scrolling
4091: retain filters in project contents collections and similar tabs.
4091: enhance code to handle the case where there are more than one item in the next page matching the previous page's last object's created_at time.
4091: update jobs fixture to set same created_at timestamp for some of the jobs.
4091: set the limit size to 50 for "Jobs and pipelines" tab.
Merge branch 'master' into 4091-test-infinite-scrolling
closes #4091
3177: look for checked persistent selections on submit.
Merge branch 'master' into 3177-collection-choose-files
Merge branch 'master' into 3177-collection-choose-files
3177: propagate checkbox selection param when dataclass is File.
3177: restrict only one file selection in collection chooser; remove clippy; clear selections on load.
3177: store input pdh value even when input dataclass is File. Update two tests to no longer look for clippy.
3177: add tests for a template that uses dataclass = File
3177: trailing white space
3177: try to partition input selection only if it is a string
3177: String != 'String'
Merge branch 'master' into 3177-collection-choose-files
Merge branch 'master' into 3177-collection-choose-files
4024: add infinite scrolling to pipeline_instances page.
4024: add integration tests for pipeline_instances page infinite scrolling.
4024: add search filter to pipeline instances page.
Merge branch 'master' into 4024-pipeline-instances-scroll
4024: add tests for search filter in pipelin_instances page.
4024: a few more test cases for search filter in pipeline instances page.
4024: trailing white space
4024: disable scrolling when search filter is used; otherwise, non-matching items are also displayed in page two onwards.
4024: comment out code that disables scrolling when search is used.
4024: scrolling + search works now that filters are included in next page href.
4024: rearrange partial handling
4024: simplify search box
Merge branch 'master' into 4024-pipeline-instances-scroll
Merge branch 'master' into 4024-pipeline-instances-scroll
Merge branch 'master' into 3177-collection-choose-files
Merge branch 'master' into 3177-collection-choose-files
Merge branch '4024-pipeline-instances-scroll' of git.curoverse.com:arvados into 4024-pipeline-instances-scroll
4024: move the search box to the top
Merge branch 'master' into 4024-pipeline-instances-scroll
Merge branch 'master' into 3177-collection-choose-files
3177: in file chooser mode, allow row selection
Merge branch 'master' into 3177-collection-choose-files
3177: file selection
Merge branch 'master' into 3177-collection-choose-files
3177: update test to select row in preview panel
Merge branch 'master' into 3177-collection-choose-files
3177: use no left padding when collection files preview panel is used in file chooser dialog.
closes #3177
4194: Add LoggingResponseWriter and RESTRouter wrapper. Update keepstore and handler_test to use the router wrapper.
4194: improve logging when large block is retrieved.
Merge branch 'master' into 4024-pipeline-instances-scroll
4194: since LoggingResponseWriter is doing the logging, remove some of the old log statements that are now covered by the LoggingResponseWriter in keepstore handlers.
4194: remove some more old log statements
4194: rename RESTRouterWrapper
4194: refactor REST Router wrapper into a separate package named keep_utils in keep directory.
Merge branch 'master' into 4194-keep-logging
4194: refactor router wrapper into a separate file.
4194: update keepstore and handler_test to make wrapper instance rather than the two step process of creating REST router and then the wrapper.
4194: logging writer copies / appends passed in data into its own buffer.
Merge branch 'master' into 4194-keep-logging
Merge branch '4024-pipeline-instances-scroll' of git.curoverse.com:arvados into 4024-pipeline-instances-scroll
Merge branch 'master' into 4024-pipeline-instances-scroll
4024: add test assertion to check no error in page.
4024: update test assert
Merge branch 'master' into 4024-pipeline-instances-scroll
Merge branch '4024-pipeline-instances-scroll' of git.curoverse.com:arvados into 4024-pipeline-instances-scroll
closes #4024
4250: update user guide to reflect current code.
Merge branch 'master' into 4194-keep-logging
4194: rename RESTRouterWrapper as LoggingRESTRouter
4194: rename wrapper
4194: MakeLoggingRESTRouter returns a pointer rather than the struct.
Merge branch 'master' into 4194-keep-logging
4194: rename wrapper as logging_router
Merge branch 'master' into 4194-keep-logging
closes #4194
4194: just print data size and never the data
4194: use go fmt to replace spaces with tabs.
4194: use same log statement for successes as well as errors.
Merge branch 'master' into 4194-keep-logging
4156: update failing tests in test_arv-put that were expecting pdh instead of uuid.
Merge branch 'master' into 4194-keep-logging
refs #4194
Merge branch 'master' into 4156-cli-tests
Merge branch 'master' into 4250-doc-updates
4250: update pdh to uuid in keep tutorial
4156: update test to expect uuid instead of pdh
4156: minor updates
4156: update tests to match current impl
Merge branch 'master' into 4156-cli-tests
Merge branch 'master' into 4250-doc-updates
4156: little cleanup
4250: keep doc update
closes #4250
Merge branch 'master' into 4156-cli-tests
4156: skip arv-tag tests while arv-tag is being worked on.
Merge branch 'master' into 4156-cli-tests
4592: Fix the nil error when a collection without a name is selected as input.
4592: add test where a collection with no name is selected as input to a pipeline instance.
Merge branch 'master' into 4156-cli-tests
Merge branch 'master' into 4592-input-selection
Merge branch 'master' into 4592-input-selection
Merge branch 'master' into 4592-input-selection
4592: add fixture to be used by the api server test "search collections with 'any' operator".
Merge branch 'master' into 4156-cli-tests
4156: skip all the tests that require api server until #4534 is completed.
4592: update fixture checksum
closes #4592
Merge branch 'master' into 4156-cli-tests
4156: 3 more tests needed to be skipped.
Merge branch 'master' into 4156-cli-tests
4156: Since all the cli tests that require apiserver are skipped, all cli tests pass now. Hence, do not skip running cli tests anymore.
4641: use selenium driver and bigger page width.
4641: click on confirm dialog OK button in sharing test on Revoke action.
closes #4641
4156: uncomment one of the asserts
Merge branch 'master' into 4156-cli-tests
refs #4156
closes #4156
4339: add test for "error while loading tab"
4339: config will be reset after each test, so no need to reset it explicitly. also, use assert_selector instead of assert has_link?
closes #4339
4644: fix div end tag placement and add filterable to directories as well.
4487: Omit "Remove selected" selection option when the current user cannot write to the project.
4644: do not add filterable to directories
Merge branch 'master' into 4644-collection-file-filtering
closes #4644
Merge branch 'master' into 4487-and-4558
4487: updates from review feedback
Merge branch 'master' into 4487-and-4558
closes #4487
4559: wrap long component name
Merge branch 'master' into 4559-long-component-names
4638: update ssh notification link to point to manage account page.
4638: trailing white space
4645: log files in provenance graph can be clicked.
closes #4645
4525: remove stale clippy code
4525: add test to verify selection on tab switching.
4525: restore add_form_selection_sources function
Merge branch '4525-remove-stale-clippy-code-TC' into 4525-remove-stale-clippy-code
4525: remove select_form_sources function
Merge branch 'master' into 4525-remove-stale-clippy-code
4525: remove add_form_selection_sources (one more time) and all references.
Merge branch 'master' into 6064-collection-output-of-perf-issue
6064: rollback cache update so that it can be done in a separate branch.
integration_performance tests: add comment describing how to run the tests; also remove the ENV variable which is not needed
closes #6064
refs #6061
6057: add projects/public page, which lists publicly accessible projects.
Merge branch 'master' into 6057-public-projects-page
6093: doc updates
6093: some more doc updates.
6093: a few more updates
6093: one more
Merge branch 'master' into 6093-refresh-docs
6057: if /projects/public is accessed when anonymous config is not enabled, show 404.
Merge branch 'master' into 6057-public-projects-page
Merge branch 'master' into 6057-public-projects-page
6057: few more minor tweaks
closes #6057
Merge branch 'master' into 6093-refresh-docs
Merge branch 'master' into 6087-collection-timing
Merge branch 'master' into 6087-collection-timing
Merge branch 'master' into 6087-collection-timing
Merge branch 'master' into 6093-refresh-docs
Merge branch 'master' into 6093-refresh-docs
6093: delete the redundant details in "alternate way to add ssh keys" section.
refs #6093
Merge branch 'master' into 6203-collection-perf-api
6203: Do not use Keep::Locator.parse to parse locator in some of the most expensive paths.
Merge branch 'master' into 6203-collection-perf-api
6203: Merge strip_manifest_text and maybe_clear_replication_confirmed into one method to avoid repeated manifest parsing related expense.
6203: add trailing newline character in munge_manifest_locators method only when the original manifest ended with one; one of the unit tests did not like it otherwise.
Merge branch 'master' into 6203-collection-perf-api
Merge branch 'master' into 6203-collection-perf-api
6203: Corrected one dumb switched order of if conditions that caused 5s lag!!
6093: Add button-override css to make any buttons added inside the documentation to appear unclickable to avoid any confusion.
Merge branch 'master' into 6093-refresh-docs
Merge branch 'master' into 6203-collection-perf-api
6203: Benchmarking revealed that regexp.match(string) is 2.5x more expensive than string =~ regexp. Updated check_signatures method accordingly.
6203: compute_pdh, computed_pdh etc etc etc confusion. clean up to make it easier to follow.
6203: further optimization of munge method; also, match[0].sub(/\+A[^+]*/, '') instead of split+append
6203: Use manifest.each_line and line.rstrip! instead of manifest.split("\n").
6203: Use LOCATION_REGEXP from sdk; also add back updated each_manifest_locator method
Merge branch 'master' into 6093-refresh-docs
Merge branch '6203-collection-perf-api-TC' of git.curoverse.com:arvados into 6203-collection-perf-api-TC
closes #6203
6093: remove "Alternate way to add SSH keys" and add the "Manage account" link blurb to "Adding your keys" section itself.
closes #6093
6277: add Manifest::valid? method in ruby sdk.
6277: improve error message for missing file tokens.
6277: valid manifest must end with new line.
6277: Add more restrictions to manifest format such as cannot start with '/' and end with '/' for file names,
6254: add "groups" to user setup process; these comma separated groups entered in the popup will be saved as an array of groups property for the user's can_login link.
6254: slight performance improvement where the can_login link is not retrieved and checked if the groups passed in are the same as those already saved.
6254: add groups and verify in test.
6254: remove redundant statement
7254: add groups to vm link.
6254: instead of tooltip, use a self explanatory label for groups text field.
6254: better groups text field label
6254: minor update to Groups text field label to avoid conflict with 'Virtual Machine' lebel.
Merge branch 'master' into 6277-manifest-validation
6277: extra white space
refs #6277 : ruby sdk with manifest validation method
closes #6254
6277: all that work and missed the basic nil and empty string check!!!
Merge branch 'master' into 6277-manifest-validation
refs #6277
6277: Add check_manifest_validity before_filter in collection model; however, at the moment, this method always true after logging the validation error.
6156: support config format for setting a node's hostname
6234: In /users page, display the "Show" button to admin users as well; however, ensure redirection to
6234: admin user can see other users' home projects; also add link to users' home projects in /users page rows.
Merge branch 'master' into 6156-hostnames-in-nodes
6156: use only sprintf formatting for node slot_number config.
Merge branch 'master' into 6277-manifest-validation-api
6277: rename the before_validation filter check_manifest_validity as log_invalid_manifest_format until we are ready to actually validate manifest formats.
refs #6277
6277: since locators are added to multilevel_collection_1 fixture, the collection retrieved after an update will have
refs #6277
Merge branch 'master' into 6156-hostnames-in-nodes
6156: convert the ping-should-fail test when hostname config is malformed into a controller test.
closes #6156
6218: add performance profiling and a sample test in python sdk.
6218: fix typo in sample test
6218: update the symlink and comment
6218: correct exception handling.
6218: update the exception handling.
refs #6218
Merge branch 'master' into 3198-writable-fuse
Merge branch 'master' into 6219-fuse-performance-testing
6219: first draft at fuse performance testing.
6219: minor test updates
Merge branch 'master' into 6234-user-edit-self
closes #6234
6219: add test to create collection and read contents using magic dir.
6219: magic dir test in progress
6219: some more magic dir based testing
Merge branch '3198-writable-fuse' of git.curoverse.com:arvados into 3198-writable-fuse
Merge branch '3198-writable-fuse' into 6219-fuse-performance-testing
6219: some more magic dir based testing
6219: add back the test with files with multiple blocks that I lost somewhere along the way.
6219: remove unused imports
6219: add test to list contents of a project with many collections.
6013: remove "repository" from user setup popup in workbench
6013: update user notification email upon setup.
6013: revert back the deactivate user button leading text in workbench user admin page.
closes #6013
6219: profile even smaller portions of code
6219: refactor functions and profile smaller fuse related blocks of code
6219: remove couple profiling statements for the tests.
6279: add login link in VM panel of manage_account page that takes user to shell_in_a_box.
Merge branch 'master' into 6279-web-shell-client
Merge branch 'master' into 6279-web-shell-client
6279: if hostname ends with .shell, strip it.
Merge branch 'master' into 6279-web-shell-client
i6279: The '-' and '_' are not working in Firefox. Upon googling, found this URL that suggested a workaround:
6279: add some comments
6279: Noticed that enabled.gif is also needed. I see this one when I use the right mouse click in shell_in_a_box terminal,
6279: added shell_in_a_box url as comment
6445: when a task does not have finished_at time, use the finished_at time from it's job - if one is available.
6279: open the webshell client in a blank window.
refs #6279
6014: Add "add new repository" page to create a tutorial repository, and update the tutorial-submit-job page to use the tutorial repository.
Merge branch 'master' into 6445-incorrect-task-runtime-calc
6445: update the job status line also to use the job's finished_at time when task has none.
6014: minor updates
closes #6445
Merge branch 'master' into 6014-repo-creation-doc
closes #6014
Merge branch 'master' into 6219-fuse-performance-testing
Merge branch 'master' into 6219-fuse-performance-testing
closes #6219
6473: add start_time additional argument to arv-ws
Merge branch 'master' into 6473-fetch-events-starting-at
6473: some more error tests
6473: remove incorrect format tests; it appears that #6451 strikes after one such test and the next test fails.
6587: Documentation to access an arvados VM using webshell. Updated pages such as "tutorial expectations",
Merge branch 'master' into 6587-workbench-webshell-login-documentation
6617: update documentation to include https git url
Merge branch 'master' into 6473-fetch-events-starting-at
6473: include last_log_id when start_time argument is provided.
Merge branch 'master' of git.curoverse.com:arvados into 6473-fetch-events-starting-at
6473: create ancestor object before subscribe
Merge branch 'master' into 6617-display-https-url
6617: add note if prompted for user name during git clone using https url.
6619: add git config to shell setup page.
6617: updated git config doc
Merge branch 'master' into 6473-fetch-events-starting-at
Merge branch 'master' into 6617-doc-repo-https-url
6617: minor text update
closes #6617
Merge branch 'master' into 6619-doc-update
6619: git config update
refs #6619
Merge branch 'master' into 6587-workbench-webshell-login-documentation
6587: minor text updates
Merge branch 'master' into 6473-fetch-events-starting-at
6473: PollClient also supports last_log_id argument.
Merge branch 'master' into 6587-workbench-webshell-login-documentation
closes #6587
Merge branch 'master' into 6473-fetch-events-starting-at
6473: fix issue with last_log_id argument.
Merge branch 'master' into 6473-fetch-events-starting-at
6473: last_log_id null error
6465: Noticed a slight dip in code coverage in two areas and added tests to cover those situations. Details:
refs #6465
Merge branch 'master' into 6473-fetch-events-starting-at
6473: update message for start_time input argument.
closes #6473
6588: split manage account
6687: install guide updates aimed at reducing jumping around during the process
refs #6279
Merge branch 'master' into 6687-install-guide-flow
6687: merge back nginx docs back into api server and workbench pages and remove the includes since they are not being shared.
6687: revisit build tools text and other minor updates
closes #6687
6602: use min_nodes * (finished_at - started_at) to calculate the allocated node time for each job.
Merge branch 'master' into 6602-job-runtime-display
Merge branch 'master' into 6602-job-runtime-display
6602: display runtime using the formula (min_nodes * (finished_at - started_at))
6602: update test assertion to match new format
Merge branch 'master' into 6602-job-runtime-display
6657: check git is installed in arv-copy
Merge branch 'master' into 6657-no-git-in-arv-copy
Merge branch 'master' into 6657-no-git-in-arv-copy
6657: check git availability only when recursive copy is being used.
Merge branch 'master' into 6657-no-git-in-arv-copy
6657: copy_pipeline_template method also updated to check git existence.
Merge branch 'master' into 6657-no-git-in-arv-copy
6657: refactor git check and reuse.
6588: update user guide to refer to the updated user settings menu items.
6602: remove render_runtime_compact; instead use render_runtime with use_words = false and round_to_min.
6602: enhance render_runtime method to display seconds when days, hours, and minutes are all zero even when round_to_min is true.
closes #6602
Merge branch 'master' into 6657-no-git-in-arv-copy
closes #6657
Merge branch 'master' into 6588-documentation
6588: User settings menu item name updates.
6476: skip require_thread_api_token filter when anonymous browsing is enabled and one of the supported types is being accessed.
Merge branch 'master' into 6476-actions-show-not-need-token
6476: simpler conditional check
Merge branch 'master' into 6476-actions-show-not-need-token
6476: add comment
closes #6476
Merge branch 'master' into 6588-documentation
6588: fix typo
refs #6588
6277: enforce manifest validation
6588: replace manage_account page references with the new SSH keys and Virtual machines page references.
refs #6588
Merge branch 'master' into 6277-check_manifest_validity
Merge branch 'master' into 6277-check_manifest_validity
6277: more tests with error checking
Merge branch 'master' into 6277-check_manifest_validity
6277: add default_empty_manifest before_validation filter and update the tests accordingly.
6277: slightly improved tests
closes #6277
6851: reorder install guide pages; right after api server, install workbench, shell server and standard objects.
closes #6851
6880: Omit delete button from users index table; add controller tests.
6880: added deletable? method and used to determine whether or not to show this button in the delete_object_button partial.
Merge branch 'master' into 6880-remove-user-delete-button
6880: improved deletable? check
closes #6880
6851: reorder install guide pages; right after api server, install workbench, shell server and standard objects.
6827: log only valid token and only the first 10 characters.
6572: add a new page in install guide to "copy a pipeline template from Curoverse cloud" using arv-copy.
Merge branch 'master' into 6827-no-passwords-in-logs
6827: add a comment that only the first 10 characters of a valid token are logged.
closes #6827
Merge branch 'master' into 6827-short-token
6827: set validApiToken flag after ensuring that getting list of repositories work with the given credentials.
6827: fix error in setting valid token.
refs #6827
Merge branch 'master' into 6572-doc-copy-pipeline
6572: update to make it clear that the user can access anonymous token to use arv-copy.
Merge branch 'master' into 6572-doc-copy-pipeline
6859: added salvage_collection.rb script.
6859: updated script to append all locators to newly created manifest text from the original salvaged collection.
Merge branch 'master' into 6859-fix-invalid-manifests
Merge branch 'master' into 6859-fix-invalid-manifests
6859: fix issue with tempfile not actually being written and also update message logging.
Merge branch 'master' into 6572-doc-copy-pipeline
6572: minor section title updates
closes #6572
Merge branch 'master' into 6859-fix-invalid-manifests
6859: use --as-stream and improved arg parsing.
6859: Add check for ENV[ARVADOS_API_TOKEN] and handle any other issues during arv-put. Split salvalge_collection method into lib module.
6859: add test for salvage_collection
6859: add one more test with error during arv-put
Merge branch 'master' into 6859-fix-invalid-manifests
6859: observed that an exit from test running is captured and misinterpreted by the Kernel.exit mock method. Hence, update salvage_collection to exit with code 200, which is outside the range of special exit codes. http://tldp.org/LDP/abs/html/exitcodes.htmla
6859: reset ARVADOS_API_HOST and ARVADOS_API_TOKEN in teardown method in test.
Merge branch 'master' into 6859-fix-invalid-manifests
6859: mocking Kernel system method had the undersirable side effect of impacting any and all invocations of this method resulting in failures in other tests. Hence, refactored the arv-put invocation into a separate method and mocked this instead in the test. All tests pass now.
Merge branch 'master' into 6859-fix-invalid-manifests
6859: Always raise an exception on errors during salvaging. Catch any such exceptions in the script and exit. Update test accordingly.
7010: add note in install guides that we need git version 1.7.10 or later.
7010: trimmed version git version note.
Merge branch 'master' into 7010-git-version
closes #7010
6260: add datamanager to run-tests.sh
6260: datamanager integration testing - WIP
Merge branch 'master' into 6260-test-datamanager
Merge branch 'master' into 6859-fix-invalid-manifests
6859: script does not catch expections during salvaging; instead they are propagated resulting in exit; update tests.
Merge branch 'master' into 6859-fix-invalid-manifests
6859: add test that creates and salvages a collection with invalid manifest text.
6260: backdate blocks
Merge branch 'master' into 6260-test-datamanager
Merge branch 'master' into 6859-fix-invalid-manifests
6859: improve locator parsing to consider any terminating [[:xdigit:]]{32}; improve tests with more invalid locators.
Merge branch 'master' into 6260-test-datamanager
6260: status check added; still not working completely since invoking datamanager singlerun issues.
6260: Add TestGetStatusRepeatedly
Merge branch 'master' into 6260-test-datamanager
6260: test updates
6260: do not force never_delete in keepstore
6260: TestPutAndGetBlocks is working.
Merge branch 'master' into 6260-test-datamanager
Merge branch 'master' into 6260-test-datamanager
6260: add check in datamanager to ensure that the configured datamanager token belongs to an admin user.
Merge branch 'master' into 6260-test-datamanager
Merge branch 'master' into 6260-test-datamanager
6260: break old blocks into used and unused sets to make it easy to understand code and some more updates.
6260: append to testargs, instead of replacing.
6260: look for two replicas
Merge branch 'master' into 6260-test-datamanager
6260: Continue to force never_delete to be true until #6221 is resolved; in the meantime, let it be false during testing.
6260: update datamanager.go to return error than Fatalf in some cases; add additional tests to test those errors raised.
6260: no longer need the keep.data-manager-token-file
7179: list out the potential tests
7179: add generic volume test; as part of this exercise, moved some "generic" tests from volume_unix_test.go
Merge branch 'master' into 7179-generic-volume-tests
Merge branch '7179-generic-volume-tests' of git.curoverse.com:arvados into 7179-generic-volume-tests
7179: add TestUnixVolumeWithGenericTestsSerialized and some more updates.
7179: A few golint suggested updates. There are still a lot of golint complaints.
7179: Rather than using a DoGenericReadOnlyVolumeTests to test read-only volumes, update each test to either support
7179: handler_test.go is now lint free according to golint.
7179: several golint suggested updates such as "don't use ALL_CAPS in Go names; use CamelCase".
7179: some more golint suggested updates
7179: Most of golint suggested updates are made. Some names such as "never_delete" are left alone.
6260: golint suggestions incorporated. Also, the earlier commit log message "no longer need the keep.data-manager-token-file" is incorrect. Keepstore still needs this and this file is still generated during testing.
closes #7179
Merge branch 'master' into 6260-test-datamanager
6260: golint suggested updates
6260: so many golint complaints!!
6260: since datamanager no longer needs the data-manager-token-file (only keep server does), remove the run-tests.sh update for this.
refs #6260
6260: most golint suggestions are addressed.
7179: more updates to address golint suggestions around the config parameters such as never-delete, blob-signature-ttl etc.
refs #7179
Merge branch 'master' into 6260-test-datamanager
6260: update keepproxy test TestPostWithoutHash to match locator+size+Ahint instead of locator+size only.
closes #6260
7015: Update arv-copy user guide to use a pipeline template instead of instance in the examples.
7015: correct typo in copied template uuid
refs #7015
7329: It appears that UnixVolume -> Compare method is falling in infinite loop due to the fact that EOF is not returned when reading an empty file.
7162: enhance SetServiceRoots to look for all service hints, not just K@ hints.
7162: Use replicasPerService to avoid sending redundant replication requests.
Merge branch 'master' into 7162-support-service-types
7162: Update datamanager code to ask for all non-proxy typed keep services and fail if any non-disk types are found in the returned list.
Merge branch 'master' into 7162-support-service-types
7162: Update code to address the bug "current code forgets to reset Using_proxy to false if the API server stops giving out proxies between one call and the next"
Merge branch 'master' into 7329-empty-block-issue
7329: add generic volume based keepstore tests to have better functional test coverage.
7329: since moved functional testing into generic testing, removing now redundant empty block tests from volume_unix_test.go and keepstore_test.go.
7161: Look for all service hints, not just keep proxy. Also, use thread count of 1 when it not a disk.
Merge branch 'master' into 7329-empty-block-issue
7329: improved TestableVolumeManagerFactory method signature, teardown logic, comments and test file name.
Merge branch 'master' into 7329-empty-block-issue
closes #7329
Merge branch 'master' into 7162-support-service-types
7161: clarify max_replicas_per_service
7161: white space
7162: SetServiceRoots keepproxy hints only (revert my update).
7162: move replicasPerThread logic outside the for loop.
7162: updated comment
Merge branch 'master' into 7162-support-service-types
Merge branch 'master' into 7161-support-all-service-types
7161: working
Merge branch 'master' into 7162-support-service-types
7162: Removed the call to get keep_disks when there is an error getting keep_services,
closes #7162
7161: Look for all service hints, not just keep proxy. Also, use thread count of 1 when it not a disk.
7161: clarify max_replicas_per_service
7161: white space
7161: Use default value of 1 for max_replicas_per_service and set it to None when writable non-disk type services are found.
Merge branch '7161-support-all-service-types' of git.curoverse.com:arvados into 7161-support-all-service-types
closes #7161
7200: add GetIndex function to keepclient; add IndexHandler to keepproxy.
7200: The incomplete response when no such prefix exists will be "\n". Update keepclient and keepproxy to expect this.
7200: rename error variables to meet golint suggestions.
Merge branch 'master' into 7200-keepproxy-index-api
refs #6859
Merge branch 'master' into 7200-keepproxy-index-api
7200: Use if statement instead of switch to check http method in keepclient; strip terminating empty line from response in keepclient.
7200: no need to set content length header in keepproxy IndexHandler since the response is always terminated by a blank new line.
7200: Use io.Copy instead of reading all bytes and writing to response. Much improved keep proxy test with code reuse.
7200: more compact CheckAuthorizationHeader block
closes #7200
7167: add keep-rsync to gostuff.
7167: keep-rsync parameter loading and intialization. Update test framework to allow creating two sets of keep servers, source and destination.
Merge branch 'master' into 7167-keep-rsync-test-setup
7167: args not avaialble in all tests; hence store keep_existing argument in a variable rather than accessing it directly from args.
7167: Refactor MakeKeepClient and DiscoverKeepServers to allow making KeepClient from input JSON as well.
7167: Update test to also put a block in dst and attempt get from src.
7167: get index from src and dst and copy any missing blocks from src to dst.
Merge branch 'master' into 7167-keep-rsync-test-setup
7167: Use struct instead of map for APIConfig
Merge branch '7167-keep-rsync-test-setup' into 7167-keep-rsync
7167: rename MakeArvadosClientWithConfig as New
Merge branch 'master' into 7167-keep-rsync-test-setup
Merge branch 'master' into 7167-keep-rsync
7167: merge test setup branch
Merge branch '7167-keep-rsync-test-setup' into 7167-keep-rsync
7167: move perms code from keepstore into keepclient go SDK.
Merge branch '7167-keep-rsync-test-setup' into 7167-keep-rsync
Merge branch 'master' into 7167-keep-rsync-test-setup
7167: add --keep-enforce-permissions to run_test_servers.py
7167: set enforce_permissions to true if blob signing key argument is provided.
Merge branch '7167-keep-rsync-test-setup' into 7167-keep-rsync
7167: honor blob signing key while getting blocks.
7167: get replications count from destination api discovery doc and use it as default.
7167: add tests to replications count
Merge branch '7167-keep-rsync-test-setup' into 7167-keep-rsync
7167: move perms code from keepstore into keepclient go SDK.
7167: add tests with prefix during rsync
7167: default replications count from discovery doc test updates.
Merge branch '7167-keep-rsync-test-setup' into 7167-keep-rsync
Merge branch 'master' into 7167-keep-rsync-test-setup
Merge branch 'master' into 7167-keep-rsync-test-setup
Merge branch '7167-keep-rsync-test-setup' into 7167-keep-rsync
7167: stop rsync operation on any errors during Get or Put operations; add additional tests.
7167: some more error tests such as error getting block from src and error putting block to dst.
7167: break load config logic out of main into loadConfig func and add several tests.
7167: Convert most of the globals in keep-sync into locals and update all the code and tests as needed.
Merge branch 'master' into 7167-keep-rsync
7167: update run_test_servers.py to use action="store_true" instead of converting string to boolean.
7167: Convert blobSigningKey also into local variable and make necessary changes to accommodate this change.
7167: replace the keep_existing logic and create all 3 keep servers at once and use the first two as src keepservers and the last one as the dst keep server.
7167: replace keep_existing with num_keep_servers and use it create all required keep servers at once.
7167: when the config file does not contain '/', use $HOME/.config/arvados/<filename>.
7167: loadConfig setupKeepclient do only one set at a time.
Merge branch 'master' into 7167-keep-rsync
7167: log progress during keep-rsync and several test improvements.
Merge branch 'master' into 7167-keep-rsync
Merge branch '7167-keep-rsync' of git.curoverse.com:arvados into 7167-keep-rsync
7167: rename the newly added StopKeepServers as StopKeepWithParams; it now sounds similar to StartKeepWithParams and is self explanatory.
7167: update the progress message to say "est. time remaining" instead of "ETA" which is not accurate in this case.
Merge branch 'master' into 7167-keep-rsync
Merge branch 'master' into 7167-keep-rsync
closes #7167
Merge branch 'master' into 7167-keep-rsync-test-setup
refs #7167
7167: Break all the code from keep-rsync main method into a separate func so that arg parsing can also be tested.
7167: Remove StartKeepWithParams and StopKeepWithParams and make StartKeep and StopKeep with parameters the only exposed funcs.
7546: enhance putReplicas method to retry.
Merge branch 'master' into 7167-keep-rsync
refs #7167
Merge branch 'master' into 7546-put-retry
7492: better error reporting of upstream errors in keepproxy.
Merge branch 'master' into 7546-put-retry
7546: also retry when status code is 0, which is the case when a closed connection was used.
7492: update keep-rsync test with bad blob signing key to expect Forbidden error instead of Block not found error.
Merge branch 'master' into 7492-keepproxy-upstream-errors
Merge branch 'master' into 7546-put-retry
7546: update some of the failure tests to use keepclient.Retries = 0, so that the tests do not waste too much time
Merge branch 'master' into 7492-keepproxy-upstream-errors
7167: expand the src and dst help messages to list the config parameters that are to be included in the config files.
Merge branch 'master' into 7167-keep-rsync
refs #7167
Merge branch 'master' into 7546-put-retry
7546: update comment to explain why we do not want to retry when status code is 503.
closes #7546
Merge branch 'master' into 7492-keepproxy-upstream-errors
7492: cleanup error checking in keepproxy
Merge branch 'master' into 7492-keepproxy-upstream-errors
7492: add a keepproxy test with temporary connection refused error.
7453: Add "Add new group" button to user admin page.
7453: do not display generic "add new" button in the groups, keep_disks, keep_services, links, nodes, and virtual_machines listing pages.
7453: Disable submit button in add group dialog until a name is entered.
Merge branch 'master' into 7492-keepproxy-upstream-errors
7453: Upon Nico's request, put back the "Add a new" button in keep disks, keep services, and virtual machines pages.
7453: revert back to no generic "add new" button in all those pages.
closes #7453
Merge branch 'master' into 7492-keepproxy-upstream-errors
7492: add a test that simulates keep server unavailable error.
Merge branch 'master' into 7492-keepproxy-upstream-errors
refs #7167
Merge branch 'master' into 7492-keepproxy-upstream-errors
7167: update keep-rsync tests to use "Contains" instead of "HasSuffix" to make sure the error message checks pass even when retries happen.
closes #7492
7534: refactor the code from create_superuser_token.rb script into lib and verify manually that existing behavior is preserved.
7534: return an existing token instead of creating a new each time; add tests.
7534: move the print statement of the token obtained out the library into script.
7661: add --by-pdh option to FUSE and use this option in crunch-job. Do not start web socket client when --by-pdh is used.
Merge branch 'master' into 7534-superuser-token
closes #7534
5538: retry failed arvados api requests when appropriate.
5538: close any idle connections before a POST or DELETE request.
Merge branch 'master' into 5538-arvadosclient-retry
5538: using fake arvados server to generate errors, added tests with retries.
Merge branch 'master' into 5538-close-idle-connections
5538: add test with a connection idle for longer than MaxIdleConnectionDuration
7719: permit never-delte to be set to false; add warning that datamanager is not yet fully tested.
closes #7719
Merge branch 'master' into 5538-close-idle-connections
5538: update test to reuse arvados client in TestCreatePipelineTemplate between idle and current connections.
refs #5538
Merge branch 'master' into 5538-arvadosclient-retry
5538: update the newly added TestFail* to use proper client with http.Transport
7490: The makeArvadosClient func, which is invoked by singlerun, should return error; not fatalf.
5538: code improvements; use switch statement instead of if statement with several status code checks, sleep between retries.
5538: Merge FailHandler and FailThenSucceedHandler into one APIStub to facilitate testing many more error states; also add update and delete retry tests.
Merge branch 'master' into 5538-arvadosclient-retry
5538: much simpler and neater api stub test case array; golint
Merge branch 'master' into 7490-datamanager-dont-die-return-error
Merge branch 'master' into 5538-arvadosclient-retry
5538: correct retryable list and use it to determine whether to close idle connections; add a few more test cases.
5538: update the test to set resp.body with the given string from stub than hard code it (overlooked in previous commit)
5538: add a test that simulates error during requesting server so that we can test the error path as well.
7490: Convert several fatalf statements into returning errors. No new tests are added yet, but all the existing tests are passing.
Merge branch 'master' into 7661-fuse-by-pdh
7661: rename MagiDirectory by_pdh as pdh_only
5538: update the test case for "error" to use better stub parameters with nil status codes and response body to avoid any confusion to the reader.
closes #5538
Merge branch 'master' into 7490-datamanager-dont-die-return-error
Merge branch 'master' into 7661-fuse-by-pdh
Merge branch 'master' into 7661-fuse-by-pdh
7661: added test with only_pdh (not working yet)
7661: Pass pdh_only when adding by_id subdir; test now passes.
Merge branch 'master' into 7661-fuse-by-pdh
closes #7661
Merge branch 'master' into 7490-datamanager-dont-die-return-error
Merge branch 'master' into 7490-datamanager-dont-die-return-error
7490: Update the previously failing keep_test.go; no new tests added. We can now add datamanager/keep to gostuff in run-tests.sh
7490: add services/datamanager/keep to gostuff
Merge branch 'master' into 7490-datamanager-dont-die-return-error
7490: a few more keep unit tests with simulated errors
Merge branch 'master' into 7490-datamanager-dont-die-return-error
7490: added several error condition check tests for datamanager/keep package; increased code coverage from 14.6% to 72%
7490: added couple more datamanager tests with errors injected during GetCollections
Merge branch 'master' into 7490-datamanager-dont-die-return-error
7255: test collections with empty manifest_text and multiple streams and blocks.
7255: add one more test with manifest_text with multiple blocks in stream.
Merge branch 'master' into 7490-datamanager-dont-die-return-error
7490: Add Err to collection.ReadCollections and keep.ServerResponse so that the error can be propagated to clients accessing these through a channel read.
7490: use loggerutil to log any datamanager errors.
7490: remove loggerutil.LogErrorMessage that I added since it is not used
Merge branch 'master' into 7490-datamanager-dont-die-return-error
Merge branch 'master' into 7255-manifests-in-datamanager
7255: improve createMultiStreamBlockCollection to create collection with multiple blocks per stream.
7255: update TestPutAndGetCollectionsWithMultipleStreamsAndBlocks to verify that expected number of unique block locators are created.
Merge branch '7490-datamanager-dont-die-return-error' of git.curoverse.com:arvados into 7490-datamanager-dont-die-return-error
Merge branch 'master' into 7490-datamanager-dont-die-return-error
closes #7490
Merge branch 'master' into 7255-manifests-in-datamanager
7255: rename test and update comments.
Merge branch 'master' into 7255-manifests-in-datamanager
closes #7255
refs #7490
7253: Refactor APIStub and KeepServerStub into a testing.go so that the code can be reused from various packages
7253: Write a test that "does not" fail when an invalid manifest text is retrieved during collection.GetCollections(). The expected behavior: GetCollections() raises error in this case and hence datamanager aborts without deleting any blocks.
7253: Update test so that it compiles and passes.
7253: add sdk/go/manifest to gostuff
7253: instead of ignoring errors during BlockIterWithDuplicates, send them to the caller.
7252: convert fatalf's into errors in logger sdk code; remove AssertFromString from blockdigest and instead use FromString in all places where it was being used.
7253: add sdk/go/blockdigest to gostuff
Merge branch 'master' into 7253-datamanager-test-errors
7253: rename arvadostest/testing.go as stub.go
7253: update arvadostest.stub to offer ServerStub and remove the redundant KeepServerStub; update tests accordingly.
7253: consume all blocks from BlockIterWithDuplicates, not just the first in test.
Merge branch 'master' into 7253-datamanager-test-errors
7253: update BlockIterWithDuplicates to return any errors through Manifest, rather than BlockLocator.
Merge branch 'master' into 7253-add-go-sdk-manifest
7253: update manigest.parseManifestStream to raise error when the given manifest does not have any block locators or file tokens.
refs #7253
closes #7253
Merge branch 'master' into 7252-go-sdk-errors
7253: update the manifest_text for the test fixture multilevel_collection_2 to be valid.
refs #7253
7748: add dry-run command line argument.
Merge branch 'master' into 7252-go-sdk-errors
Merge branch 'master' into 7748-datamanager-dry-run
closes #7252
Merge branch 'master' into 7748-datamanager-dry-run
7748: address dryRun in summary.WritePullLists and keep.SendTrashLists
7748: when dryRun continue and leave the regular flow untouched.
7710: add crunchrunner to gostuff
Merge branch 'master' into 7748-datamanager-dry-run
7748: use local variables for arvLogger.Update
7710: improve keepclient to use non-disk timeouts when any non-disk typed keepservices are found; previously, this was done for proxy typed keepservices only.
7710: Remove KeepClient.Using_proxy and all it's references since we are no longer using it.
7710: update serviceType argument handling to ignore any keepservices that are of a different type. Add a datamanager test with one extra unsupported-typed keepserver, which should hence be ignored by datamanager.
7710: update crunchrunner.upload.go to compile; this was broken after the keep-web updates; also, add this package to gostuff in run-tests.sh
7710: text updates around the -service-type argument.
closes #7710
Merge branch 'master' into 7748-datamanager-dry-run
7748: correct the local variable declaration
closes #7748
refs #7710
6027: update git-arvados-guide to instruct the users to create scripts in crunch_scripts subdirectory.
Merge branch 'master' into 6027-git-repo-doc
6027: fix typo
closes #6027
7753: Add a Log link to pipeline_instance component panel
7753: The name "Job log" has a little problem; on screen small screen sizes, "Job log" is displayed in two lines even when the rest of the items are displayed in single line due to the limited real estate in the panel header.
7753: Now that there are 1+ "Log" links in the pipeline instance page, update tests to address capybara ambigous match error.
7753: when a job is in "Running" state, the Log link still should be shown in the Component display panel header.
7753: show the job Log link in the component panel header only when it is a pipeline_instance display; hide it for job#show
closes #7753
7607: Update application layout tests; by default getting_started is disabled and hence the help menu will not include this option.
7607: updated user_profile_test to set getting_started config in test.
Merge branch 'master' into 7607-getting-started-flag
closes #7607
8028: add services/crunch-dispatch-local
8177: add trust_all_content config to Workbench.
8028: crunch-dispatch-local implementation
8028: After getting list of Queued containers, instead of looking for containers.ItemsAvailable, look for len(containers.Items)
8028: add signal handling to dispatcher.
8028: when a signal is received, terminate all running commands and wait in a WaitGroup.
8028: add command to waitGroup during run method itself; not during signal handling.
8028: In runningCmds loop, no need to wait for process termination.
8028: Update the stub based test to use waitGroup.Wait() instead of sleep.
closes #8028
Merge branch 'master' into 8028-crunch-dispatch-local
refs #8028
6833: add poll_time to CollectionDirectory.
6833: get blobSignatureTtl from discovery document and use it to set the poll_time.
refs #6833
8178: rename Delete api as Trash; add Untrash to volume interface; add UndeleteHandler and test for this endpoint.
8178: All three currently supported volumes return error when trash-lifetime period is not configured. azure blob and s3 volumes are updated to do so.
8198: consider X-Forwarded-For header by way of request.repote_ip while setting node ip address.
8178: rename Delete api as Trash; add Untrash to volume interface; add UndeleteHandler and test for this endpoint.
8178: All three currently supported volumes return error when trash-lifetime period is not configured. azure blob and s3 volumes are updated to do so.
8178: (for now) all volumes must return ErrNotImplemented if trash-lifetime != 0
Merge branch '8178-keepstore-trash-interface' of git.curoverse.com:arvados into 8178-keepstore-trash-interface
8178: rename Delete api as Trash; add Untrash to volume interface; add UndeleteHandler and test for this endpoint.
8178: All three currently supported volumes return error when trash-lifetime period is not configured. azure blob and s3 volumes are updated to do so.
8178: (for now) all volumes must return ErrNotImplemented if trash-lifetime != 0
8178: untrash should fail when ErrNotImplemented is returned.
Merge branch '8178-keepstore-trash-interface' of git.curoverse.com:arvados into 8178-keepstore-trash-interface
refs #8178
Merge branch 'master' into 8198-node-ip-address
closes #8198
8286: add "star" route for projects. Update projects dropdown in breadcrumbs to display only starred projects instead of all my_projects.
8183: show only toplevel projects in the Projects dropdown in breadcrumbs.
8183: set limit on my_toplevel_projects
8183: When there are more than 200 readable projects, build the tree in steps;
8183: add test to check build of my projects tree with the new method; update the method implementation
8079: add uuid to api_client_authorizations
8079: update the migration script to use the api_token as the seed
8079: add uuid to api_client_authorizations_search_index and add uuid to all api_client_authorizations test fixtures.
8079: Added support get using uuid and list using uuid or api_token and added tests.
Merge branch 'master' into 8079-api-client-auth-uuid
Merge branch 'master' into 8286-fav-projects
8079: add down migration to api_client_authorizations_search_index
Merge branch 'master' into 8286-fav-projects
8286: added test "unshare project and verify that it is no longer included in shared user's starred projects"
8286: add an integration test to star / unstar project by clicking on the icon.
Merge branch 'master' into 8183-projects-dropdown
8183: while displaying "my projects" tree, just show only the user's projects and omit shared projects
8183: retrieve only 3 levels of projects while building projects dropdown.
8183: display top three levels of projects in menu, improve message when there are too many projects, improve the test.
Merge branch 'master' into 8079-api-client-auth-uuid
8178: add generic volume tests for trash / untrash interface.
Merge branch 'master' into 8183-projects-dropdown
Merge branch '8183-projects-dropdown' into 8286-fav-projects
8079: add rescue to drop index
8286: include favorites and top-level my projects in projects drowdown.
Merge branch 'master' into 8183-projects-dropdown
closes #8183
Merge branch 'master' into 8286-fav-projects
8286: convert star method into action controller action and refresh the star icon in place rather than a full page refresh.
8286: to facilitate in-place star icon refresh without the whole page refresh, it became necessary
8286: check if project is starred only when current_user is not null (anonymous user case).
Merge branch 'master' into 8079-api-client-auth-uuid
closes #8079
Merge branch 'master' into 8286-fav-projects
8286: also include link to Home in favorites section of the Projects dropdown, so that the user does not have to scroll through all of his favorites to seek in MyProjects section.
8286: bigger star icon with stronger color contrast.
closes #8286
8441: Update project chooser modal to display favorites and top-level-my-projects and remove the now unused build_project_trees and shared_project_tree code.
Merge branch 'master' into 8441-project-chooser
closes #8441
5694: display 2000 historic log lines in running job's Log tab.
7399: throttle partial segments of long lines to once every "n" seconds
Merge branch 'master' into 5694-running-job-log
5694: remove the extra white space in log tab display.
closes #5694
7399: added test for partial line segment throttling.
7399: use regexp to look for partial line segments.
7399: do not handle crunchstat stderr partial line segments in this story.
closes #7399
Merge branch '8508-datamanager-test-badpaths' of https://github.com/wtsi-hgi/arvados into wtsi-hgi-8508-datamanager-test-badpaths
Merge branch '8497-datamanager-batchsize-1000' of https://github.com/wtsi-hgi/arvados into wtsi-hgi-8497-datamanager-batchsize-1000
closes #8497
8558: add min_cores_per_node and min_ram_mb_per_node to runtime_constraints from builder resources
Merge branch 'master' into 8558-cwl-propagate-resource-req
8558: include min_scratch_mb_per_node in the keys propagated
8554: trash and untrash implementation
8554: Add EmptyTrash api to Volume and implementation in volume_unix. Add emptyTrash goroutine.
8508: go fmt
closes #8508
8558: add cwl to jenkins
8558: update field names and ensure the tests pass
8558: compacted code
refs #8558
8558: when cores not found in resources, use default.
6518: create sbatch and strigger commands using func declarations so that they can be overridden in tests.
Merge branch 'master' into 8558-cwl-propagate-resource-req
closes #8558
8554: Untrash returns os.ErrNotExist when no trash found for the given locator and other updates.
8554: improved tests
8554: test improvements
8561: rather than getting node's ipadd, use it's ec2_instance_id
8554: removed testTrashUntrashWithEmptyTrashGoroutine
8561: test tweaks
Merge branch 'master' into 8554-trash-untrash-unix-volume
closes #8554
Merge branch 'master' into 8561-node-pairing
Merge branch '8561-node-pairing' of git.curoverse.com:arvados into 8561-node-pairing
8561: delete redundant "maxDiff = None"
Merge branch 'master' into 8561-node-pairing
closes #8561
Merge branch '8087-arv-cli-request-body-from-file' of https://github.com/wtsi-hgi/arvados into wtsi-hgi-8087-arv-cli-request-body-from-file
7852: add a failing test that does: readline and then readlines. The expectation would be the second call,
7852: Added one more test "test_readline_then_readall" which does a readline followed by a readall.
8703: add "components" hash to job.
Merge branch 'master' into 8703-job-components
8703: better organized tests
Merge branch 'master' into 8703-job-components
closes #8703
7658: add reconnect logic when a websocket is closed unexpectedly.
7658: update EventClient.on_closed to retry on connect errors.
8724: add keep-block-check script
8724: test assertion improvements
8724: performKeepBlockCheck() returns error when any of the listed blocks are not found.
Merge branch 'master' into 7658-websockets-reconnect-on-close
7658: add test that verifies reconnect retry behavior
7658: improve the log verification in case of unexpected close.
7658: update connect error test to use stream handler to read the log file, instead of using a temp file.
Merge branch 'master' into 7658-websockets-reconnect-on-close
closes #7658
8724: test updates
Merge branch 'master' into 8724-keep-block-check-script
8724: some more cleanup of tests.
Merge branch 'master' into 8724-keep-block-check-script
closes #8724
8936: consider blobSigningTtl while generating and verifying signatures.
8936: update blob-signing-ttl related documentation.
Merge branch '8936-ttl-in-signing-key-TC' into 8936-ttl-in-signing-key
8936: address review comments
8936: update comment on keepstore and go fmt
8936: update the blob_test to use a specific blob_signature_ttl to ensure consistent results.
8936: update go tests to use a blob-signature-ttl different than 1s.
8936: updated blob_test.rb to continue to use the default blob_signature_ttl.
closes #8936
8936: update keep-block-check and keep-rsync to properly use blob-signature-ttl to perform the signature verification.
8936: add test to verify blobSignatureTTL from discovery when it is not provided.
Merge branch 'master' into 8936-ttl-in-signing-key
closes #8936
8937: add head request to python keep client.
8937: bypass cache for all head requests.
8937: test updates
8937: Return True for Head requests in KeepClients. The tests in KeepClientRetryHeadTestCase are failing due to this and need to be worked on.
8937: updated arvados_testutil.py to skip setting resp_body to writer when it is a boolean.
8937: invalidate cache and create new one if there are errors on head request during ResumeCache.
8937: refactor cache check logic into a check_cache method and update all references.
refs #8937
8017: pass ram and vcpus runtime_constraints from Container to sbatch command.
8464: Add stdout redirection in crunch2.
Merge branch 'master' into 8017-slurm-runtime-constraints
8017: mem-per-cpu
8017: pass ram and vcpus runtime_constraints from Container to sbatch command.
8017: mem-per-cpu
8464: stdout handling
Merge branch '8017-slurm-runtime-constraints' of git.curoverse.com:arvados into 8017-slurm-runtime-constraints
8017: RuntimeConstraints uses int64
closes #8464
8017: RuntimeConstraints uses int64
closes #8017
8876: add work_unit to workbench models
8876: work_unit views in progress
8876: misplaced divs!
8876: work unit views
8556: implement trash/untrash for azure volumes.
closes #8556
8876: progress bar and strage component display
8876: add an integration with job with components.
8876: remove some job specific bits and make them generic
8876: For jobs also, compute progress from it's children if present; otherwise, use task_summary.
Merge branch 'master' into 8876-work-unit
8876: improve jobs_with_components test to have components that can be un/read
8876: correct the job_reader2 fixture
8876: typo in fixture
Merge branch 'master' into 8876-work-unit
8876: child_summary_str checks if total > 0
8650: add container_work_unit
8876: move some methods such as log and output from job_work_unit into proxy_work_unit to aid reusability.
Merge branch '8876-work-unit' into 8650-container-work-unit
8650: add children to container_work_unit
8876: success? includes Canceled as well.
Merge branch '8876-work-unit' into 8650-container-work-unit
9318: "Active" and "Recently finished" panes in dashboard are updated to use work_unit interface.
9318: Compute node summary pane includes queued and locked containers.
9275: create pipeline_instance in submit mode as well and add the runner job to it's components.
8876: display "no process has been submitted" when a child uuid is not presented.
8876: introduce view helper methods such as link_to_log and queuedtime etc so that the views do not have to do too many decisions based on the state of the work unit.
8876: add tests for link_to_log and queuedtime etc.
Merge branch '8876-work-unit' into 8650-container-work-unit
Merge branch '8650-container-work-unit' into 9318-dashboard-uses-work-units
9275: add record to cwl_runner_job as components
8876: Use JobWorkUnit for pipeline components and cleanup.
Merge branch '8876-work-unit' into 8650-container-work-unit
8650: some more methods in ContainerWorkUnit
Merge branch '8650-container-work-unit' into 9318-dashboard-uses-work-units
9275: Update the update_pipeline_component method to check if pipeline is null.
9318: running and finished containers and fixtures updated.
8876: when computing cpu and running times, use the work unit's start and finished times if there are no children.
Merge branch '8876-work-unit' into 8650-container-work-unit
Merge branch '8650-container-work-unit' into 9318-dashboard-uses-work-units
8876: remove show_child_summary and replace it with is_running?
closes #8876
Merge branch 'master' into 9318-dashboard-uses-work-units
8876: Pass work unit to determine_wallclock_runtime, not the original object.
refs #8876
Merge branch 'master' into 9318-dashboard-uses-work-units
Merge branch '8087-arv-cli-request-body-from-file' of https://github.com/wtsi-hgi/arvados into wtsi-hgi-8087-arv-cli-request-body-from-file
Merge branch 'master' into wtsi-hgi-8087-arv-cli-request-body-from-file
closes #8087
Merge branch 'master' into 8650-container-work-unit
8650: test and fixture update
Merge branch '8650-container-work-unit' into 9318-dashboard-uses-work-units
9318: remove :output method in favor of :outputs method and correct the logic for various object models.
Merge branch 'master' into 9318-dashboard-uses-work-units
9318: Update the compute node status pane to make sure the Details option is only offered when there are any active nodes.
Merge branch 'master' into 9318-dashboard-uses-work-units
closes #9318, closes #8650
9274: while creating a ContainerRequest, set requesting_container_uuid based on the token.
9318: fixed outputs display issue where "outpus: []" is being shown when there are no outputs.
refs #9318
9372: container display
closes #9274
Merge branch 'master' into 9372-container-display
9372: test cancel button for jobs and container_requests.
9372: name and description can be edited on containers and container_requests despite the state.
9372: Corrected dashboard display to list container_requests, not containers as implementd in #9318.
Merge branch 'master' into 9372-container-display
9372: variable names and comments.
9372: The children of a Container will be ContainerRequests to which this is the requesting_container_uuid.
9372: improve container_work_unit -> uri method to check if @proxied will respond_to table_name instead of using a rescue.
Merge branch 'master' into 9372-container-display
closes #9372
9407: container log view
closes #9407
9498: show top-level container requests in project#Pipelines_and_processes tab.
closes #9498
9319: /all_processes index page.
closes #9319
9552: update cwltool version, which offer compute_checksum flag.
8568: Enhance image_hash parsing when using Docker 1.10+
Merge branch 'master' into 9552-compute-checksum-flag
closes #9552
Merge branch 'master' into 8568-docker-version-support
closes #8568
9581: add json config file handling to slurm dispatcher.
closes #9581
9552: add compute-checksum flag to arvados-cwl-runner
refs #9552
9684: add workflow resource to api server
closes #9745
9658: when arvados-cwl-runner is invoked with --api containers, display the container_request uuid to console.
9678: enhance container log tab display to consider the "combined" object logs.
9678: enhance test to verify that the combined logs are displayed.
closes #9658
9709: do not include manifest_text in collection logs.
closes #9678
closes #9709
9678: use safe_yaml and other updates.
9684: workflow yaml error logging
closes #9684
9684: Add safe_yaml initializer
9684: update arvados_model -> recursive_stringify to convert ":foo" to "foo"
9684: update arvados_model -> recursive_stringify to convert ":foo" to "foo"
9767: include workflows in the template chooser.
refs #9767
9043: added tests for various input types
9043: minor test tweaks
9824: include workflows in project#pipeline_templates tab.
9767: add a link to workflow in the container_request#show page.
closes #9767
closes #9824
9893: update java sdk test files to match the pom.xml updates.
closes #9893
9898: add lock and unlock endpoints to containers_controller.
9898: update crunch-dispatch to use lock and unlock apis instead of setting state directly.
9898: controller test with lock / unlock state transitions.
9898: test updates
9898: update crunch-dispatch-local tests to be in sync with api server updates.
9835: Container and ContainerRequest documentation.
Merge branch 'master' into 9898-container-lock-api
9898: add Lock and Unlock methods to dispatch go sdk.
9898: add unlock method also on the container model.
9835: more details such as mount_types added.
10005: Rename workflow attribute in workflow object as definition.
10005: update workflow fixture name and description as well and update workbench invocations of workflow.workflow.
closes #10005
9836: Add workflow to api documentation.
9835: mount_types improvements.
9836: Remove the workflow definition related discussion and provide a link to CWL instead.
Merge branch 'master' into 9835-container-api-doc
closes #9835
Merge branch 'master' into 9836-workflows-doc
closes #9836
9970: Invalidate record if a requesting_container_uuid is provided at create.
closes #9970
9836: Add container reuse to ContainerRequest documentation.
refs #9835
9514: add rake task to delete logs for old containers whose log is already stored in keep.
10036: Use status label "Ready" for a canceled container request (priority 0), when it's container_uuid is in Queued or Locked states.
closes #10036
10028: replace _____-uuid-prefix with site_config_uuid_prefix-uuid-prefix
Merge branch 'master' into 9514-only-delete-old-container-logs
9514: fixture correction
9514: if logs are not found in logs table, do not show the log div for a completed container.
9514: Use sql to issue delete sql without getting ids in batch.
9514: update delete_old_job_logs task also to use the better performing sql.
refs #10028
closes #9514
refs #9498 Fix failing workbench integration test due to containers fixture update.
10078: Update the Node.filter so that this query is not executed three times: once for nodes.sort_by and twice in compute_node_summary for nodes.select
10078: Update recent_collections so that the collection query it is not executed twice, once in this method and once in show_dashboard for r[:collections].each statement.
Merge branch 'master' into 10078-dashboard-perf
10078: Restore master version
10078: preload collections and links during dashboard display
10078: config parameter to hide / show recent_collections panel in dashboard.
10078: add configuration parameter to suppress display of user notifications to improve dashboard performance
10078: Now that the recent_collections panel includes collections from Home directory as well
10078: When fetching recent_processes, do not select 'components' which is not needed in this context. For very large pipelines, this will perform better.
10078: Since we are not fetching components for pipeline instances (at least until the performance issues are resolved),
10078: fix failing test due to new fixture addition
10078: Update the Node.filter so that this query is not executed three times: once for nodes.sort_by and twice in compute_node_summary for nodes.select
10078: Update recent_collections so that the collection query it is not executed twice, once in this method and once in show_dashboard for r[:collections].each statement.
10078: preload collections and links during dashboard display
10078: config parameter to hide / show recent_collections panel in dashboard.
10078: add configuration parameter to suppress display of user notifications to improve dashboard performance
10078: Now that the recent_collections panel includes collections from Home directory as well
10078: When fetching recent_processes, do not select 'components' which is not needed in this context. For very large pipelines, this will perform better.
10078: Since we are not fetching components for pipeline instances (at least until the performance issues are resolved),
refs #10078
Merge branch 'master' into 10078-dashboard-perf
10078: Rename the workbench config param 'show_user_notifications_on_dashboard' as 'show_user_notifications' because they are in topnav, not on dashboard!
refs #10078
10144: Replace work_unit -> link_to_log with partial so that preloaded objects are used.
10144: Update link_to_if_arvados_object to use collection_for_pdh when resource_class is a Collection and attrvalue is a pdh.
10144: preload repositories during work unit component display.
10144: preload work_unit children
10144: Preloading all the collection using one single large query is much slower than making several requests
10144: Rename :by_name as "by_attr" for preload_objects_for_dataclass
10144: Preload children during (main) component display. The Job.where in pipeline_instance_work_unit is fetching all jobs even if they are not readable.
10144: Conver the link_to_log tests into controller tests so that the #show is executed.
10144: limit find_object_by_uuid update to the case where find is used; this will ensure that the object is not fetch again when show is used.
10144: call object_for_dataclass only when attr value is not-nil.
refs #10144
10227: The diagnostics tests have been consistently failing since Sep 18th due to web sockets based refresh is not happening in the page.
refs #10227
10231: add keep_cache_ram in runtime_constraints
10231: update arvados-cwl-runner for keep_cache_ram runtime_constrant.
10231: update crunchrun to pass Container.RuntimeConstraints.KeepCacheRAM as file-cache argument to arv mount.
10079: Move Log and Output links into child details. This will make it possible
10079: ajax to display child component details; not working yet.
10079: show_child_component method needs anonymous access
10079: checking log_link on children needs to be an integration test.
10079: fixed route so that @object is available
10079: add "parent" to work_unit to aid the child display
10079: show spinner when a child component details are being displayed using ajax.
closes #10079
Merge branch 'master' into 10231-keep-cache-runtime-constraints
10231: add keep_cache_ram to doc
10028: when an "is_a" filter is used while fetching contents, do not search in all "other" types.
refs #10028
closes #10231
10231: validate_runtime_constraints should occur before set_container filter.
refs #10231
10310: invoke index and show actions only when they are not blacklisted by the api server.
closes #10310
10293: add output_uuid to container_request and set it during finalize.
10290: Add scheduling_parameters map to containers and container_requests, and move partitions array out of runtime_constraints into this.
10290: update test assertions
10290: Add scheduling_parameters map to containers and container_requests, and move partitions array out of runtime_constraints into this.
closes #10290
Merge branch 'master' into 10293-container-request-output-uuid
10293: added log_uuid to container_requests
Merge branch 'master' into 10293-container-request-output-uuid
10293: add output and log uuids to containter_request search index.
Merge branch '10290-container-partitions' of git.curoverse.com:arvados into 10290-container-partitions
Merge branch 'master' into 10290-container-partitions
10290: scheduling_parameters added to container_requests controller.
10293: update cwl runner
Merge branch 'master' into 10293-cwl-cr-output
closes #10290
Merge branch 'master' into 10293-container-request-output-uuid
Merge branch 'master' into 10293-cwl-cr-output
Merge branch '10293-container-request-output-uuid' into 10293-cwl-cr-output
10293: add scheduling_parameters to containers controller
Merge branch 'master' into 10293-container-request-output-uuid
refs #10293
Merge branch 'master' into 10293-cwl-cr-output
10293: update documentation with scheduling_parameters, log_uuid and output_uuid.
Merge branch 'master' into 10293-cwl-cr-output
10524: update workbench titles to use "processes" instead of "pipelines and processes" when pipelines are disabled.
10524: update tests to look for 'Run a process' button instead of 'Run a pipeline'
10524: add additional test assertions to disabled_api_test
closes #10524
Merge branch 'master' into 10293-cwl-cr-output
10293: invoke output_callback in a try/except block
10293: done_outputs in a try/except block
Merge branch 'master' into 10293-cwl-cr-output
10524: update test assertions
refs #10524
10290: scheduling_parameters documentation
refs #10290
Merge branch 'master' into 10293-cwl-cr-output
10518: update API groups_controller.contents method to exclude classes with index disabled and update user guide to use "Run a process" instead of "Run a pipeline"
closes #10518
10604: Update the "Running a pipeline using Workbench" to talk about "workflows"
10604: add pages to write and run workflows.
10293: update container_work_unit to look for output_uuid and log_uuid for a container_request, rather than using those from the associated container.
10604: add crunch1 only note to arv-copy documentation.
Merge branch 'master' into 10293-cr-log-and-output-in-wb
10293: update test assertion to expect output collection name, instead of pdh.
refs #10293
Merge branch 'master' into 10519-cr-fiddlesticks
10519: added a test to check Run button when an empty workflow is used with no required inputs.
10215: if version arg is used, just display version info and exit.
Merge branch 'master' into 10604-crunch2-user-guide
10604: rename tutorial-pipeline-workbench.html as tutorial-workbench-workbench.html
closes #10604
Merge branch 'master' into 10215-arvados-cwl-runner-version
closes #10215
10300: old pipeline display is failing due to min_nodes in runtime_constraints being a string; convert to integer to be able to view it.
closes #10300
10677: Add a Re-run button to container_request#show page.
closes #10677
10028: upgrade full text search indexes to not include leading space.
10241: add a diagnostics test to run a process using a workflow.
refs #10241
refs #10028
10516: set finished_at to updated_at on pipeline_instances if the pipeline is finished but the finished_at is not set.
9831: update test that started failing due to unique name collision logic update; removed the second "Create a new project" from test since this is already tested on the API server end.
refs #9831
refs #10516
10516: compute cputime recursively using cputime of any children.
10516: update test to expect 0 cputime, instead of nil.
10516: update work_unit test to compare cputime and walltime.
closes #10516
10110: arv-copy workflows
10110: also copy any docker images during arv-copy of a workflow and update documentation.
10110: add ruamel.yaml to setup.py
closes #10110
5666: Use "warning" bootstrap style for Cancel button display.
closes #5666
9397: raise error if there any writable mount point underneath the output_dir.
9397: only mount points of kind "collection" are allowed under the output_dir.
9397: pre-populate output with mounts under output_dir (wip)
9397: testing in progress for mount points under output_dir.
9397: test with multiple mount points under subdirectories
9397: more testing
Merge branch 'master' into 9397-prepopulate-output-directory
9397: if mount.Path ends with "/", trim it.
9397: update comments
9397: Add ExcludeFromOutput to Mount struct.
9397: Use manifest.FileSegmentForPath to get manifest segment for a file path. Cache collections to avoid fetching
9397: get manifest segment for a subdir or file using manifest from sdk.
9397: NormalizeManifest
9397: add handling of denormalized manifest
9397: add normalized manifest test
Merge branch 'master' into 9397-prepopulate-output-directory
9397: add pre-population of output directory documentation
10903: Add cancel method with cascade to pipeline_instance and update a job's cancel method to accept a cascade parameter.
10903: use cancel with cascade to cancel jobs and pause pipelines.
10945: show jobs in which this job is used in job#show page.
10945: "Used in jobs" panel placement
10903: support need_transaction for job and pipeline_instance cancel methods.
closes #10945
closes #10903
10979: trackContainers func in crunch-dispatch-slurm.go
11103: Omit mounts from container_requests and definition from workflows for full text search index.
11103: ensure the index exists before attempting to drop it.
closes #11103
11104: add fa-caret-down class to hyperlink rather than adding a span with icon.
closes #11104
Merge branch 'master' into 10979-cancelled-job-nodes
Merge branch 'master' into 10979-cancelled-job-nodes
10979: TrackContainer implementation and test
10979: scancel orphaned job nodes in crunch1.
10979: refactor squeue invocations
Merge branch 'master' into 10979-cancelled-job-nodes
10979: ruby way of doing it
Merge branch 'master' into 10979-cancelled-job-nodes
closes #10979
10979: fix failing test
refs #10979
11015: use multiple writers to increate throughput of goUpload.
Merge branch 'master' into 11015-crunch-run-output-upload
closes #11015
11017: Use count='none' for full text search on workbench.
Merge branch 'master' into 11071-fts-perf-test
11071: test count=none in groups#contents method.
11071: formatting on the long if statement
closes #11071
11290: Reverted "fa-caret-down" usage back to 'span class="caret"' with href added to it. Working with Firefox and Chrome now.
closes #11290
3821: added 'Remove selected files' to collection#show page.
3821: remove selection file in collection file row
3821: Rename collection file path
3821: collection controller tests for remove and rename files.
3821: integration test to remove files from a collection
3821: add an integration test to rename a file in a collection
3821: attr_accessor not needed on collection "filename"
3821: The "Remove selected files" option should only be presented if the user can update the collection.
3821: collection viewed from another object
Merge branch 'master' into 3821-collection-file-manage
3821: update test assertion that randomly fails; in fact, I don't see how it ever worked with that syntax.
3821: improve the tooltip on rename file option.
closes #3821
9132: Use ThinDockerClientProxy to overcome the import issue around "github.com/docker/docker/vendor/golang.org/x/net/context".
11368: show error if user tries to rename a collection file with a duplicate name.
closes #11368
8465: added stdin redirection for collection
8465: added stdin redirection for json mount.
8465: test clean up
Merge branch 'master' into 8465-stdin-redirection
8465: do not wait in the go routine while doing io.Copy of stdin and also invoke runner.stop on error.
8465: added stderr redirection and tests
8465: test cleanup
Merge branch '8465-stdin-redirection' into 8465-stderr-redirection
Merge branch 'master' into 8465-stderr-redirection
refs #8465
11365: "lock" collection to prevent user from deleting / renaming files until user explicitly unlocks by clicking the button.
closes #11365
11465: add tooltips to disabled collection file controls
11465: add tooltips to disabled collection file controls
11465: correct disabled tooltip behavior on image types
closes #11465
11465: tooltip on disabled pencil icon
refs #11465
8019: expose crunch log throttling config parameters via discovery document
11450: workflows index page
closes #11450
8019: rateLimit crunch-run logging using API configuration parameters
Merge branch 'master' into 8019-crunchrun-log-throttle
8019: add tests and address issues identified
8019: more testing
8019: load rate limiting config parameters from discovery document once per log writer creation.
Merge branch 'master' into 8019-crunchrun-log-throttle
8019: rateLimit method signature
8019: load log throttling config params during NewContainerRunner
10112: add Run button to workflow#show page
8019: partial line throttling etc
10645: container request mounts display
8019: need to check and write log if things are closing down
Merge branch 'master' into 10645-cr-mounts-display
10645: add test for container_request display with input mounts
Merge branch 'master' into 10112-workflow-show
Merge branch 'master' into 10645-cr-mounts-display
10645: rename file
11250: when a container's priority is zero and in Queued state, show the state as "Cancelled"
11522: remove link to users storage page in users activity page.
closes #11250
Merge branch 'master' into 10112-workflow-show
closes #11522
8019: uncomment the test statement
10645: mount inputs display
Merge branch 'master' into 10645-cr-mounts-display
Merge branch 'master' into 10645-cr-mounts-display
10645: small style updates
Merge branch 'master' into 8019-crunchrun-log-throttle
closes #8019
10645: corrections to test fixture
Merge branch 'master' into 10645-cr-mounts-display
Merge branch 'master' into 10112-workflow-show
10112: add Definition tab to workflow page
10645: table width 100%
10645: improve location regex usage
Merge branch 'master' into 10645-cr-mounts-display
10645: L&F improvements
10645: display keep locator in mounts as pdh/file_name
10645: display PDHs in command as clickable collections.
10645: test update
10645: regexp for keep_locator_in_json format
10645: collection links in command
10645: refactor the display of text with locators into a partial
Merge branch 'master' into 10645-cr-mounts-display
10645: added comment
refs #10645
Merge branch 'master' into 10112-workflow-show
10112: use show_text_with_locators partial to display the various attributes of the definition
10112: preload collections
Merge branch 'master' into 10112-workflow-show
closes #10112
10645: display collection link with disposition. preload mount and command collections.
Merge branch 'master' into 10645-cr-mounts-display
closes #10645
11580: add container_requests index page
11580: preload containers
closes #11580
9587: add support for include_trash in groups_controller -> contents method
refs #9587
11710: preload / batch retrieval of children of a container_work_unit
11710: fetching requesting containers
11710: fix typo in finding children
closes #11710
11739: preload containers and children of all container_requests in dashboard display.
closes #11739 , #11751
9587: trash page
9587: tab line buttons in trash page
9587: untrash option
Merge branch 'master' into 9587-trash-page
9587: expose untrash api
9587: display name instead of clickable link and remove tags
9587: cleanup trashed_items method
9587: test /trash page
9587: include "owner" in trash rows; update trash icon.
4921: clear preview pane when the selectable container is cleared
closes #4921
9587: sort order issue
9587: show trash page contents in Recent tab pane
4921: also clear preview pane when a project is selected from projects dropdown menu
closes #4921
9587: trash page
9587: expose untrash api
9587: sort order issue
closes #9587
Merge branch 'master' into 9587-trash-page
9587: sort the trash page rows by trash_at timestamp
9587: rename pane name to "Recent trash"
closes #9587
Merge branch 'master' into 11652-recursive-contents
11652: use "recursive=true" in workbench search requests
Merge branch 'master' into 11652-recursive-contents-wb
11652: fix comment on test
closes #11652
9587: Show "delete_at" timestamp instead of "created_at" timestamp in trash page
refs #9587
11837: write tests
closes #11837
11821: add include_trash for collections.index and recursive for groups.contents to discovery doc
closes #11821
11823: document collection lifecycle
11060: show all repositories that are accessible to the user in <user_uuid>/repositories page
11823: update title as "keep collection lifecycle"
11823: update collection lifecycle documentation to use "index" action instead of "list" action to list trashed collections
Merge branch 'master' into 11060-all-visible-repos
11060: accessbile repositories query
11060: test updates
11060: sort order and delete prompt updated
Merge branch 'master' into 11060-all-visible-repos
closes #11060
11823: few small doc updates
Merge branch 'master' into 11823-collection-lifecycle-doc
closes #11823
11840: support ensure_unique_name parameter in collection untrash method
11846: update file path parameter name in renaming collection image files and add a test assertion
closes #11846
11840: include ensure_unique_name parameter in untrash requests from workbench.
9426: collection Tags tab (in progress)
9426: tags display
9426: delete all tags
9426: test Tags tab
closes #9426
Merge branch 'master' into 11840-unique-constraint-untrash-coll
closes #11840
11860: list method is a duplicate of index method
closes #11860
9426: save "tags" as "properties" on collection, rather than as a field in properties.
closes # 9426
11870: add infinite scrolling and "add new repository" button to repositories index page
11870: link repositories menu item in user menu to repositories index page.
11870: update tests to be compatible with the user menu repositories link change.
11870: minor updates
11870: minor update
Arvados-DCO-1.1-Signed-off-by: Radhika Chippada <radhika at curoverse.com>
11870: add copyright note to new files
Merge branch 'master' into 11870-repositories-index
closes #11870
11857: arvados-cwl-runner defaults to crunch2 if available
refs #11857
11682: when a container_request is being destroyed, update it's container's priority
closes #11682
11682: add infinite scrolling to container_requests index page test.
refs #11682
11906: add _health/ping to api server
11906: keepstore ping
11906: pass a func() to healthCheckDo from HealthCheckPingHandler
refs #11906
11906: added api ping tests
11906: add _health/.* to errors_test routes
refs #11906
11906: healthcheck ping json updated
refs #11906
11906: healthcheck ping for workbench
refs #11906
1195: add .bed and .cwl to file types that can previewed
closes #11995
11906: add /_health/ping to keepproxy
refs #11906
11906: add /_health/ping to keep-web
11906: add /_health/ping to arv-git-httpd
refs #11906
refs #11906
11906: add /_health/ping to nodemanager
11906: use the same name "ManagementToken" to enable healthcheck in all services
closes #11906
radhika chippada (158):
minor updates to user guide
Merge branch 'master' into 2107-radhika-user-guide-minor-updates
Fix Task#2268 by removing the extra space character
Merge branch '2107-radhika-user-guide-minor-updates'
The very first set of unit tests for rails API server
Add functional test to create a new user
WIP - update the create action in users controller to create user object and also create the links
Merge branch 'master' into 2187-enhance-user-setup
create and some links tested
Merge branch 'master' into 2187-enhance-user-setup
More accomplished
Test just_probe and openid_prefix options
All links in place
A simplified version of setup_new_user.rb script, where all the create user and links logic is moved into users controller.
Fix an error introduced elsewhere in the file.
Fix a nil check
Added functional tests for controller updates
Functional tests to verify links
Remove force option
Some more refactoring
remove hard coded uuids from tests
More asserts for links
When using user_param, the user object is not expected. If passed, it will be discarded.
When create is invoked with user object as input, check if object exists for the given uuid or email.
Update comments in the tests
Remove -n aka just probe option from the setup-new-user script as per Ward's core review recommendation.
Replace tabs with spaces as per our coding standards
Merge branch 'master' into 2187-enhance-user-setup
Use rails configuration for default openid prefix.
Introduce a new setup method in the users controller
Update the functional tests to use the setup method
Use setup method instead of create method to setup a new user
setup-new-user script using the setup method.
Merge branch 'master' into 2187-enhance-user-setup
testing setup method implementation
All tests passing
All functional tests passing with refactored code.
some more refactoring and attention to coding standards
Pass openid_prefix to setup method only when -o option is used
Remove print statement
One more print statement
Add negative functional tests where a non-admin user invokes create or setup actions
Expect openid_prefix from the clients instead of managing it on the api server.
Render 404 if no user is found for the uuid provided
Repo name and VM are optional to the setup method
Include all the newly created link objects in the response from the setup method. Update all functional tests to verify links found in teh response.
Make sure oid_login_perm exists when setup is executed for an existing user
Fix tabs in the test class
Raise exceptions on errors such as 'All users' group does not exist and a vm for the vm_uuid param does not exist. Add additional functional tests and update previous tests accordingly.
Raise ArgumentError when required openid_prefix is not passed.
Raise error if both uuid and user.email are omitted in input parameters
Merge branch 'master' into 2187-enhance-user-setup
Include first link of a given type even on reruns of the setup method.
Merge branch 'master' into 2187-enhance-user-setup
Add unit test for user model
More unit testing
Merge branch 'master' into 2187-enhance-user-setup
When uuid is passed, openid_prefix can be optional
some minor refactoring
Update user model to return a List instead of HashList
Update all functional tests to work with the new response format of List
Ground work to remove found object logic from user model's setup method
setup method in user model assumes that the user object passed in is valid and hence no longer tries to find it.
The controller invokes the create links method from the model when working on an existing user.
Minor test updates
Merge branch 'master' into 2187-enhance-user-setup
Expose user unsetup method from the API server. Added functional tests.
Story #2068: Invoke the user -> unsetup method when an admin user clicks on the deactivate button in workbench
Use post method instead of get method when invoking the unsetup method from the api server.
Update the text on the deactivate link
Merge branch 'master' into 2068-reset-user
Update UI to use button_to instead of link_to to deactivate the user
Improve the api call invocation
Use before_filter admin_required in the users controller for setup and unsetup actions
Update tests to match controller updates towards admin_required
Clean up the users controller
minor cleanup
reload object before unsetup
Merge branch '2068-reset-user'
Add users integration test class. Beginnings of setup and unsetup method tests.
Confirm dialog check commented our for now. Why is that I can't intercept the dialog and instead why is the OK button clicked automatically?
Add a setup action to users controller and user model
Working on setup popup
Setup popup display
Working on setup popup
Popup closes on submit
Added bootstrap css
Multiple input elements in popup
Disable submit button until all input fields are filled in.
Pass all parameters to the setup method
Minor UI tweaks
Virtual machines in a dropdown option in setup popup
"Add a new user" link uses setup method to create a "loaded" user.
Pre-fill form fields with previous selections where applicable.
New user creation using setup action is working
Require only email and openid prefix in the setup popup. The other fields are optional.
Look for repo_name in VM link
Get the default openid_prefix from config.
popup submit button
Submit button enabled when existing user
Submit button enabling revisited
Integration test for setup user
Added integration tests for create new user, setup and unsetup existing user. The tests use selenium driver without headless. Hence, we see the browser while these tests are running.
Use webkit driver for Capybara instead of selenium for tests that need dialog handling.
Need to remove the user's "all users" group read permission during unsetup.
During setup of an existing user, create oid login permission, if it does not already exist.
Add email format validation to the input text field in the setup modal.
Add header and footer to the setup user modal
Bootstrap modal css updated to include only what is needed.
Submit
Reset form fields on cancel.
Add paste event check for submit button validation
Remove trailing white spaces.
Move the javascript into a js file instead of using <script> tag in html file.
Added bootstrap css
Disable submit button until all input fields are filled in.
Minor UI tweaks
"Add a new user" link uses setup method to create a "loaded" user.
New user creation using setup action is working
Require only email and openid prefix in the setup popup. The other fields are optional.
Look for repo_name in VM link
Get the default openid_prefix from config.
popup submit button
Submit button enabled when existing user
Submit button enabling revisited
Integration test for setup user
Added integration tests for create new user, setup and unsetup existing user. The tests use selenium driver without headless. Hence, we see the browser while these tests are running.
Use webkit driver for Capybara instead of selenium for tests that need dialog handling.
Need to remove the user's "all users" group read permission during unsetup.
During setup of an existing user, create oid login permission, if it does not already exist.
Add email format validation to the input text field in the setup modal.
Add header and footer to the setup user modal
Bootstrap modal css updated to include only what is needed.
Submit
Reset form fields on cancel.
Add paste event check for submit button validation
Remove trailing white spaces.
Move the javascript into a js file instead of using <script> tag in html file.
Story #1776: Add mailer and update users_controller. Also, update workbench and setup script to pass the additional send notification parameter.
Merge branch 'master' of git.curoverse.com:arvados
Use selenium driver with headless for testing.
Story 1776: Add tests for setup email
Story 1776: Add theme to api server similar to workbench. Updated files: Gemfile, app/controllers/application_controller.rb, config/application.default.yml, config/routes.rb
Merge branch 'master' of git.curoverse.com:arvados
Merge branch 'master' into 1776-setup-user-email
Comment out failing assertions for now.
Fix the users test to get the created user id
Merge branch 'master' of git.curoverse.com:arvados
Merge branch '2272-fix-setup-user-in-workbench'
Merge branch 'master' into 1776-setup-user-email
Merge branch 'master' into 1776-setup-user-email
2352: Add state to pipeline_instance. Db migration and unit testing.
Set state during save
Rename is_ready method to components_look_ready? and add additional tests for state change.
Use the renamed components_look_ready? method during migration.
Merge branch 'master' into 2352-phased-pipeline-instance-state
Merge branch 'master' into 2352-phased-pipeline-instance-state
Story 2352: Adding state and components_summary is completed.
sguthrie (2):
Closes #7235. Instead of setting KeepService's pycurl.TIMEOUT_MS, set pycurl.LOW_SPEED_LIMIT and pycurl.LOW_SPEED_TIME.
Closes #7235. Merge branch '7235-python-keep-client-timeout'
tetron (1):
Merge pull request #1 from gitaway/patch-1
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list