[arvados] created: 2.6.0
git repository hosting
git at public.arvados.org
Thu Apr 6 18:30:40 UTC 2023
at 5bebbc28836ffa3a722fd514e27919916b4e3437 (tag)
tagging 3e0786c27f85e715ac3d624a397ad84ae938384d (commit)
replaces 2.5.0
tagged by Peter Amstutz
on Thu Apr 6 14:29:44 2023 -0400
Release notes at https://arvados.org/release-notes/2.6.0/
AnetaSta22 (1):
speed up write function
Brett Smith (79):
Update container state table headers
19895: Upgrade google-api-python-client
19894: Copyedit the 2.5.0 PostgreSQL upgrade note
19886: Bugfix UUID generation in crunch-run tests
19886: Refine crunch-run output collection tests
19886: Save log collection when crunch-run starts a container
Merge branch '19895-google-api-python-client-2'
19886: DRY up test code for better readability
19886: Tighten test regexps for better validation
19886: crunch-run records initial log with PDH
19886: Correct API documentation notes about container logs
19886: Update API documentation notes about container request logs
19917: Synthesize scheduling parameters when retrying a container
Merge branch '19917-retry-scheduling-parameters'
19886: Slim down API documentation about container logs
19886: Describe what component updates container logs more generically
Merge branch '19886-crunch-run-early-log-commit'
19960: Organize systemTagKeys
19960: Add container_uuid to systemTagKeys
19960: Fix container_uuid property name for intermediate collections
19791: Remove Python SDK "Notes" section
19792: Add anchors to some dedicated API methods
19792: Revamp the Python SDK cookbook
19792: Update sharing link recipe to handle v2 API tokens
19792: Reorganize cookbook vocabulary section into properties section
19792: Add cookbook recipe to update properties generally
19792: Add cookbook recipes for managing permissions
19792: Add missing words
19792: Simplify example for modifying permissions
19792: Correct filter for revoking permissions example
Merge branch '19960-vocabulary-system-keys'
19792: Add overview sentence to PySDK cookbook introduction
19792: Remove "none" from list of permission names
19792: Expand explanation of vocabulary in PySDK cookbook
19792: Split collection recipes into separate cookbook sections
19792: Split container request recipes into separate cookbook sections
19792: Add PySDK cookbook recipe to walk a collection
19792: Split container request queue recipes into separate sections
19792: Improve wording of keyset_list_all first argument comment
19897: Return ENOTSUP when the user tries to make a file in a project
19897: Add tests for writing immutable mount objects
19792: Add 'wb' as an example Collection.open mode argument
Merge branch '19792-pysdk-cookbook'
19792: Fix typos throughout
19792: Use binary mode in cookbook download+upload examples
Merge branch '19897-fuse-enotsup'
19926: Flatten structure of PySDK install instructions
19926: Use noteboxes consistently in PySDK install instructions
19926: Modernize pip install instructions
19624: Layout container request lifecycle in two columns
19926: Remove "option 1/2" remnants from the Python SDK install docs
Merge branch '19926-pysdk-install-docs-fix'
20137: Fix incorrect keyword in RailsAPI config reload loop
19980: Recognize the arv: prefix for all system properties
19980: Expand documentation of reserved properties
20137: Fix incorrect keyword in Workbench1 config reload loop
20138: Avoid timing failures in TestSyncNonCanonicalManifest
Merge branch '20138-fix-sync-test-race'
19980: System properties only require arv: prefix
Merge branch '19980-arv-prefix-properties'
19986: Separate collection of cgroup memory stats
19986: DRY up logger setup in crunchstat tests
19986: Log when a container uses nearly max RAM
19986: Log max resource usage after a container finishes
19986: Prevent division by zero when reporting max disk usage
Merge branch '19986-crunchstat-max-reporting'
16325: Expand tests for container request collection finalization
Merge branch '16325-test-container-collection-properties'
16325: Document the purposes of the different log collections
19981: Generalize vary_parameters test method
19981: Reuse containers with various keep_cache constraints
19981: Add config note about impact of changing DefaultKeepCacheRAM
19981: Document the keep_cache_disk runtime constraint
19981: Fix verb tense typo
19981: Explain how keep_cache_* runtime constraints affect reuse
Merge branch '19981-reuse-flex-keep-cache'
19967: Update container cost when crunch-run receives SIGUSR2
Merge branch '19967-crunch-run-cost-updates'
19889: Add an id to the container log method documentation
Konrad Rudolph (2):
Fix collection tree caching bug
Add unit test
Lucas Di Pentima (37):
Adds Arvados legal logo to the doc site. Refs #20079
20035: Ensures proper permissions on places where certificate's keys are saved.
20035: Manages AWS secret and gives read access to service nodes.
20035: Adds configuration to enable the use of encryped SSL private keys.
20035: Adds pillars & states to retrieve the password from AWS secrets manager.
20035: Updates nginx pillars to read the certificate privkey password.
20035: Improves the connector script in various ways.
20035: Improves file/dir permission setting.
20035: Enhances code redability, quotes jinja vars as yaml values.
20035: Manages named pipe file from systemd instead of salt.
20035: Add missing jinja vars quoting on systemd's unit file template.
20035: Adds a warning on systemd's unit file re: runtime dirs.
20035: Fixes systemd unit Exec* value quoting.
Merge branch '20035-salt-installer-privkey'. Closes #20035
20074: Disables statement timeout on rails migration.
Merge branch '20074-bigint-migration-fix'. Refs #20074
Fixes statement_timeout unit handling on int->bigint migration. Refs #20074
Adds 'awscli' package to the salt installer state for encrypted ssl privkeys.
19937: Avoids processing empty input (e.g: when statically linked).
19937: Only output development versions.
Merge branch '19937-build-packages-fix'. Closes #19937
20270: Splits hosts into public & private, saving on public IPs requirements.
20270: Copies .gitignore file to avoid adding binaries to the repository.
20270: Refactors the VPC code so that private nodes can access the Internet.
20270: Removes keep1 & keepproxy nodes. Uses SSH jumphost to deploy nodes.
16379: Adds prometheus service for monitoring arvados & postgresql.
16379: Adds prometheus' node_exporter to all nodes.
16379: Moves prometheus to its own subdomain instead of 'mon.prefix.domain.tld'
16379: Adds basic auth to the prometheus UI.
16379: Adds grafana with default dashboards, moves prometheus hostname.
16379: Adds nginx dependency to the htpasswd setup.
16379: Makes sure the nginx formula is included in the keepweb's top file.
16379: Sets grafana's admin user password through the CLI.
16379: Removes prometheus config for keep1 & keepproxy nodes.
16379: Fixes missing 'monitoring' role on the workbench node.
Merge branch '20270-salt-installer-less-instances'. Closes #20270
Merge branch '16379-installer-prometheus-grafana'. Closes #16379
Peter Amstutz (85):
Update version numbers
19425: Add experimental --fast-parser
19425: cwltool dev version bump
19425: cwltool dev
19425: Fix passing --fast-parser
19425: Clear codegen_idx before reloading
19425: Set loadingContext.skip_resolve_all
19425: Bump to released cwltool & schema-salad
19425: Fix rebase error
19385: Messy work in progress for uploading workflows to collections
19385: new_upload_workflow work in progress
19385: Work in progress checkpoint, submitting uses wrappers
19385: Fixing tests
19385: More test fixing
19385: Continuing to fix tests
19385: Fix submit secrets test
19385: update_refs fixes prefixes
19385: Fix submit_secret
19385: Fixing tests WIP
19385: Unit tests pass
19385: Reference rewriting fixups
19385: WIP for complex schemadef + $import case
19385: more WIP
19385: Finally passing schemadef test.
19385: Down to just a few failing conformance tests, I hope
19385: Passing the v1.2.1 conformance tests!
19385: fast-parser fixups
19385: Fix typo, passing with --fast-parser now
19385: Update cwl dependencies to stable releases
Merge branch 'main' into 19385-cwl-fast-pack
19385: Fix tests because wrapper collection now includes original files
19385: Clean up debug prints
19385: Replace upload_workflow code
19385: Fix test_copy_deps
19385: Fix tests
19385: Add comments and minor reorganization
Rename Workbench.BannerURL to Workbench.BannerUUID
Merge branch '19385-cwl-fast-pack' refs #19385
20096: Tweak test_with_arvbox.sh
20096: Report cwl runner version too
Merge branch '20096-cwltest' refs #20096
20182: Add option to limit the number of supervisor containers
Add workaround for 'ffi' build in debian11 package container
Add --disable-binary to rvm install for debian 11 for upstream bug
20182: Add 'supervisor' to default scheduling params
20182: API server sets "supervisor" flag now
Merge branch '20182-supervisor-limit' refs #20182
20200: Add limiter for log create requests
20200: Add test for limiting log create requests
19975: Initial work to auto-retry on OOM
19975: Retrying RAM wip
19975: Add OutOfMemoryRetry extension
20200: Cleanups, mention changes in upgrade notes
20200: Fix defer mistake
20200: Fix test typo
19975: Add integration test for out-of-memory resubmit
20200: Set MaxConcurrentRequests for test
Merge branch '20200-log-limit' refs #20200
Merge branch '20183-update-priority-thread' refs #20183
Fix keep-web, refs #19899
19975: Add OutOfMemoryRetry to CWL extensions page.
19975: Typo fix.
19975: Add note about runtime.ram
19975: s/request/specify
Merge branch '19975-oom-resubmit' refs #19975
Merge branch '20214-r-sdk' refs #20214
20223: Try to make priority update much lighter weight
20223: Use pluck to avoid loading full record just to get log
Merge branch '20223-container-bloat' refs #20223
20223: Make trash sweep use in_batches
20223: Use in_batches in used_by to tamp down memory usage
20223: Need to use each_record instead of each
20223: Even more efficient to use pluck instead of batches
20227: Change default BalacePeriod to 6 hours.
Merge branch '20223-trash-sweep' refs #20223
20259: Add documentation for banner and tooltip features
Merge branch '20259-banner-tooltip-doc' refs #20259
16379: doc updates
16379: Add prometheus and grafana to workbench aliases
16379: Updates for grafana install
16379: Small doc updates about prometheus and grafana
16379: Reorganize local.params examples
16379: Tweak doc on starring the dashboards on grafana
Merge branch '20295-fix-collection-tree-caching-bug' refs #20295
Add 2.6.0 release to upgrade notes
Stephen Smith (2):
19899: Add Cache-Control header to keep-web CORS response
Merge branch '19899-webdav-cache-control' into main. Refs #19899
Tom Clegg (158):
18693: De-duplicate permission links on create/update/delete.
18693: Return existing instead of creating redundant can_login link.
18693: Delete redundant login permission links on update/delete.
18693: Remove existing redundant permission links from db.
18693: Add row locking.
19597: Parse multipart/form-data request body.
19890: Use 64 bit ints for id columns.
19896: Configurable minimum TLS version for LDAP connection.
19894: Update documentation re: dispatcher connecting to postgresql.
Merge branch '19894-pg-access'
19597: Comment explicit ParseForm usage.
Merge branch '19597-multipart-request'
19890: Note slow upgrade due to schema migration.
19320: Use AWS spot price data to calculate container cost.
Merge branch '19890-id-bigint'
19896: Use StartTLS + MinTLSVersion regardless of Insecure flag.
19896: Mention TLS version error message in config doc.
18693: Use row locking only if enabled in config.
19320: Fix env var leak in test.
Merge branch '19896-ldap-tls-downgrade'
19320: Fix AWS InstanceStatus and SpotPrice API usage.
Merge branch '18693-dedup-permissions'
19320: Log instance price changes in crunch-run.txt.
19923: keep-balance option to process a subset of blocks.
19320: Account for AddedScratch in spot instance cost estimates.
19320: Disable spot price checks if configured update interval <= 0.
19320: Comment re future use of spot attr in priceKey.
19320: Deduplicate instance types in spot price request.
19320: Fix live test.
19923: Log progress while reading keepstore indexes.
19923: Log keep-balance memory usage.
Merge branch '19320-spot-pricing'
19923: Remove debug sleep.
19923: Sanity check provided chunk prefix.
19993: Fix postgresql<12 incompatibility.
19923: Report heap and stack stats from Go runtime.
18075: Add CloudVMs.MaxInstances config, retire MaxComputeVMs.
Merge branch '19923-keep-balance-chunk'
Merge branch '19993-groupby-forupdate'
18075: Kill off MaxComputeVMs properly.
18075: Remove tests for crunch1 Node.
19624: Container request state transition / lifecycle diagram.
19624: Redo diagram to show container states.
19624: Redo diagram to show CR and container states.
19954: Update API docs to reflect permission link deduplication.
19624: Layout graph L->R, say "update" instead of "assign".
18075: Update AtQuota comment, check quota in Create, log at limit.
18075: Note obsolete/deprecated config.
19954: Move special behavior comments into the method sections.
Merge branch '18075-max-compute-vms'
20032: Fix unnecessary race in test.
Merge branch '19624-priority-doc'
19954: Update wording to admit multiple overlapping permission links.
Merge branch '19954-permission-dedup-doc'
19907: Log when caching negative result for OIDC token check.
19907: Don't cache network/5xx errors when checking UserInfo.
19907: Update go-oidc to v3.
Merge branch '20032-tunnel-test'
Merge branch '19907-oidc-cache-error'
19973: Limit arvados.Client request concurrency.
19973: Limit container concurrency when API is returning 503.
20083: Test false positive in "save local changes" logic.
20083: Track both loadedPDH and savedPDH for checking local changes.
20083: Don't copy remote blocks when computing local PDH.
19973: Add metrics for automatic container concurrency limit.
19973: Update tests.
19961: Detect and log EC2 spot interruption notices.
19973: Test throttling after receiving 503 error.
19973: Fix tests.
20121: Update prometheus client.
Merge branch '20121-update-prom-client'
Merge branch '20083-sync-readonly'
19961: Fix tests sabotaging other tests.
19961: Fix races in tests.
19973: un-embed requestLimiter.
19973: Merge branch 'main'
19973: Split unit test cases into multiple funcs.
19961: Merge branch 'main'
19961: Reuse metadata token within expected TTL, adjust retry logic.
19961: Mention interrupt handling on admin doc page.
Merge branch '19973-dispatch-throttle' into main
19963: Fix crash on no-cache + invalid token.
19963: Dedup interface type.
19963: Enable login flow without anonymous token configured.
19963: Test workbench2 login flow starting with no token.
19961: Save separate preemptionNotice key in runtime_status.
Merge branch '19963-keep-web-auth-flow' into main
19961: Clarify preemptionNotice significance.
19961: Update preemptionNotice text.
Merge branch '19961-spot-interruption'
20198: Fix unclosed response body.
20198: Run config-watcher thread in a worker process, not preloader.
Merge branch '20198-passenger-restart'
20183: Move priority update thread from rails to controller.
20183: Deduplicate test suite setup.
20183: Fixup auth contexts in tests.
20183: Move update_priority tests. Fix updater starvation.
20183: Fix test fixture.
20183: Don't do a periodic priority-update check at startup.
20188: Recognize InsufficientFreeAddressesInSubnet as quota error.
19972: Auto retry with exponential backoff.
19972: Disable auto-retry in ws and dispatcher.
19972: Disable auto-retry in diagnostics and rpc.
20183: Explain priority update strategy in comments.
20183: Clean up sql queries.
20183: Update container priority in creation order.
20188: Handle InsufficientVolumeCapacity as quota error.
Merge branch '20188-insufficient-subnet'
19972: Remove unused code.
19972: Lint.
19972: Fix racy test case.
Merge branch '19972-go-client-retry'
20187: Validate and cache discovery doc in controller.
20187: Update test.
20187: 1 minute timeout for getting discovery doc from rails.
20187: Test discovery doc cache.
20187: Preserve CORS and other misc headers.
Merge branch '20183-update-priority-thread'
20187: Check that the discovery doc is really a discovery doc.
20187: Fix test race.
20187: Tidy up test case.
20187: Use validate func instead of making cache discoverydoc-aware.
20187: Expire cache and start returning errors after 24h.
20187: Don't use rails cache for discovery doc.
19889: Serve live logs via webdav.
19889: Stop using old x/net/context library.
19889: Give sensitive test a prand sequence that passes.
19889: Remove unneeded workaround for not saving empty dirs.
19889: Fix container update error when CR is already final somehow.
19889: Update docs.
19889: Add comments.
19889: Add routing test.
19889: Fix webdav methods condition.
20187: Remove references to discovery doc cache.
20242: Trash only one when identical replicas are eligible to trash.
Merge branch '20187-cache-discovery-doc'
20235: Install crunch-run before run-probe even if boot-probe fails.
20264: Ignore superfluous :443 and :80 in trusted origin check.
20140: Accept wildcards in TrustedClients.
Merge branch '20242-dup-trash-lists'
19889: Fix comment.
19889: Remove unused code.
19889: Add www-authenticate header with 401 Unauthorized response.
19889: Preserve WebDAV path when proxying to keep-web.
20264: Reject redirect target with userinfo.
Merge branch '20235-probe-after-upgrade'
Merge branch '20264-trusted-clients-portnumber'
18790: Fix .../containers/.../log routing.
18790: Fix logging noise from ENOENT.
19889: Fix crash on gateway connect error.
20240: Update test to reveal bug.
20240: Fix test.
20240: Expose deadlock in multi-level workflow test case.
20240: Lock containers table before cascading priority updates.
20240: Don't accept/retry after deadlock in test case.
20240: Change locking mode to EXCLUSIVE.
Merge branch '20240-container-update-deadlock'
Merge branch '19889-live-log-webdav'
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list