[ARVADOS] updated: 2.1.0-10-g817d7985e
Git user
git at public.arvados.org
Fri Oct 16 20:32:12 UTC 2020
Summary of changes:
doc/_config.yml | 1 +
doc/api/keep-s3.html.textile.liquid | 14 ++----
doc/api/keep-webdav.html.textile.liquid | 78 +++------------------------------
3 files changed, 11 insertions(+), 82 deletions(-)
via 817d7985e27cf188195fed1c7c3e412fb53beb5b (commit)
from 6c8ca0862ed8a66dd42a85ec498fd6e32fad310f (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
commit 817d7985e27cf188195fed1c7c3e412fb53beb5b
Author: Peter Amstutz <peter.amstutz at curii.com>
Date: Fri Oct 16 16:31:50 2020 -0400
16558: Tweak S3 docs and move keep-web URL patterns to its own page
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz at curii.com>
diff --git a/doc/_config.yml b/doc/_config.yml
index 7117f5f4a..ef9d1d771 100644
--- a/doc/_config.yml
+++ b/doc/_config.yml
@@ -126,6 +126,7 @@ navbar:
- Data management:
- api/keep-webdav.html.textile.liquid
- api/keep-s3.html.textile.liquid
+ - api/keep-web-urls.html.textile.liquid
- api/methods/collections.html.textile.liquid
- api/methods/repositories.html.textile.liquid
- Container engine:
diff --git a/doc/api/keep-s3.html.textile.liquid b/doc/api/keep-s3.html.textile.liquid
index b56b232f3..cb23a5889 100644
--- a/doc/api/keep-s3.html.textile.liquid
+++ b/doc/api/keep-s3.html.textile.liquid
@@ -64,17 +64,11 @@ Can be used to determine if an object exists and if client has read access to it
h4. GetBucketVersioning
-This is a stub to avoid breaking clients that request information about versioning. Versioning is not actually supported.
+Bucket versioning is presently not supported, so this will always respond that bucket versioning is not enabled.
h3. Authorization mechanisms
-Accepts AWS Signature Version 4 format @Authorization: AWS AWS4-HMAC-SHA256 Credential=...,SignedHeaders=...,Signature=...@
+Keep-web accepts AWS Signature Version 4 (AWS4-HMAC-SHA256) as well as the older V2 AWS signature.
-Can be used one of two ways:
-
-* The "Credential" is the Arvados token uuid, and the "Signature" is derived from request headers and the Arvados token secret
-* The "Credential" is the Arvados token secret, and "Signature" is derived from the same Arvados token secret
-
-Consult the S3 documentation for information on how to produce AWS4-HMAC-SHA256 signatures. If you are using a S3 client library, it should be handled for you already.
-
-Keep-web also accepts older format header @Authorization: AWS AccessKey:signature@ . Provide the Arvados secret token for the "Access Key". In this case, the signature can be anything as keep-web will ignore it.
+* If your client uses V4 signatures exclusively: use the Arvados token's UUID part as AccessKey, and its secret part as SecretKey. This is preferred.
+* If your client uses V2 signatures, or a combination of V2 and V4, or the Arvados token UUID is unknown: use the secret part of the Arvados token for both AccessKey and SecretKey.
diff --git a/doc/api/keep-webdav.html.textile.liquid b/doc/api/keep-webdav.html.textile.liquid
index 1e9620cf0..f068a49c2 100644
--- a/doc/api/keep-webdav.html.textile.liquid
+++ b/doc/api/keep-webdav.html.textile.liquid
@@ -3,8 +3,8 @@ layout: default
navsection: api
navmenu: API Methods
title: "WebDAV"
-
...
+
{% comment %}
Copyright (C) The Arvados Authors. All rights reserved.
@@ -15,7 +15,7 @@ SPDX-License-Identifier: CC-BY-SA-3.0
Most major operating systems include built-in support for mounting WebDAV resources as network file systems, see user guide sections for "Windows":{{site.baseurl}}/user/tutorials/tutorial-keep-mount-windows.html , "macOS":{{site.baseurl}}/user/tutorials/tutorial-keep-mount-os-x.html , "Linux (Gnome)":{{site.baseurl}}/user/tutorials/tutorial-keep-mount-gnu-linux.html#gnome . WebDAV is also supported by various standalone storage browser applications such as "Cyberduck":https://cyberduck.io/ and client libraries exist in many languages for programmatic access.
-Keep-web provides read/write HTTP (WebDAV) access to files stored in Keep. It serves public data to anonymous and unauthenticated clients, and serves private data to clients that supply Arvados API tokens. It can be installed anywhere with access to Keep services, typically behind a web proxy that supports TLS.
+Keep-web provides read/write HTTP (WebDAV) access to files stored in Keep. It serves public data to anonymous and unauthenticated clients, and serves private data to clients that supply Arvados API tokens.
h3. Supported Operations
@@ -25,7 +25,7 @@ Does not support @LOCK@ or @UNLOCK at . These methods will be accepted, but are no
h3. Browsing
-Fetching the root path an Arvados WebDAV service will return a 401 Unauthorized response with a @WWW-Authenticate@ header indicating "support for RFC 7617 Basic Authentication":https://tools.ietf.org/html/rfc7617 . Requests may provide an Arvados token using @Authorization: Basic@ as described in "Authorization mechanisms":#auth .
+Requests can be authenticated a variety of ways as described below in "Authentication mechanisms":#auth . An unauthenticated request will return a 401 Unauthorized response with a @WWW-Authenticate@ header indicating "support for RFC 7617 Basic Authentication":https://tools.ietf.org/html/rfc7617 .
Getting a listing from keep-web starting at the root path @/@ will return two folders, @by_id@ and @users at .
@@ -33,75 +33,9 @@ The @by_id@ folder will return an empty listing. However, a path which starts w
The @users@ folder will return a listing of the users for whom the client has permission to read the "home" project of that user. Browsing an individual user will return the collections and projects directly owned by that user. Browsing those collections and projects return listings of the files, directories, collections, and subprojects they contain, and so forth.
-h3. URL structure
-
-Files served by @keep-web@ can be rendered directly in the browser, or @keep-web@ can instruct the browser to only download the file.
-
-When serving files that will render directly in the browser, it is important to properly configure the WebDAV service to migitate cross-site-scripting (XSS) attacks. A HTML page can be stored in a collection. If an attacker causes a victim to visit that page through Workbench, the HTML will be rendered by the browser. If all collections are served at the same domain, the browser will consider collections as coming from the same origin, which will grant access to the same browsing data (cookies and local storage). This would enable malicious Javascript on that page to access Arvados on behalf of the victim.
-
-This can be mitigated by having separate domains for each collection, or limiting preview to circumstances where the collection is not accessed with the user's regular full-access token. For cluster administrators that understand the risks, this protection can also be turned off.
-
-The following "same origin" URL patterns are supported for public collections and collections shared anonymously via secret links (i.e., collections which can be served by keep-web without making use of any implicit credentials like cookies). See "Same-origin URLs" below.
-
-<pre>
-http://collections.example.com/c=uuid_or_pdh/path/file.txt
-http://collections.example.com/c=uuid_or_pdh/t=TOKEN/path/file.txt
-</pre>
-
-The following "multiple origin" URL patterns are supported for all collections:
-
-<pre>
-http://uuid_or_pdh--collections.example.com/path/file.txt
-http://uuid_or_pdh--collections.example.com/t=TOKEN/path/file.txt
-</pre>
-
-In the "multiple origin" form, the string "--" can be replaced with "." with identical results (assuming the downstream proxy is configured accordingly). These two are equivalent:
-
-<pre>
-http://uuid_or_pdh--collections.example.com/path/file.txt
-http://uuid_or_pdh.collections.example.com/path/file.txt
-</pre>
-
-The first form (with "--" instead of ".") avoids the cost and effort of deploying a wildcard TLS certificate for *.collections.example.com at sites that already have a wildcard certificate for *.example.com. The second form is likely to be easier to configure, and more efficient to run, on a downstream proxy.
-
-In all of the above forms, the "collections.example.com" part can be anything at all: keep-web itself ignores everything after the first "." or "--". (Of course, in order for clients to connect at all, DNS and any relevant proxies must be configured accordingly.)
-
-In all of the above forms, the "uuid_or_pdh" part can be either a collection UUID or a portable data hash with the "+" character optionally replaced by "-". (When "uuid_or_pdh" appears in the domain name, replacing "+" with "-" is mandatory, because "+" is not a valid character in a domain name.)
-
-In all of the above forms, a top level directory called "_" is skipped. In cases where the "path/file.txt" part might start with "t=" or "c=" or "_/", links should be constructed with a leading "_/" to ensure the top level directory is not interpreted as a token or collection ID.
-
-Assuming there is a collection with UUID zzzzz-4zz18-znfnqtbbv4spc3w and portable data hash 1f4b0bc7583c2a7f9102c395f4ffc5e3+45, the following URLs are interchangeable:
-
-<pre>
-http://zzzzz-4zz18-znfnqtbbv4spc3w.collections.example.com/foo/bar.txt
-http://zzzzz-4zz18-znfnqtbbv4spc3w.collections.example.com/_/foo/bar.txt
-http://zzzzz-4zz18-znfnqtbbv4spc3w--collections.example.com/_/foo/bar.txt
-</pre>
-
-The following URLs are read-only, but otherwise interchangeable with the above:
-
-<pre>
-http://1f4b0bc7583c2a7f9102c395f4ffc5e3-45--foo.example.com/foo/bar.txt
-http://1f4b0bc7583c2a7f9102c395f4ffc5e3-45--.invalid/foo/bar.txt
-http://collections.example.com/by_id/1f4b0bc7583c2a7f9102c395f4ffc5e3%2B45/foo/bar.txt
-http://collections.example.com/by_id/zzzzz-4zz18-znfnqtbbv4spc3w/foo/bar.txt
-</pre>
-
-If the collection is named "MyCollection" and located in a project called "MyProject" which is in the home project of a user with username is "bob", the following read-only URL is also available when authenticating as bob:
-
-pre. http://collections.example.com/users/bob/MyProject/MyCollection/foo/bar.txt
-
-An additional form is supported specifically to make it more convenient to maintain support for existing Workbench download links:
-
-pre. http://collections.example.com/collections/download/uuid_or_pdh/TOKEN/foo/bar.txt
-
-A regular Workbench "download" link is also accepted, but credentials passed via cookie, header, etc. are ignored. Only public data can be served this way:
-
-pre. http://collections.example.com/collections/uuid_or_pdh/foo/bar.txt
-
-Collections can also be accessed (read-only) via "/by_id/X" where X is a UUID or portable data hash.
+In addition to the @/by_id/@ path prefix, the collection or project can be specified using a path prefix of @/c=<uuid or pdh>/@ or (if the cluster is properly configured) as a virtual host. This is described on "Keep-web URLs":keep-web-urls.html
-h3(#auth). Authorization mechanisms
+h3(#auth). Authentication mechanisms
A token can be provided in an Authorization header as a @Bearer@ token:
@@ -124,7 +58,7 @@ Cookie: api_token=bzA3ajRweDdSbEpLNEN1TVlwN0MwTERUNEN6UjFKMXFCRTVBdm83ZUNjVWpPVG
A token can be provided in an URL-encoded query string:
<pre>
-GET /foo/bar.txt?api_token=o07j4px7RlJK4CuMYp7C0LDT4CzR1J1qBE5Avo7eCcUjOTikxK<
+GET /foo/bar.txt?api_token=o07j4px7RlJK4CuMYp7C0LDT4CzR1J1qBE5Avo7eCcUjOTikxK
</pre>
A token can be provided in a URL-encoded path (as described in the previous section):
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list