[ARVADOS] updated: 2.1.0-1009-gc89213f5a
Git user
git at public.arvados.org
Wed Jun 30 19:49:19 UTC 2021
Summary of changes:
...restricting-upload-download.html.textile.liquid | 148 ++++++++++++++++
lib/config/config.default.yml | 28 +++
lib/config/export.go | 3 +
lib/config/generated_config.go | 28 +++
sdk/go/arvados/config.go | 14 ++
sdk/go/arvados/fs_collection.go | 1 +
sdk/go/arvadostest/fixtures.go | 3 +
services/keep-web/cache.go | 69 ++++++--
services/keep-web/handler.go | 142 ++++++++++++++-
services/keep-web/handler_test.go | 190 ++++++++++++++++++++-
services/keep-web/s3.go | 28 ++-
services/keep-web/server_test.go | 4 +-
services/keepproxy/keepproxy.go | 158 ++++++++++++-----
services/keepproxy/keepproxy_test.go | 165 +++++++++++++++---
tools/user-activity/arvados_user_activity/main.py | 20 +++
15 files changed, 910 insertions(+), 91 deletions(-)
create mode 100644 doc/admin/restricting-upload-download.html.textile.liquid
via c89213f5a5e303050caaebe4f8fdf2980fc65605 (commit)
via acb37250ac86042f8715206c3f91eec29aea2deb (commit)
via 47e278f4286ec2c53c1faefd740b5119cb757b0a (commit)
via 2ffe1a900d4cfcceac02b44768a13a47edf18184 (commit)
via 60457ce8efe39a0929fd5e103ab8c042d322a1cb (commit)
via 891783a0a90b37336455c81748dd3fe002e95a9c (commit)
via 5726e94f389f4ee696a551e5222ddea93c029f76 (commit)
via 85d86b801817958117db0a39f35d70f1a1f36ea7 (commit)
via fe92cee1dd48aca3ee1f2e9465eba4abb2c5b433 (commit)
via 10afc7ae2f95aad09937465702be7c44d07920d4 (commit)
via 903e0d1c2e209f4d62e0d83ce3a062b823a71da3 (commit)
via 572e375e1c950f35d7e159bac031de5205354590 (commit)
via 046d4591c694537dc1a415744413986785fc6b9c (commit)
via 154ba68b8f66e50d9ddb53f3a5f19890588f42f4 (commit)
via a271aaf8e345f8035a8818cee15868bd1c38ceb0 (commit)
via e093e9ea2e7c204c5c58de006d8b316c9013b6dd (commit)
via 7ab068369174b2a4de6251202a021f20ca1f36f3 (commit)
via 59107add9b9d01d98cca49933c90f4e320d47fef (commit)
via 23f6c01aaf4a3f501e908a689f128e77fc9f23aa (commit)
via 3a8485be4eeb09f082338bc3dabe11a3702e538c (commit)
via ce4ece370e62774f1e3ef797c714529603943f41 (commit)
via 834ba51d22fa93297e66c60c3eb51cc1cf05fccc (commit)
via 69b64f2fe8ccd2b92d1aa55b9fb39b03a342468e (commit)
via de8448b3b546eb1eee8a45261954028e3ea22252 (commit)
from 40e3e6461dd3276a4f3ebcec7259ac1b5748220c (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
commit c89213f5a5e303050caaebe4f8fdf2980fc65605
Merge: 40e3e6461 acb37250a
Author: Peter Amstutz <peter.amstutz at curii.com>
Date: Wed Jun 30 15:49:01 2021 -0400
Merge branch '17464-download-activity' refs #17464
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz at curii.com>
diff --cc lib/config/config.default.yml
index 645da5671,ff2f649fa..e28d5cbb7
--- a/lib/config/config.default.yml
+++ b/lib/config/config.default.yml
@@@ -551,8 -554,36 +551,36 @@@ Clusters
# Persistent sessions.
MaxSessions: 100
+ # Selectively set permissions for regular users and admins to
+ # download or upload data files using the upload/download
+ # features for Workbench, WebDAV and S3 API support.
+ WebDAVPermission:
+ User:
+ Download: true
+ Upload: true
+ Admin:
+ Download: true
+ Upload: true
+
+ # Selectively set permissions for regular users and admins to be
+ # able to download or upload blocks using arv-put and
+ # arv-get from outside the cluster.
+ KeepproxyPermission:
+ User:
+ Download: true
+ Upload: true
+ Admin:
+ Download: true
+ Upload: true
+
+ # Post upload / download events to the API server logs table, so
+ # that they can be included in the arv-user-activity report.
+ # You can disable this if you find that it is creating excess
+ # load on the API server and you don't need it.
+ WebDAVLogEvents: true
+
Login:
- # One of the following mechanisms (SSO, Google, PAM, LDAP, or
+ # One of the following mechanisms (Google, PAM, LDAP, or
# LoginCluster) should be enabled; see
# https://doc.arvados.org/install/setup-login.html
diff --cc lib/config/generated_config.go
index 1bdc269c0,34cc07115..b15bf7eeb
--- a/lib/config/generated_config.go
+++ b/lib/config/generated_config.go
@@@ -557,8 -560,36 +557,36 @@@ Clusters
# Persistent sessions.
MaxSessions: 100
+ # Selectively set permissions for regular users and admins to
+ # download or upload data files using the upload/download
+ # features for Workbench, WebDAV and S3 API support.
+ WebDAVPermission:
+ User:
+ Download: true
+ Upload: true
+ Admin:
+ Download: true
+ Upload: true
+
+ # Selectively set permissions for regular users and admins to be
+ # able to download or upload blocks using arv-put and
+ # arv-get from outside the cluster.
+ KeepproxyPermission:
+ User:
+ Download: true
+ Upload: true
+ Admin:
+ Download: true
+ Upload: true
+
+ # Post upload / download events to the API server logs table, so
+ # that they can be included in the arv-user-activity report.
+ # You can disable this if you find that it is creating excess
+ # load on the API server and you don't need it.
+ WebDAVLogEvents: true
+
Login:
- # One of the following mechanisms (SSO, Google, PAM, LDAP, or
+ # One of the following mechanisms (Google, PAM, LDAP, or
# LoginCluster) should be enabled; see
# https://doc.arvados.org/install/setup-login.html
diff --cc services/keepproxy/keepproxy_test.go
index c569a05e7,9dc07ddaa..2d4266d8d
--- a/services/keepproxy/keepproxy_test.go
+++ b/services/keepproxy/keepproxy_test.go
@@@ -228,30 -236,8 +236,30 @@@ func (s *ServerRequiredSuite) TestStora
c.Check(hdr.Get("X-Keep-Storage-Classes"), Equals, "secure")
}
+func (s *ServerRequiredSuite) TestStorageClassesConfirmedHeader(c *C) {
+ runProxy(c, false, false)
+ defer closeListener()
+
+ content := []byte("foo")
+ hash := fmt.Sprintf("%x", md5.Sum(content))
+ client := &http.Client{}
+
+ req, err := http.NewRequest("PUT",
+ fmt.Sprintf("http://%s/%s", listener.Addr().String(), hash),
+ bytes.NewReader(content))
+ c.Assert(err, IsNil)
+ req.Header.Set("X-Keep-Storage-Classes", "default")
+ req.Header.Set("Authorization", "OAuth2 "+arvadostest.ActiveToken)
+ req.Header.Set("Content-Type", "application/octet-stream")
+
+ resp, err := client.Do(req)
+ c.Assert(err, IsNil)
+ c.Assert(resp.StatusCode, Equals, http.StatusOK)
+ c.Assert(resp.Header.Get("X-Keep-Storage-Classes-Confirmed"), Equals, "default=2")
+}
+
func (s *ServerRequiredSuite) TestDesiredReplicas(c *C) {
- kc := runProxy(c, false, false)
+ kc, _ := runProxy(c, false, false, nil)
defer closeListener()
content := []byte("TestDesiredReplicas")
commit acb37250ac86042f8715206c3f91eec29aea2deb
Author: Peter Amstutz <peter.amstutz at curii.com>
Date: Wed Jun 30 15:47:16 2021 -0400
17464: Couple of doc fixes
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz at curii.com>
diff --git a/doc/admin/restricting-upload-download.html.textile.liquid b/doc/admin/restricting-upload-download.html.textile.liquid
index 8d5c9f544..ea1075234 100644
--- a/doc/admin/restricting-upload-download.html.textile.liquid
+++ b/doc/admin/restricting-upload-download.html.textile.liquid
@@ -59,6 +59,8 @@ The default policy allows anyone to upload or download.
WebDAVLogEvents: true
</pre>
+If you create a sharing link as an admin user, and then give someone the token from the sharing link to download a file over HTTP (WebDAV or S3 API), because the downloader is anonymous, the download permission will be restricted based on the "User" role and not the "Admin" role.
+
h2. Shell node and container permissions
Be aware that even when upload and download from outside the network is not allowed, a user who has access to a shell node or runs a container still has internal access to Keep. (This is necessary to be able to run workflows). From the shell node or container, a user could send data outside the network by some other method, although this requires more intent than accidentally clicking on a link and downloading a file. It is possible to set up a firewall to prevent shell and compute nodes from making connections to hosts outside the private network. Exactly how to configure firewalls is out of scope for this page, as it depends on the specific network infrastructure of your cluster.
@@ -99,7 +101,7 @@ For ease of access auditing, this policy prevents downloads using @arv-get at . Do
h3. Disallow downloads by regular users
-This policy prevents regular users (non-admin) from downloading data. Uploading is allowed. This supports the case where restricted data sets are shared users so that they may run their own data analysis on the cluster, while preventing them from downloading the data set to their local workstation. Note that users won't be able to download the results of their analysis, either, requiring an admin in the loop or some other process to release results.
+This policy prevents regular users (non-admin) from downloading data. Uploading is allowed. This supports the case where restricted data sets are shared with users so that they may run their own data analysis on the cluster, while preventing them from downloading the data set to their local workstation. Be aware that users won't be able to download the results of their analysis, either, requiring an admin in the loop or some other process to release results.
<pre>
Collections:
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list