[ARVADOS] updated: 1.3.0-1630-g2602abaee
Git user
git at public.curoverse.com
Fri Sep 27 20:22:50 UTC 2019
Summary of changes:
doc/_config.yml | 10 ++---
doc/admin/federation.html.textile.liquid | 46 ++++++++++++----------
doc/admin/merge-remote-account.html.textile.liquid | 39 ++++++++++++------
doc/admin/upgrade-crunch2.html.textile.liquid | 2 +-
doc/architecture/federation.html.textile.liquid | 9 -----
5 files changed, 59 insertions(+), 47 deletions(-)
via 2602abaee640b1c50e9780d951b15006e43e203a (commit)
from de3276d1b2067be4aa40b51605c2ce509ea80915 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
commit 2602abaee640b1c50e9780d951b15006e43e203a
Author: Peter Amstutz <pamstutz at veritasgenetics.com>
Date: Fri Sep 27 16:20:27 2019 -0400
15531: Update arv-federation-migrate documentation for LoginCluster
Also take out some mentions of the obsolete application.yml and
reorganize a couple other pages that describe obsolete features.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <pamstutz at veritasgenetics.com>
diff --git a/doc/_config.yml b/doc/_config.yml
index 0547c8ee9..314b5b5cd 100644
--- a/doc/_config.yml
+++ b/doc/_config.yml
@@ -150,11 +150,11 @@ navbar:
- admin/index.html.textile.liquid
- Configuration:
- admin/config.html.textile.liquid
+ - admin/federation.html.textile.liquid
+ - admin/collection-managed-properties.html.textile.liquid
- Upgrading and migrations:
- admin/upgrading.html.textile.liquid
- admin/config-migration.html.textile.liquid
- - install/migrate-docker19.html.textile.liquid
- - admin/upgrade-crunch2.html.textile.liquid
- Users and Groups:
- install/cheat_sheet.html.textile.liquid
- admin/activation.html.textile.liquid
@@ -171,12 +171,12 @@ navbar:
- admin/cloudtest.html.textile.liquid
- Data Management:
- admin/collection-versioning.html.textile.liquid
- - admin/collection-managed-properties.html.textile.liquid
- - Other:
- - admin/federation.html.textile.liquid
- admin/controlling-container-reuse.html.textile.liquid
- admin/logs-table-management.html.textile.liquid
+ - Other:
- admin/troubleshooting.html.textile.liquid
+ - install/migrate-docker19.html.textile.liquid
+ - admin/upgrade-crunch2.html.textile.liquid
installguide:
- Overview:
- install/index.html.textile.liquid
diff --git a/doc/admin/federation.html.textile.liquid b/doc/admin/federation.html.textile.liquid
index 3728507f9..85258955b 100644
--- a/doc/admin/federation.html.textile.liquid
+++ b/doc/admin/federation.html.textile.liquid
@@ -14,28 +14,11 @@ This page describes how to enable and configure federation capabilities between
An overview on how this feature works is discussed in the "architecture section":{{site.baseurl}}/architecture/federation.html
-h3. API Server configuration
+h2. Configuration
-To accept users from remote clusters, some settings need to be added to the @application.yml@ file. There are two ways in which a remote cluster can be identified: either explictly by listing its prefix-to-hostname mapping, or implicitly by assuming the given remote cluster is public and belongs to the @.arvadosapi.com@ subdomain.
+To enable a cluster to communicate with other clusters, some settings need to be added to the @config.yml@ file. Federated clusters are identified by listing the cluster-to-hostname mapping in the @RemoteClusters@ section.
-For example, if you want to set up a private cluster federation, the following configuration will only allow access to users from @clsr2@ & @clsr3@:
-
-<pre>
-production:
- remote_hosts:
- clsr2: api.cluster2.com
- clsr3: api.cluster3.com
- remote_hosts_via_dns: false
- auto_activate_users_from: []
-</pre>
-
-The additional @auto_activate_users_from@ setting can be used to allow users from the clusters in the federation to not only read but also create & update objects on the local cluster. This feature is covered in more detail in the "user activation section":{{site.baseurl}}/admin/activation.html. In the current example, only manually activated remote users would have full access to the local cluster.
-
-h3. Arvados controller & keepstores configuration
-
-Both @arvados-controller@ and @keepstore@ services also need to be configured, as they proxy requests to remote clusters when needed.
-
-Continuing the previous example, the necessary settings should be added to the @/etc/arvados/config.yml@ file as follows:
+Here is an example of the settings that should be added to the @/etc/arvados/config.yml@ file:
<pre>
Clusters:
@@ -44,14 +27,35 @@ Clusters:
clsr2:
Host: api.cluster2.com
Proxy: true
+ ActivateUsers: true
clsr3:
Host: api.cluster3.com
Proxy: true
+ ActivateUsers: false
</pre>
Similar settings should be added to @clsr2@ & @clsr3@ hosts, so that all clusters in the federation can talk to each other.
-h3. Testing
+The @ActivateUsers@ setting indicates whether users from a given cluster are automatically activated or they require manual activation. User activation is covered in more detail in the "user activation section":{{site.baseurl}}/admin/activation.html. In the current example, users from @clsr2@ would be automatically, activated, but users from @clsr3@ would require an admin to activate the account.
+
+h2(#LoginCluster). Federation user management
+
+A federation of clusters can be configured to use a separate user database per cluster, or delegate a central cluster to manage the database.
+
+If clusters belong to separate organizations, each cluster will have its own user database for the members of that organization. Through federation, a user from one organization can be granted access to the cluster of another organization. The admin of the seond cluster controls access on a individual basis by choosing to activate or deactivate accounts from other organizations (with the default policy the value of @ActivateUsers@).
+
+On the other hand, if all clusters belong to the same organization, and users in that organization should have access to all the clusters, user management can be simplified by setting the @LoginCluster@ which manages the user database used by all other clusters in the federation. The @LoginCluster@ configuration should be set for all clusters in the federation to the id of one cluster in the federation which will be the 'master cluster'. This directs all logins to the LoginCluster, and the LoginCluster will issue API tokens which are valid on any cluster in the federation. Users are activated or deactivated across the entire federation based on their status on the master cluster.
+
+<pre>
+Clusters:
+ clsr2:
+ Login:
+ LoginCluster: clsr1
+</pre>
+
+To migrate users of an existing federation with separate user databases to use a single LoginCluster, use "arv-federation-migrate":merge-remote-account.html .
+
+h2. Testing
Following the above example, let's suppose @clsr1@ is our "home cluster", that is to say, we use our @clsr1@ user account as our federated identity and both @clsr2@ and @clsr3@ remote clusters are set up to allow users from @clsr1@ and to auto-activate them. The first thing to do would be to log into a remote workbench using the local user token. This can be done following these steps:
diff --git a/doc/admin/merge-remote-account.html.textile.liquid b/doc/admin/merge-remote-account.html.textile.liquid
index c6188095e..8ef4e632d 100644
--- a/doc/admin/merge-remote-account.html.textile.liquid
+++ b/doc/admin/merge-remote-account.html.textile.liquid
@@ -25,6 +25,21 @@ As part of migrating a user, any data or permissions associated with old user ac
h2. Get user report
+h3. With a LoginCluster
+
+Set the @ARVADOS_API_HOST@ and @ARVADOS_API_TOKEN@ environment variables to be an admin user on the @LoginCluster at . It will automatically determine the other clusters that are listed in the federation.
+
+Next, run @arv-federation-migrate@ with the @--report@ flag:
+
+<pre>
+$ arv-federation-migrate --report users.csv
+Getting user list from x6b1s
+Getting user list from x3982
+Wrote users.csv
+</pre>
+
+h3. Without a LoginCluster
+
The first step is to create @tokens.csv@ and list each cluster and API token to access the cluster. API tokens must be trusted tokens with administrator access. This is a simple comma separated value file and can be created in a text editor. Example:
_tokens.csv_
@@ -44,36 +59,38 @@ Getting user list from x3982
Wrote users.csv
</pre>
+h2. Update the user report
+
This will produce a report of users across all clusters listed in @tokens.csv@, sorted by email address. This file can be loaded into a text editor or spreadsheet program for ease of viewing and editing.
_users.csv_
<pre>
-email,user uuid,primary cluster/user
-person_a at example.com,x6b1s-tpzed-hb5n7doogwhk6cf,x6b1s
-person_b at example.com,x3982-tpzed-1vl3k7knf7qihbe,
-person_b at example.com,x6b1s-tpzed-w4nhkx2rmrhlr54,
+email,username,user uuid,primary cluster/user
+person_a at example.com,person_a,x6b1s-tpzed-hb5n7doogwhk6cf,x6b1s
+person_b at example.com,person_b,x3982-tpzed-1vl3k7knf7qihbe,
+person_b at example.com,person_b,x6b1s-tpzed-w4nhkx2rmrhlr54,
</pre>
-The third column describes that user's home cluster. If a user only has one account (identified by email address), the column will be filled in and there is nothing to do. If the column is blank, that means there is more than one Arvados account associated with the user. Edit the file and provide the desired home cluster for each user. In this example, <code>person_b at example.com</code> is assigned the home cluster @x3982 at .
+The fourth column describes that user's home cluster. If a user only has one account (identified by email address), the column will be filled in and there is nothing to do. If the column is blank, that means there is more than one Arvados account associated with the user. Edit the file and provide the desired home cluster for each user as necessary (note: if there is a LoginCluster, all users will be migrated to the LoginCluster). It is also possible to change the desired username for a user. In this example, <code>person_b at example.com</code> is assigned the home cluster @x3982 at .
_users.csv_
<pre>
-email,user uuid,primary cluster/user
-person_a at example.com,x6b1s-tpzed-hb5n7doogwhk6cf,x6b1s
-person_b at example.com,x3982-tpzed-1vl3k7knf7qihbe,x3982
-person_b at example.com,x6b1s-tpzed-w4nhkx2rmrhlr54,x3982
+email,username,user uuid,primary cluster/user
+person_a at example.com,person_a,x6b1s-tpzed-hb5n7doogwhk6cf,x6b1s
+person_b at example.com,person_b,x3982-tpzed-1vl3k7knf7qihbe,x3982
+person_b at example.com,person_b,x6b1s-tpzed-w4nhkx2rmrhlr54,x3982
</pre>
h2. Migrate users
To avoid disruption, advise users to log out and avoid running workflows while performing the migration.
-After updating @users.csv@, use the @--migrate@ option:
+After updating @users.csv@, use the @--migrate@ option (add @--tokens tokens.csv@ if not using LoginCluster):
<pre>
-$ arv-federation-migrate --tokens tokens.csv --migrate users.csv
+$ arv-federation-migrate --migrate users.csv
(person_b at example.com) Migrating x6b1s-tpzed-w4nhkx2rmrhlr54 to x3982-tpzed-1vl3k7knf7qihbe
</pre>
diff --git a/doc/admin/upgrade-crunch2.html.textile.liquid b/doc/admin/upgrade-crunch2.html.textile.liquid
index 19463581f..7d5aac472 100644
--- a/doc/admin/upgrade-crunch2.html.textile.liquid
+++ b/doc/admin/upgrade-crunch2.html.textile.liquid
@@ -10,7 +10,7 @@ Copyright (C) The Arvados Authors. All rights reserved.
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-The "containers" API is the recommended way to submit compute work to Arvados. It supersedes the "jobs" API, which is deprecated.
+The "containers" API is the recommended way to submit compute work to Arvados. It supersedes the "jobs" API, which is end-of-life as of Arvados 1.5.
h2. Benefits over the "jobs" API
diff --git a/doc/architecture/federation.html.textile.liquid b/doc/architecture/federation.html.textile.liquid
index 08dad1e31..82f7a33be 100644
--- a/doc/architecture/federation.html.textile.liquid
+++ b/doc/architecture/federation.html.textile.liquid
@@ -43,15 +43,6 @@ Clusters:
Proxy: true
</pre>
-The @remote_hosts@ section of @application.yml@ (for the API server)
-
-<pre>
-production:
- remote_hosts:
- clsr2: api.cluster2.com
- clsr3: api.cluster3.com
-</pre>
-
In this example, the cluster @clsr1@ is configured to contact @api.cluster2.com@ for requests involving @clsr2@ and @api.cluster3.com@ for requests involving @clsr3 at .
h2(#identity). Identity
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list