[ARVADOS] updated: 1.3.0-1957-gbf2aee923
Git user
git at public.arvados.org
Tue Dec 10 20:41:37 UTC 2019
Summary of changes:
doc/_config.yml | 2 +-
doc/_includes/_install_rails_command.liquid | 11 -
doc/_includes/_install_ruby_and_bundler.liquid | 10 +-
.../install-arv-git-httpd.html.textile.liquid | 242 +++++++--------------
doc/install/install-composer.html.textile.liquid | 77 ++++---
doc/install/install-controller.html.textile.liquid | 197 -----------------
.../install-shell-server.html.textile.liquid | 116 ++--------
.../install-workbench-app.html.textile.liquid | 193 +++++-----------
.../install-workbench2-app.html.textile.liquid | 81 ++++---
doc/install/install-ws.html.textile.liquid | 172 +++++----------
10 files changed, 298 insertions(+), 803 deletions(-)
delete mode 100644 doc/install/install-controller.html.textile.liquid
via bf2aee923894b06592fa87977787559bb14aac18 (commit)
from a1d166358687d0f9c82e41c98c69fccc68eafa64 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
commit bf2aee923894b06592fa87977787559bb14aac18
Author: Peter Amstutz <peter.amstutz at curii.com>
Date: Tue Dec 10 15:40:53 2019 -0500
15572: Update workbench, composer, websocket, git, shell install docs
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz at curii.com>
diff --git a/doc/_config.yml b/doc/_config.yml
index 4ce347b50..2003594da 100644
--- a/doc/_config.yml
+++ b/doc/_config.yml
@@ -211,8 +211,8 @@ navbar:
- install/install-composer.html.textile.liquid
- Additional services:
- install/install-ws.html.textile.liquid
- - install/install-shell-server.html.textile.liquid
- install/install-arv-git-httpd.html.textile.liquid
+ - install/install-shell-server.html.textile.liquid
- Containers API support on SLURM:
- install/crunch2-slurm/install-prerequisites.html.textile.liquid
- install/crunch2-slurm/install-slurm.html.textile.liquid
diff --git a/doc/_includes/_install_rails_command.liquid b/doc/_includes/_install_rails_command.liquid
index 027f64beb..10c17a0c2 100644
--- a/doc/_includes/_install_rails_command.liquid
+++ b/doc/_includes/_install_rails_command.liquid
@@ -28,17 +28,6 @@ This template recognizes four variables:
{% assign railscmd = "bundle exec rails console" %}
{% endunless %}
-Using RVM:
-
-<notextile>
-<pre><code>{{railshost}}~$ <span class="userinput">cd {{railsdir}}</span>
-{{railshost}}{{railsdir}}$ <span class="userinput">sudo -u <b>webserver-user</b> RAILS_ENV=production `which rvm-exec` default {{railscmd}}</span>
-{% if railsout %}{{railsout}}
-{% endif %}</code></pre>
-</notextile>
-
-Not using RVM:
-
<notextile>
<pre><code>{{railshost}}~$ <span class="userinput">cd {{railsdir}}</span>
{{railshost}}{{railsdir}}$ <span class="userinput">sudo -u <b>webserver-user</b> RAILS_ENV=production {{railscmd}}</span>
diff --git a/doc/_includes/_install_ruby_and_bundler.liquid b/doc/_includes/_install_ruby_and_bundler.liquid
index c104e3df1..7871f2dc6 100644
--- a/doc/_includes/_install_ruby_and_bundler.liquid
+++ b/doc/_includes/_install_ruby_and_bundler.liquid
@@ -45,13 +45,19 @@ h3. Install RVM
\curl -sSL https://get.rvm.io | bash -s stable --ruby=2.5
</span></code></pre></notextile>
-Either log out and log back in to activate RVM, or explicitly load it in all open shells like this:
+To use Ruby installed from RVM, load it in an open shell like this:
<notextile>
<pre><code><span class="userinput">. /usr/local/rvm/scripts/rvm
</span></code></pre></notextile>
-Once RVM is activated in your shell, install Bundler:
+Alternately you can use @rvm-exec@ (the first parameter is the ruby version to use, or "default"), for example:
+
+<notextile>
+<pre><code><span class="userinput">rvm-exec default rails console
+</span></code></pre></notextile>
+
+Finally, install Bundler:
<notextile>
<pre><code>~$ <span class="userinput">gem install bundler</span>
diff --git a/doc/install/install-arv-git-httpd.html.textile.liquid b/doc/install/install-arv-git-httpd.html.textile.liquid
index c25fdee1d..964f5a25a 100644
--- a/doc/install/install-arv-git-httpd.html.textile.liquid
+++ b/doc/install/install-arv-git-httpd.html.textile.liquid
@@ -9,63 +9,46 @@ Copyright (C) The Arvados Authors. All rights reserved.
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-Arvados allows users to create their own private and public git repositories, and clone/push them using SSH and HTTPS.
+# "Introduction":#introduction
+# "Install dependencies":#dependencies
+# "Create "git" user and storage directory":#create
+# "Install gitolite":#gitolite
+# "Configure gitolite":#config-gitolite
+# "Configure git synchronization":#sync
+# "Update config.yml":#update-config
+# "Update nginx configuration":#update-nginx
+# "Install arvados-git-httpd package":#install-packages
+# "Restart the API server and controller":#restart-api
+# "Confirm working installation":#confirm-working
+
+h2(#introduction). Introduction
+
+Arvados support for git repository management enables using Arvados permissions to control access to git repositories. Users can create their own private and public git repositories and share them with others.
The git hosting setup involves three components.
* The "arvados-git-sync.rb" script polls the API server for the current list of repositories, creates bare repositories, and updates the local permission cache used by gitolite.
-* Gitolite provides SSH access.
-* arvados-git-http provides HTTPS access.
+* Gitolite provides SSH access. Users authenticate by SSH keys.
+* arvados-git-http provides HTTPS access. Users authenticate by Arvados tokens.
-It is not strictly necessary to deploy _both_ SSH and HTTPS access, but we recommend deploying both:
-* SSH is a more appropriate way to authenticate from a user's workstation because it does not require managing tokens on the client side;
-* HTTPS is a more appropriate way to authenticate from a shell VM because it does not depend on SSH agent forwarding (SSH clients' agent forwarding features tend to behave as if the remote machine is fully trusted).
-* HTTPS is also used by Arvados Composer to access git repositories from the browser.
+Git services must be installed on the same host as the Arvados Rails API server.
-The HTTPS instructions given below will not work if you skip the SSH setup steps.
+h2(#dependencies). Install dependencies
-h2. Set up DNS
-
-By convention, we use the following hostname for the git service:
+h3. Centos 7
<notextile>
-<pre><code>git.<span class="userinput">uuid_prefix</span>.your.domain
+<pre><code># <span class="userinput">yum install git perl-Data-Dumper openssh-server</span>
</code></pre>
</notextile>
-{% include 'notebox_begin' %}
-Here, we show how to install the git hosting services *on the same host as your API server.* Using a different host is not yet fully supported. On this page we will refer to it as your git server.
-{% include 'notebox_end' %}
-
-DNS and network configuration should be set up so port 443 reaches your HTTPS proxy, and port 22 reaches the OpenSSH service on your git server.
-
-h2. Generate an API token
-
-{% assign railshost = "gitserver" %}
-{% assign railscmd = "bundle exec ./script/create_superuser_token.rb" %}
-{% assign railsout = "zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz" %}
-Use the following command to generate an API token. {% include 'install_rails_command' %}
-
-Copy that token; you'll need it in a minute.
-
-h2. Install git and other dependencies
-
-On Debian-based systems:
+h3. Debian and Ubuntu
<notextile>
-<pre><code>gitserver:~$ <span class="userinput">sudo apt-get install git openssh-server</span>
+<pre><code># <span class="userinput">apt-get --no-install-recommends install git openssh-server</span>
</code></pre>
</notextile>
-On Red Hat-based systems:
-
-<notextile>
-<pre><code>gitserver:~$ <span class="userinput">sudo yum install git perl-Data-Dumper openssh-server</span>
-</code></pre>
-</notextile>
-
-{% include 'install_git' %}
-
-h2. Create a "git" user and a storage directory
+h2(#create). Create "git" user and storage directory
Gitolite and some additional scripts will be installed in @/var/lib/arvados/git@, which means hosted repository data will be stored in @/var/lib/arvados/git/repositories at . If you choose to install gitolite in a different location, make sure to update the @git_repositories_dir@ entry in your API server's @application.yml@ file accordingly: for example, if you install gitolite at @/data/gitolite@ then your @git_repositories_dir@ will be @/data/gitolite/repositories at .
@@ -93,15 +76,16 @@ git at gitserver:~$ <span class="userinput">rm .ssh/authorized_keys</span>
</code></pre>
</notextile>
-h2. Install gitolite
+h2(#gitolite). Install gitolite
Check "https://github.com/sitaramc/gitolite/tags":https://github.com/sitaramc/gitolite/tags for the latest stable version. This guide was tested with @v3.6.4 at . _Versions below 3.0 are missing some features needed by Arvados, and should not be used._
Download and install the version you selected.
<notextile>
-<pre><code>git at gitserver:~$ <span class="userinput">echo 'PATH=$HOME/bin:$PATH' >.profile</span>
-git at gitserver:~$ <span class="userinput">source .profile</span>
+<pre><code># <span class="userinput">su git</span>
+git at gitserver:~$ <span class="userinput">echo 'PATH=$HOME/bin:$PATH' >.profile</span>
+git at gitserver:~$ <span class="userinput">. .profile</span>
git at gitserver:~$ <span class="userinput">git clone --branch <b>v3.6.4</b> https://github.com/sitaramc/gitolite</span>
...
Note: checking out '5d24ae666bfd2fa9093d67c840eb8d686992083f'.
@@ -137,7 +121,7 @@ Everything up-to-date
</code></pre>
</notextile>
-h3. Configure gitolite
+h2(#config-gitolite). Configure gitolite
Configure gitolite to look up a repository name like @username/reponame.git@ and find the appropriate bare repository storage directory.
@@ -175,17 +159,17 @@ Uncomment the 'Alias' line in the section that begins @ENABLE => [@:
</span></code></pre>
</notextile>
-h2. Configure git synchronization
+h2(#sync). Configure git synchronization
Create a configuration file @/var/www/arvados-api/current/config/arvados-clients.yml@ using the following template, filling in the appropriate values for your system.
-* For @arvados_api_token@, use the token you generated above.
+* For @arvados_api_token@, use @SystemRootToken@
* For @gitolite_arvados_git_user_key@, provide the public key you generated above, i.e., the contents of @~git/.ssh/id_rsa.pub at .
<notextile>
<pre><code>production:
gitolite_url: /var/lib/arvados/git/repositories/gitolite-admin.git
gitolite_tmp: /var/lib/arvados/git
- arvados_api_host: <span class="userinput">uuid_prefix.example.com</span>
+ arvados_api_host: <span class="userinput">ClusterID.example.com</span>
arvados_api_token: "<span class="userinput">zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz</span>"
arvados_api_host_insecure: <span class="userinput">false</span>
gitolite_arvados_git_user_key: "<span class="userinput">ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC7aBIDAAgMQN16Pg6eHmvc+D+6TljwCGr4YGUBphSdVb25UyBCeAEgzqRiqy0IjQR2BLtSirXr+1SJAcQfBgI/jwR7FG+YIzJ4ND9JFEfcpq20FvWnMMQ6XD3y3xrZ1/h/RdBNwy4QCqjiXuxDpDB7VNP9/oeAzoATPZGhqjPfNS+RRVEQpC6BzZdsR+S838E53URguBOf9yrPwdHvosZn7VC0akeWQerHqaBIpSfDMtaM4+9s1Gdsz0iP85rtj/6U/K/XOuv2CZsuVZZ52nu3soHnEX2nx2IaXMS3L8Z+lfOXB2T6EaJgXF7Z9ME5K1tx9TSNTRcYCiKztXLNLSbp git at gitserver</span>"
@@ -196,115 +180,37 @@ h3. Enable the synchronization script
The API server package includes a script that retrieves the current set of repository names and permissions from the API, writes them to @arvadosaliases.pl@ in a format usable by gitolite, and triggers gitolite hooks which create new empty repositories if needed. This script should run every 2 to 5 minutes.
-If you are using RVM, create @/etc/cron.d/arvados-git-sync@ with the following content:
-
-<notextile>
-<pre><code><span class="userinput">*/5 * * * * git cd /var/www/arvados-api/current && /usr/local/rvm/bin/rvm-exec default bundle exec script/arvados-git-sync.rb production</span>
-</code></pre>
-</notextile>
-
-Otherwise, create @/etc/cron.d/arvados-git-sync@ with the following content:
+Create @/etc/cron.d/arvados-git-sync@ with the following content:
<notextile>
<pre><code><span class="userinput">*/5 * * * * git cd /var/www/arvados-api/current && bundle exec script/arvados-git-sync.rb production</span>
</code></pre>
</notextile>
-h3. Configure the API server to advertise the correct SSH URLs
+h2(#update-config). Update config.yml
-Edit the cluster config at @/etc/arvados/config.yml@ and set @Services.GitSSH.ExternalURL at . Replace @uuid_prefix@ with your cluster id.
+Edit the cluster config at @/etc/arvados/config.yml@ .
<notextile>
-<pre><code>Clusters:
- <span class="userinput">uuid_prefix</span>:
- Services:
+<pre><code> Services:
GitSSH:
- ExternalURL: <span class="userinput">git at git.uuid_prefix.your.domain:</span>
-</code></pre>
-</notextile>
-
-Make sure to include the trailing colon.
-
-h2. Install the arvados-git-httpd package
-
-This is needed only for HTTPS access.
-
-The arvados-git-httpd package provides HTTP access, using Arvados authentication tokens instead of passwords. It is intended to be installed on the system where your git repositories are stored, and accessed through a web proxy that provides SSL support.
-
-On Debian-based systems:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo apt-get install git arvados-git-httpd</span>
-</code></pre>
-</notextile>
-
-On Red Hat-based systems:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo yum install git arvados-git-httpd</span>
-~$ <span class="userinput">sudo systemctl enable arvados-git-httpd</span>
-</code></pre>
-</notextile>
-
-Verify that @arvados-git-httpd@ and @git-http-backend@ can be run:
-
-<notextile>
-<pre><code>~$ <span class="userinput">arvados-git-httpd -h</span>
-[...]
-Usage: arvados-git-httpd [-config path/to/arvados/git-httpd.yml]
-[...]
-~$ <span class="userinput">git http-backend</span>
-Status: 500 Internal Server Error
-Expires: Fri, 01 Jan 1980 00:00:00 GMT
-Pragma: no-cache
-Cache-Control: no-cache, max-age=0, must-revalidate
-
-fatal: No REQUEST_METHOD from server
-</code></pre>
-</notextile>
-
-h3. Enable arvados-git-httpd
-
-{% include 'notebox_begin' %}
-
-The arvados-git-httpd package includes configuration files for systemd. If you're using a different init system, you'll need to configure a service to start and stop an @arvados-git-httpd@ process as desired.
-
-{% include 'notebox_end' %}
-
-Edit the cluster config at @/etc/arvados/config.yml@ and set the following values. Replace @uuid_prefix@ with your cluster id.
-
-<notextile>
-<pre><code>Clusters:
- <span class="userinput">uuid_prefix</span>:
- Services:
+ ExternalURL: <span class="userinput">git at git.ClusterID.example.com:</span>
GitHTTP:
- ExternalURL: <span class="userinput">https://git.uuid_prefix.your.domain/</span>
+ ExternalURL: <span class="userinput">https://git.ClusterID.example.com/</span>
InternalURLs:
- <span class="userinput">"http://localhost:9001": {}</span>
+ <span class="userinput">"http://git.ClusterID.example.com:9001": {}</span>
Git:
GitCommand: <span class="userinput">/var/lib/arvados/git/gitolite/src/gitolite-shell</span>
GitoliteHome: <span class="userinput">/var/lib/arvados/git</span>
- Repositories: <span class="userinput">/var/lib/arvados/git/repositories</span>
-</code></pre>
-</notextile>
-
-Make sure to include the trailing slash for @Services.GitHTTP.ExternalURL at .
-
-Restart the systemd service to ensure the new configuration is used.
-
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo systemctl restart arvados-git-httpd</span>
+ Repositories: <span class="userinput">/var/lib/arvados/git/repositories</span>
</code></pre>
</notextile>
-h3. Set up a reverse proxy to provide SSL service
-
-The arvados-git-httpd service will be accessible from anywhere on the internet, so we recommend using SSL.
+Make sure to include the trailing colon in @Services.GitSSH.ExternalURL at .
-This is best achieved by putting a reverse proxy with SSL support in front of arvados-git-httpd, running on port 443 and passing requests to @arvados-git-httpd@ on port 9001 (or whichever port you used in your run script).
+h2(#update-nginx). Update nginx configuration
-Add the following configuration to the @http@ section of your Nginx configuration:
+Use a text editor to create a new file @/etc/nginx/conf.d/arvados-git.conf@ with the following configuration. Options that need attention are marked with "TODO".
<notextile>
<pre><code>
@@ -313,7 +219,7 @@ upstream arvados-git-httpd {
}
server {
listen <span class="userinput">[your public IP address]</span>:443 ssl;
- server_name git.<span class="userinput">uuid_prefix.your.domain</span>;
+ server_name git.<span class="userinput">ClusterID.example.com</span>;
proxy_connect_timeout 90s;
proxy_read_timeout 300s;
@@ -322,7 +228,7 @@ server {
ssl_certificate_key <span class="userinput">/YOUR/PATH/TO/cert.key</span>;
# The server needs to accept potentially large refpacks from push clients.
- client_max_body_size 50m;
+ client_max_body_size 128m;
location / {
proxy_pass http://arvados-git-httpd;
@@ -331,55 +237,55 @@ server {
</code></pre>
</notextile>
-h2. Restart Nginx
+h2(#install-packages). Install the arvados-git-httpd package
+
+The arvados-git-httpd package provides HTTP access, using Arvados authentication tokens instead of passwords. It must be installed on the system where your git repositories are stored.
+
+h3. Centos 7
+
+<notextile>
+<pre><code># <span class="userinput">yum install arvados-git-httpd</span>
+</code></pre>
+</notextile>
-Restart Nginx to make the Nginx and API server configuration changes take effect.
+h3. Debian and Ubuntu
<notextile>
-<pre><code>gitserver:~$ <span class="userinput">sudo nginx -s reload</span>
+<pre><code># <span class="userinput">apt-get --no-install-recommends install arvados-git-httpd</span>
</code></pre>
</notextile>
-h2. Clone Arvados repository
+h2(#restart-api). Restart the API server and controller
-Here we create a repository object which will be used to set up a hosted clone of the arvados repository on this cluster.
+After adding Workbench to the Services section, make sure the cluster config file is up to date on the API server host, and restart the API server and controller processes to ensure the changes are applied.
<notextile>
-<pre><code>~$ <span class="userinput">uuid_prefix=`arv --format=uuid user current | cut -d- -f1`</span>
-~$ <span class="userinput">echo "Site prefix is '$uuid_prefix'"</span>
-~$ <span class="userinput">all_users_group_uuid="$uuid_prefix-j7d0g-fffffffffffffff"</span>
-~$ <span class="userinput">repo_uuid=`arv --format=uuid repository create --repository "{\"owner_uuid\":\"$uuid_prefix-tpzed-000000000000000\", \"name\":\"arvados\"}"`</span>
-~$ <span class="userinput">echo "Arvados repository uuid is '$repo_uuid'"</span>
-</code></pre></notextile>
+<pre><code># <span class="userinput">systemctl restart nginx arvados-controller</span>
+</code></pre>
+</notextile>
+
+h2(#confirm-working). Confirm working installation
-Create a link object to make the repository object readable by the "All users" group, and therefore by every active user. This makes it possible for users to run the bundled Crunch scripts by specifying @"script_version":"master","repository":"arvados"@ rather than pulling the Arvados source tree into their own repositories.
+Create 'testrepo' in the Arvados database.
<notextile>
-<pre><code>~$ <span class="userinput">read -rd $'\000' newlink <<EOF; arv link create --link "$newlink"</span>
-<span class="userinput">{
- "tail_uuid":"$all_users_group_uuid",
- "head_uuid":"$repo_uuid",
- "link_class":"permission",
- "name":"can_read"
-}
-EOF</span>
+<pre><code>~$ <span class="userinput">arv --format=uuid repository create --repository '{"name":"testrepo"}'</span>
</code></pre></notextile>
-In a couple of minutes, your arvados-git-sync cron job will create an empty repository on your git server. Seed it with the real arvados repository. If your git credential helpers were configured correctly when you "set up your shell server":install-shell-server.html, the "git push" command will use your API token instead of prompting you for a username and password.
+The arvados-git-sync cron job will notice the new repository record and create a repository on disk. Because it is on a timer (default 5 minutes) you may have to wait a minute or two for it to show up.
+
+h3. SSH
<notextile>
-<pre><code>~$ <span class="userinput">cd /tmp</span>
-/tmp$ <span class="userinput">git clone --bare https://github.com/curoverse/arvados.git</span>
-/tmp <span class="userinput">git --git-dir arvados.git push https://git.<b>uuid_prefix.your.domain</b>/arvados.git '*:*'</span>
+<pre><code>~$ <span class="userinput">git clone git at git.ClusterID.example.com:username/testrepo.git</span>
</code></pre>
</notextile>
-If you did not set up a HTTPS service, you can push to <code>git at git.uuid_prefix.your.domain:arvados.git</code> using your SSH key, or by logging in to your git server and using sudo.
+h3. HTTP
+
+Set up git credential helpers as described in "install shell server":install-shell-server.html for the "git push" command to use your API token instead of prompting you for a username and password.
<notextile>
-<pre><code>gitserver:~$ <span class="userinput">sudo -u git -i bash</span>
-git at gitserver:~$ <span class="userinput">git clone --bare https://github.com/curoverse/arvados.git /tmp/arvados.git</span>
-git at gitserver:~$ <span class="userinput">cd /tmp/arvados.git</span>
-git at gitserver:/tmp/arvados.git$ <span class="userinput">gitolite push /var/lib/arvados/git/repositories/<b>your_arvados_repo_uuid</b>.git '*:*'</span>
+<pre><code>~$ <span class="userinput">git clone https://git.ClusterID.example.com/username/testrepo.git</span>
</code></pre>
</notextile>
diff --git a/doc/install/install-composer.html.textile.liquid b/doc/install/install-composer.html.textile.liquid
index f0938e860..d27db4d4e 100644
--- a/doc/install/install-composer.html.textile.liquid
+++ b/doc/install/install-composer.html.textile.liquid
@@ -11,57 +11,70 @@ SPDX-License-Identifier: CC-BY-SA-3.0
Arvados Composer is a web-based javascript application for building Common Workflow Languge (CWL) Workflows.
-h2. Prerequisites
+# "Install dependencies":#dependencies
+# "Update config.yml":#update-config
+# "Update Nginx configuration":#update-nginx
+# "Install arvados-composer":#install-packages
+# "Restart the API server and controller":#restart-api
+# "Confirm working installation":#confirm-working
-In addition to Arvados core services, Composer requires "Arvados hosted git repositories":install-arv-git-httpd.html which are used for storing workflow files.
-
-h2. Install
+h2(#dependencies). Install dependencies
-Composer may be installed on the same host as Workbench, or on a different host. Composer communicates directly with the Arvados API server. It does not require its own backend and should be served as a static file.
-
-On a Debian-based system, install the following package:
+In addition to Arvados core services, Composer requires "Arvados hosted git repositories":install-arv-git-httpd.html which are used for storing workflow files.
-<notextile>
-<pre><code>~$ <span class="userinput">sudo apt-get install arvados-composer</span>
-</code></pre>
-</notextile>
+h2(#configure). Update config.yml
-On a Red Hat-based system, install the following package:
+Edit @config.yml@ and set @Services.Composer.ExternalURL@ to the location from which it is served:
<notextile>
-<pre><code>~$ <span class="userinput">sudo yum install arvados-composer</span>
-</code></pre>
+<pre><code> Services:
+ Composer:
+ ExternalURL: <span class="userinput">https://workbench.CusterID.example.com/composer</span></code></pre>
</notextile>
-h2. Configure
+h2(#update-nginx). Update nginx configuration
-h3. Nginx
+Composer may be served from the same host as Workbench. Composer communicates directly with the Arvados API server. It does not require its own backend and should be served as a static file.
-Add Composer to your Nginx configuration. This example will host Composer at @/composer at .
+Use a text editor to create a new file @/etc/nginx/conf.d/arvados-composer.conf@ with the following configuration. Options that need attention are marked with "TODO".
-<pre>
-location /composer {
+<notextile>
+<pre><code>location /composer {
root /var/www/arvados-composer
index index.html
}
-</pre>
-h3. composer.yml
+location /composer.yml {
+ return 200 '{ "API_HOST": "<span class="userinput">ClusterID.example.com</span>" }';
+}
+</code></pre>
+</notextile>
+
+h2(#install-packages). Install arvados-composer
-Create @/var/www/arvados-composer/composer.yml@ and set @API_HOST@ to your API server:
+h3. Centos 7
-<pre>
-API_HOST: zzzzz.arvadosapi.com
-</pre>
+<notextile>
+<pre><code># <span class="userinput">yum install arvados-composer</span>
+</code></pre>
+</notextile>
-h3. Workbench link to composer
+h3. Debian and Ubuntu
-Edit @config.yml@ and set @Services.Composer.ExternalURL@ to the location from which it is served:
+<notextile>
+<pre><code># <span class="userinput">apt-get --no-install-recommends install arvados-composer</span>
+</code></pre>
+</notextile>
+
+h2(#restart-api). Restart the API server and controller
+
+After adding Workbench to the Services section, make sure the cluster config file is up to date on the API server host, and restart the API server and controller processes to ensure the changes are applied.
<notextile>
-<pre><code>Clusters:
- zzzzz:
- Services:
- Composer:
- ExternalURL: <span class="userinput">https://workbench.zzzzz.arvadosapi.com/composer</span></code></pre>
+<pre><code># <span class="userinput">systemctl restart nginx arvados-controller</span>
+</code></pre>
</notextile>
+
+h2(#confirm-working). Confirm working installation
+
+Visit @https://workbench.ClusterID.example.com/composer@ in a browser. You should be able to log in using the login method you configured previously.
diff --git a/doc/install/install-controller.html.textile.liquid b/doc/install/install-controller.html.textile.liquid
deleted file mode 100644
index f78467f5b..000000000
--- a/doc/install/install-controller.html.textile.liquid
+++ /dev/null
@@ -1,197 +0,0 @@
----
-layout: default
-navsection: installguide
-title: Install the controller
-...
-{% comment %}
-Copyright (C) The Arvados Authors. All rights reserved.
-
-SPDX-License-Identifier: CC-BY-SA-3.0
-{% endcomment %}
-
-The arvados-controller service must be installed on your API server node.
-
-On Debian-based systems:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo apt-get install arvados-controller</span>
-</code></pre>
-</notextile>
-
-On Red Hat-based systems:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo yum install arvados-controller</span>
-</code></pre>
-</notextile>
-
-Verify the @arvados-controller@ program is functional:
-
-<notextile>
-<pre><code>~$ <span class="userinput">arvados-controller -h</span>
-Usage:
- -config file
-[...]
-</code></pre>
-</notextile>
-
-h3. Configure Nginx to route requests to the controller
-
-Add @upstream@ and @server@ definitions inside the @http@ section of your Nginx configuration using the following template.
-
-{% include 'notebox_begin' %}
-
-If you are adding arvados-controller to an existing system as part of the upgrade procedure, do not add a new "server" part here. Instead, add only the "upstream" part as shown here, and update your existing "server" section by changing its @proxy_pass@ directive from @http://api@ to @http://controller@.
-
-{% include 'notebox_end' %}
-
-<notextile>
-<pre><code>upstream controller {
- server 127.0.0.1:9004 fail_timeout=10s;
-}
-
-server {
- listen <span class="userinput">[your public IP address]</span>:443 ssl;
- server_name <span class="userinput">uuid_prefix.your.domain</span>;
-
- ssl on;
- ssl_certificate <span class="userinput">/YOUR/PATH/TO/cert.pem</span>;
- ssl_certificate_key <span class="userinput">/YOUR/PATH/TO/cert.key</span>;
-
- # Refer to the comment about this setting in the passenger (arvados
- # api server) section of your Nginx configuration.
- client_max_body_size 128m;
-
- location / {
- proxy_pass http://controller;
- proxy_redirect off;
- proxy_connect_timeout 90s;
- proxy_read_timeout 300s;
-
- proxy_set_header X-Forwarded-Proto https;
- proxy_set_header Host $http_host;
- proxy_set_header X-External-Client $external_client;
- proxy_set_header X-Real-IP $remote_addr;
- proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
- }
-}
-</code></pre>
-</notextile>
-
-Restart Nginx to apply the new configuration.
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo nginx -s reload</span>
-</code></pre>
-</notextile>
-
-h3(#configuration). Configure arvados-controller
-
-Create the cluster configuration file @/etc/arvados/config.yml@ using the following template.
-
-<notextile>
-<pre><code>Clusters:
- <span class="userinput">uuid_prefix</span>:
- Services:
- Controller:
- InternalURLs:
- "http://localhost:<span class="userinput">9004</span>": {} # must match the "upstream controller" section of your Nginx config
- RailsAPI:
- arvados-api-server:
- "http://localhost:<span class="userinput">8000</span>": {} # must match the "upstream api" section of your Nginx config
- PostgreSQL:
- ConnectionPool: 128
- Connection:
- host: localhost
- dbname: arvados_production
- user: arvados
- password: <span class="userinput">xxxxxxxx</span>
- sslmode: require
-</code></pre>
-</notextile>
-
-Create the host configuration file @/etc/arvados/environment at .
-
-<notextile>
-<pre><code>ARVADOS_NODE_PROFILE=apiserver
-</code></pre>
-</notextile>
-
-h3. Start the service (option 1: systemd)
-
-If your system does not use systemd, skip this section and follow the "runit instructions":#runit instead.
-
-If your system uses systemd, the arvados-controller service should already be set up. Restart it to load the new configuration file, and check its status:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo systemctl restart arvados-controller</span>
-~$ <span class="userinput">sudo systemctl status arvados-controller</span>
-● arvados-controller.service - Arvados controller
- Loaded: loaded (/lib/systemd/system/arvados-controller.service; enabled; vendor preset: enabled)
- Active: active (running) since Tue 2018-07-31 13:17:44 UTC; 3s ago
- Docs: https://doc.arvados.org/
- Main PID: 25066 (arvados-control)
- CGroup: /system.slice/arvados-controller.service
- └─25066 /usr/bin/arvados-controller
-
-Jul 31 13:17:44 zzzzz systemd[1]: Starting Arvados controller...
-Jul 31 13:17:44 zzzzz arvados-controller[25191]: {"Listen":"[::]:9004","Service":"arvados-controller","level":"info","msg":"listening","time":"2018-07-31T13:17:44.521694195Z"}
-Jul 31 13:17:44 zzzzz systemd[1]: Started Arvados controller.
-</code></pre>
-</notextile>
-
-Skip ahead to "confirm the service is working":#confirm.
-
-h3(#runit). Start the service (option 2: runit)
-
-Install runit to supervise the arvados-controller daemon. {% include 'install_runit' %}
-
-Create a supervised service.
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo mkdir /etc/service/arvados-controller</span>
-~$ <span class="userinput">cd /etc/service/arvados-controller</span>
-~$ <span class="userinput">sudo mkdir log log/main</span>
-~$ <span class="userinput">printf '#!/bin/sh\nset -a\n. /etc/arvados/environment\nexec arvados-controller 2>&1\n' | sudo tee run</span>
-~$ <span class="userinput">printf '#!/bin/sh\nexec svlogd main\n' | sudo tee log/run</span>
-~$ <span class="userinput">sudo chmod +x run log/run</span>
-~$ <span class="userinput">sudo sv exit .</span>
-~$ <span class="userinput">cd -</span>
-</code></pre>
-</notextile>
-
-Use @sv stat@ and check the log file to verify the service is running.
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo sv stat /etc/service/arvados-controller</span>
-run: /etc/service/arvados-controller: (pid 12520) 2s; run: log: (pid 12519) 2s
-~$ <span class="userinput">tail /etc/service/arvados-controller/log/main/current</span>
-{"Listen":"[::]:9004","Service":"arvados-controller","level":"info","msg":"listening","time":"2018-07-31T13:17:44.521694195Z"}
-</code></pre>
-</notextile>
-
-h3(#confirm). Confirm the service is working
-
-Confirm the service is listening on its assigned port and responding to requests.
-
-<notextile>
-<pre><code>~$ <span class="userinput">curl -X OPTIONS http://0.0.0.0:<b>9004</b>/login</span>
-{"errors":["Forbidden"],"error_token":"1533044555+684b532c"}
-</code></pre>
-</notextile>
-
-h3(#confirm-config). Confirm the public configuration is OK
-
-Confirm the publicly accessible configuration endpoint does not reveal any sensitive information (e.g., a secret that was mistakenly entered under the wrong configuration key). Use the jq program, if you have installed it, to make the JSON document easier to read.
-
-<notextile>
-<pre><code>~$ <span class="userinput">curl http://0.0.0.0:<b>9004</b>/arvados/v1/config | jq .</span>
-{
- "API": {
- "MaxItemsPerResponse": 1000,
- "MaxRequestAmplification": 4,
- "RequestTimeout": "5m"
- },
- ...
-</code></pre>
-</notextile>
diff --git a/doc/install/install-shell-server.html.textile.liquid b/doc/install/install-shell-server.html.textile.liquid
index 1cbe74997..5b35aeb89 100644
--- a/doc/install/install-shell-server.html.textile.liquid
+++ b/doc/install/install-shell-server.html.textile.liquid
@@ -1,7 +1,7 @@
---
layout: default
navsection: installguide
-title: Install a shell server
+title: Install a shell node
...
{% comment %}
Copyright (C) The Arvados Authors. All rights reserved.
@@ -9,61 +9,14 @@ Copyright (C) The Arvados Authors. All rights reserved.
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-There is nothing inherently special about an Arvados shell server. It is just a GNU/Linux machine with Arvados utilites and SDKs installed. For optimal performance, the Arvados shell server should be on the same LAN as the Arvados cluster, but that is not required.
+Arvados support for shell nodes enables using Arvados permissions to grant shell accounts to users.
-h2. Install API tokens
+A shell node runs the @arvados-login-sync@ service, and has some additional configuration to make it convenient for users to use Arvados utilites and SDKs. Users are allowed to log in and run arbitrary programs. For optimal performance, the Arvados shell server should be on the same LAN as the Arvados cluster.
-Please follow the "API token guide":../user/reference/api-tokens.html to get API tokens for your Arvados account and install them on your shell server. We will use those tokens to test the SDKs as we install them.
+h2. Install Dependecies and SDKs
-h2. Install the Ruby SDK and utilities
-
-First, install the curl development libraries necessary to build the Arvados Ruby SDK. On Debian-based systems:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo apt-get install libcurl4-openssl-dev</span>
-</code></pre>
-</notextile>
-
-On Red Hat-based systems:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo yum install libcurl-devel</span>
-</code></pre>
-</notextile>
-
-Next, install the arvados-cli Ruby gem. If you're using RVM:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo /usr/local/rvm/bin/rvm-exec default gem install arvados-cli</span>
-</code></pre>
-</notextile>
-
-If you're not using RVM:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo -i gem install arvados-cli</span>
-</code></pre>
-</notextile>
-
-h2. Install the Python SDK and utilities
-
-{% assign rh_version = "7" %}
-{% include 'note_python_sc' %}
-
-On Red Hat-based systems:
-
-<notextile>
-<pre><code>~$ <span class="userinput">echo 'exclude=python2-llfuse' | sudo tee -a /etc/yum.conf</span>
-~$ <span class="userinput">sudo yum install python-arvados-python-client python-arvados-fuse crunchrunner</span>
-</code></pre>
-</notextile>
-
-On Debian-based systems:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo apt-get install python-arvados-python-client python-arvados-fuse crunchrunner</span>
-</code></pre>
-</notextile>
+# "Install the CLI":{{site.baseurl}}/sdk/cli/install.html
+# "Install the R SDK":{{site.baseurl}}/sdk/R/index.html (optional)
h2. Install Git and curl
@@ -80,7 +33,7 @@ Configure git to use the ARVADOS_API_TOKEN environment variable to authenticate
</pre>
</notextile>
-h2. Install arvados-login-sync
+h2. Create database entry for VM
This program makes it possible for Arvados users to log in to the shell server -- subject to permissions assigned by the Arvados administrator -- using the SSH keys they upload to Workbench. It sets up login accounts, updates group membership, and adds users' public keys to the appropriate @authorized_keys@ files.
@@ -93,7 +46,9 @@ zzzzz-2x53u-zzzzzzzzzzzzzzz</code>
</pre>
</notextile>
-Create a token that is allowed to read login information for this VM.
+h2. Create token
+
+As an admin arvados user (such as the system root user), create a token that is allowed to read login information for this VM.
<notextile>
<pre>
@@ -108,62 +63,21 @@ Create a token that is allowed to read login information for this VM.
Note the UUID and the API token output by the above commands: you will need them in a minute.
-Install the arvados-login-sync program.
-
-If you're using RVM:
-
-<notextile>
-<pre>
-<code>shellserver:~$ <span class="userinput">sudo -i `which rvm-exec` default gem install arvados-login-sync</span></code>
-</pre>
-</notextile>
+h2. Install arvados-login-sync
-If you're not using RVM:
+Install the arvados-login-sync program.
<notextile>
<pre>
-<code>shellserver:~$ <span class="userinput">sudo -i gem install arvados-login-sync</span></code>
+<code>shellserver:# <span class="userinput">gem install arvados-login-sync</span></code>
</pre>
</notextile>
-Install cron.
-
-On Red Hat-based distributions:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo yum install cronie</span>
-~$ <span class="userinput">sudo systemctl enable crond</span>
-~$ <span class="userinput">sudo systemctl start crond</span>
-</code></pre>
-</notextile>
-
-On Debian-based systems:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo apt-get install cron</span>
-</code></pre>
-</notextile>
-
Configure cron to run the @arvados-login-sync@ program every 2 minutes.
-If you're using RVM:
-
-<notextile>
-<pre>
-<code>shellserver:~$ <span class="userinput">sudo bash -c 'umask 077; tee /etc/cron.d/arvados-login-sync' <<'EOF'
-ARVADOS_API_HOST="<strong>uuid_prefix.your.domain</strong>"
-ARVADOS_API_TOKEN="<strong>the_token_you_created_above</strong>"
-ARVADOS_VIRTUAL_MACHINE_UUID="<strong>zzzzz-2x53u-zzzzzzzzzzzzzzz</strong>"
-*/2 * * * * root /usr/local/rvm/bin/rvm-exec default arvados-login-sync
-EOF</span></code>
-</pre>
-</notextile>
-
-If you're not using RVM:
-
<notextile>
<pre>
-<code>shellserver:~$ <span class="userinput">sudo bash -c 'umask 077; tee /etc/cron.d/arvados-login-sync' <<'EOF'
+<code>shellserver:# <span class="userinput">umask 077; tee /etc/cron.d/arvados-login-sync <<EOF
ARVADOS_API_HOST="<strong>uuid_prefix.your.domain</strong>"
ARVADOS_API_TOKEN="<strong>the_token_you_created_above</strong>"
ARVADOS_VIRTUAL_MACHINE_UUID="<strong>zzzzz-2x53u-zzzzzzzzzzzzzzz</strong>"
@@ -176,3 +90,5 @@ A user should be able to log in to the shell server when the following condition
* The user has uploaded an SSH public key: Workbench → Account menu → "SSH keys" item → "Add new SSH key" button.
* As an admin user, you have given the user permission to log in: Workbench → Admin menu → "Users" item → "Show" button → "Admin" tab → "Setup shell account" button.
* Two minutes have elapsed since the above conditions were satisfied, and the cron job has had a chance to run.
+
+
diff --git a/doc/install/install-workbench-app.html.textile.liquid b/doc/install/install-workbench-app.html.textile.liquid
index 72a80fd83..cf33cca35 100644
--- a/doc/install/install-workbench-app.html.textile.liquid
+++ b/doc/install/install-workbench-app.html.textile.liquid
@@ -9,41 +9,35 @@ Copyright (C) The Arvados Authors. All rights reserved.
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-h2. Install prerequisites
+# "Install dependencies":#dependencies
+# "Update config.yml":#update-config
+# "Update Nginx configuration":#update-nginx
+# "Install arvados-workbench":#install-packages
+# "Restart the API server and controller":#restart-api
+# "Confirm working installation":#confirm-working
+# "Trusted client setting":#trusted_client
-The Arvados package repository includes a Workbench server package that can help automate much of the deployment.
+h2(#dependencies). Install dependencies
-h3(#install_ruby_and_bundler). Install Ruby and Bundler
+# "Install Ruby and Bundler":ruby.html
+# "Install nginx":nginx.html
+# "Install Phusion Passenger":https://www.phusionpassenger.com/library/walkthroughs/deploy/ruby/ownserver/nginx/oss/install_passenger_main.html
-{% include 'install_ruby_and_bundler' %}
+h2(#configure). Update config.yml
-h2(#install_workbench). Install Workbench and dependencies
-
-Workbench doesn't need its own database, so it does not need to have PostgreSQL installed.
-
-{% assign rh_version = "7" %}
-{% include 'note_python_sc' %}
-
-On a Debian-based system, install the following packages:
+Edit @/etc/arvados/config.yml@ to set the keys below. The full set of configuration options are in the "Workbench section of config.yml":{{site.baseurl}}/admin/config.html
<notextile>
-<pre><code>~$ <span class="userinput">sudo apt-get install bison build-essential graphviz git python-arvados-python-client arvados-workbench</span>
-</code></pre>
-</notextile>
-
-On a Red Hat-based system, install the following packages:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo yum install bison make automake gcc gcc-c++ graphviz git python-arvados-python-client arvados-workbench</span>
+<pre><code> Services:
+ Workbench:
+ ExternalURL: <span class="userinput">"https://workbench.ClustedID.example.com"</span>
+ Workbench:
+ SecretKeyBase: <span class="userinput">aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa</span>
+ Users:
+ AutoAdminFirstUser: true
</code></pre>
</notextile>
-h2(#configure). Configure Workbench
-
-Edit @/etc/arvados/config.yml@ to set the keys below. Only the most important configuration options are listed here. The full set of configuration options are in the "Workbench section of config.yml":{{site.baseurl}}/admin/config.html
-
-h3. Workbench.SecretKeyBase
-
This application needs a secret token. Generate a new secret:
<notextile>
@@ -54,72 +48,23 @@ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
Then put that value in the @Workbench.SecretKeyBase@ field.
-<notextile>
-<pre><code>Cluster:
- zzzzz:
- Workbench:
- SecretKeyBase: <span class="userinput">aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa</span>
-</code></pre>
-</notextile>
-
-h3. Services.Controller.ExternalURL
-
-Ensure that @Services.Controller.ExternalURL@ is configured for "Arvados Controller":install-controller.html . For example like this:
-
-<notextile>
-<pre><code>Cluster:
- zzzzz:
- Services:
- Controller:
- ExternalURL: <span class="userinput">https://prefix_uuid.your.domain</span>
-</code></pre>
-</notextile>
-
-h3. Workbench.SiteName
-
- at Workbench.SiteName@ can be set to any arbitrary string. It is used to identify this Workbench to people visiting it.
-
-
-<notextile>
-<pre><code>Cluster:
- zzzzz:
- Workbench:
- SiteName: <span class="userinput">My Arvados</span>
-</code></pre>
-</notextile>
-
-h3. TLS.Insecure
-
-For testing only. Allows use of self-signed certificates. If true, workbench will not verify the TLS certificate of Arvados Controller.
-
-<notextile>
-<pre><code>Cluster:
- zzzzz:
- TLS:
- Insecure: <span class="userinput">false</span>
-</code></pre>
-</notextile>
-
-h2. Configure Piwik (optional)
+You probably want to enable @Users.AutoAdminFirstUser@ . The first user to log in when no other admin user exists will automatically be made an admin.
-Piwik can be used to gather usage analytics. In @/var/www/arvados-workbench/current/config@, copy @piwik.yml.example@ to @piwik.yml@ and edit to suit.
+h2(#update-nginx). Update nginx configuration
-h2. Set up Web server
-
-For best performance, we recommend you use Nginx as your Web server front-end, with a Passenger backend to serve Workbench. To do that:
+Use a text editor to create a new file @/etc/nginx/conf.d/arvados-workbench.conf@ with the following configuration. Options that need attention are marked with "TODO".
<notextile>
-<ol>
-<li><a href="https://www.phusionpassenger.com/library/walkthroughs/deploy/ruby/ownserver/nginx/oss/install_passenger_main.html">Install Nginx and Phusion Passenger</a>.</li>
-
-<li><p>Edit the http section of your Nginx configuration to run the Passenger server, and act as a front-end for it. You might add a block like the following, adding SSL and logging parameters to taste:</p>
-
<pre><code>server {
- listen 127.0.0.1:9000;
- server_name localhost-workbench;
+ listen <span class="userinput">[your public IP address]</span>:443 ssl;
+ server_name workbench.<span class="userinput">ClusterID.example.com</span>;
+
+ ssl on;
+ ssl_certificate <span class="userinput">/YOUR/PATH/TO/cert.pem</span>;
+ ssl_certificate_key <span class="userinput">/YOUR/PATH/TO/cert.key</span>;
root /var/www/arvados-workbench/current/public;
- index index.html index.htm index.php;
+ index index.html;
passenger_enabled on;
# If you're using RVM, uncomment the line below.
@@ -129,65 +74,39 @@ For best performance, we recommend you use Nginx as your Web server front-end, w
# the API.MaxRequestSize and Controller's server's Nginx configuration.
client_max_body_size 128m;
}
+</code></pre>
+</notextile>
-upstream workbench {
- server 127.0.0.1:9000 fail_timeout=10s;
-}
-
-proxy_http_version 1.1;
-
-server {
- listen <span class="userinput">[your public IP address]</span>:443 ssl;
- server_name workbench.<span class="userinput">uuid-prefix.your.domain</span>;
-
- ssl on;
- ssl_certificate <span class="userinput">/YOUR/PATH/TO/cert.pem</span>;
- ssl_certificate_key <span class="userinput">/YOUR/PATH/TO/cert.key</span>;
+h2(#install-packages). Install arvados-workbench
- index index.html index.htm index.php;
- # `client_max_body_size` should match the corresponding setting in
- # the API.MaxRequestSize and Controller's server's Nginx configuration.
- client_max_body_size 128m;
+h3. Centos 7
- location / {
- proxy_pass http://workbench;
- proxy_redirect off;
- proxy_connect_timeout 90s;
- proxy_read_timeout 300s;
-
- proxy_set_header X-Forwarded-Proto https;
- proxy_set_header Host $http_host;
- proxy_set_header X-Real-IP $remote_addr;
- proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
- }
-}
+<notextile>
+<pre><code># <span class="userinput">yum install arvados-workbench</span>
</code></pre>
-</li>
+</notextile>
-<li>Restart Nginx.</li>
+h3. Debian and Ubuntu
-</ol>
+<notextile>
+<pre><code># <span class="userinput">apt-get --no-install-recommends install arvados-workbench</span>
+</code></pre>
</notextile>
-h2. Prepare the Workbench deployment
+h2(#restart-api). Restart the API server and controller
-{% assign railspkg = "arvados-workbench" %}
-{% include 'install_rails_reconfigure' %}
+After adding Workbench to the Services section, make sure the cluster config file is up to date on the API server host, and restart the API server and controller processes to ensure the changes are applied.
-{% include 'notebox_begin' %}
-You can safely ignore the following error message you may see when Ruby Gems are installed:
<notextile>
-<pre><code>themes_for_rails at /usr/local/rvm/gems/ruby-2.1.1/bundler/gems/themes_for_rails-1fd2d7897d75 did not have a valid gemspec.
-This prevents bundler from installing bins or native extensions, but that may not affect its functionality.
-The validation message from Rubygems was:
- duplicate dependency on rails (= 3.0.11, development), (>= 3.0.0) use:
- add_runtime_dependency 'rails', '= 3.0.11', '>= 3.0.0'
-Using themes_for_rails (0.5.1) from https://github.com/holtkampw/themes_for_rails (at 1fd2d78)
+<pre><code># <span class="userinput">systemctl restart nginx arvados-controller</span>
</code></pre>
</notextile>
-{% include 'notebox_end' %}
-h2. Trusted client setting
+h2(#confirm-working). Confirm working installation
+
+Visit @https://workbench.ClusterID.example.com@ in a browser. You should be able to log in using the login method you configured in the previous step. If @Users.AutoAdminFirstUser@ is true, you will be an admin user.
+
+h2(#trusted_client). Trusted client flag
Log in to Workbench once to ensure that the Arvados API server has a record of the Workbench client. (It's OK if Workbench says your account hasn't been activated yet. We'll deal with that next.)
@@ -204,17 +123,3 @@ irb(main):003:0> <span class="userinput">act_as_system_user do wb.update_attr
</code></pre>
</notextile>
-h2(#admin-user). Add an admin user
-
-Next, we're going to use the Rails console on the <strong>API server</strong> to activate your account and give yourself admin privileges. {% include 'install_rails_command' %}
-
-Enter the following commands at the console:
-
-<notextile>
-<pre><code>irb(main):001:0> <span class="userinput">Thread.current[:user] = User.all.select(&:identity_url).last</span>
-irb(main):002:0> <span class="userinput">Thread.current[:user].update_attributes is_admin: true, is_active: true</span>
-irb(main):003:0> <span class="userinput">User.where(is_admin: true).collect &:email</span>
-=> ["root", "<b>your_address at example.com</b>"]
-</code></pre></notextile>
-
-At this point, you should have a working Workbench login with administrator privileges. Revisit your Workbench URL in a browser and reload the page to access it.
diff --git a/doc/install/install-workbench2-app.html.textile.liquid b/doc/install/install-workbench2-app.html.textile.liquid
index b5bdcd42c..566d87878 100644
--- a/doc/install/install-workbench2-app.html.textile.liquid
+++ b/doc/install/install-workbench2-app.html.textile.liquid
@@ -9,43 +9,44 @@ Copyright (C) The Arvados Authors. All rights reserved.
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
+# "Update config.yml":#update-config
+# "Update Nginx configuration":#update-nginx
+# "Install arvados-workbench2":#install-packages
+# "Restart the API server and controller":#restart-api
+# "Confirm working installation":#confirm-working
+# "Trusted client setting":#trusted_client
+
Workbench2 is the web-based user interface for Arvados.
{% include 'notebox_begin' %}
Workbench2 is the replacement for Arvados Workbench. Workbench2 is currently in <i>beta</i>, it is not yet feature complete.
{% include 'notebox_end' %}
-h2(#install_workbench). Install Workbench2 and dependencies
-
-Workbench2 does not require its own database. It is a set of html, javascript and css files that are served as static files from a web server like Nginx or Apache2.
+h2(#configure). Update config.yml
-On a Debian-based system, install the following package:
+Edit @/etc/arvados/config.yml@ to set the keys below. The full set of configuration options are in the "Workbench section of config.yml":{{site.baseurl}}/admin/config.html
<notextile>
-<pre><code>~$ <span class="userinput">sudo apt-get install arvados-workbench2</span>
+<pre><code> Services:
+ Workbench2:
+ ExternalURL: <span class="userinput">"https://workbench2.ClustedID.example.com"</span>
</code></pre>
</notextile>
-On a Red Hat-based system, install the following package:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo yum install arvados-workbench2</span>
-</code></pre>
-</notextile>
+h2. Vocabulary configuration (optional)
-h2. Set up Web server
+Workbench2 can load a vocabulary file which lists available metadata properties for groups and collections. To configure the property vocabulary definition, please visit the "Workbench2 Vocabulary Format":{{site.baseurl}}/admin/workbench2-vocabulary.html page in the Admin section.
-For best performance, we recommend you use Nginx as your Web server to serve Workbench2. Workbench2 consists entirely of static files. To do that:
+h2(#update-nginx). Update Nginx configuration
-<notextile>
-<ol>
-<li>Install Nginx</li>
+Workbench2 does not require its own database. It is a set of html, javascript and css files that are served as static files from Nginx.
-<li><p>Edit the http section of your Nginx configuration to serve Workbench2's files. You might add a block like the following, adding SSL and logging parameters to taste:</p>
+Use a text editor to create a new file @/etc/nginx/conf.d/arvados-workbench2.conf@ with the following configuration. Options that need attention are marked with "TODO".
+<notextile>
<pre><code>server {
listen <span class="userinput">[your public IP address]</span>:443 ssl;
- server_name workbench2.<span class="userinput">uuid-prefix.your.domain</span>;
+ server_name workbench2.<span class="userinput">ClusterID.example.com</span>;
ssl on;
ssl_certificate <span class="userinput">/YOUR/PATH/TO/cert.pem</span>;
@@ -55,7 +56,7 @@ For best performance, we recommend you use Nginx as your Web server to serve Wor
# Workbench2 uses a call to /config.json to bootstrap itself and talk to the desired API server
location /config.json {
- return 200 '{ "API_HOST": "<span class="userinput">uuid-prefix.your.domain</span>" }';
+ return 200 '{ "API_HOST": "<span class="userinput">ClusterID.example.com</span>" }';
}
location / {
@@ -68,30 +69,50 @@ For best performance, we recommend you use Nginx as your Web server to serve Wor
}
}
</code></pre>
-</li>
+</notextile>
-<li>Restart Nginx.</li>
+h2(#install-packages). Install arvados-workbench2
-</ol>
+h3. Centos 7
+
+<notextile>
+<pre><code># <span class="userinput">yum install arvados-workbench2</span>
+</code></pre>
</notextile>
-h2. Trusted client setting
+h3. Debian and Ubuntu
-Log in to Workbench2 once to ensure that the Arvados API server has a record of the Workbench2 client.
+<notextile>
+<pre><code># <span class="userinput">apt-get --no-install-recommends install arvados-workbench2</span>
+</code></pre>
+</notextile>
+
+h2(#restart-api). Restart the API server and controller
+
+After adding Workbench to the Services section, make sure the cluster config file is up to date on the API server host, and restart the API server and controller processes to ensure the changes are applied.
+
+<notextile>
+<pre><code># <span class="userinput">systemctl restart nginx arvados-controller</span>
+</code></pre>
+</notextile>
+
+h2(#confirm-working). Confirm working installation
+
+Visit @https://workbench2.ClusterID.example.com@ in a browser. You should be able to log in using the login method you configured in the previous step. If @Users.AutoAdminFirstUser@ is true, you will be an admin user.
+
+h2(#trusted_client). Trusted client flag
+
+Log in to Workbench once to ensure that the Arvados API server has a record of the Workbench client. (It's OK if Workbench says your account hasn't been activated yet. We'll deal with that next.)
In the <strong>API server</strong> project root, start the Rails console. {% include 'install_rails_command' %}
-At the console, enter the following commands to locate the ApiClient record for your Workbench2 installation (typically, while you're setting this up, the @last@ one in the database is the one you want), then set the @is_trusted@ flag for the appropriate client record:
+At the console, enter the following commands to locate the ApiClient record for your Workbench installation (typically, while you're setting this up, the @last@ one in the database is the one you want), then set the @is_trusted@ flag for the appropriate client record:
<notextile><pre><code>irb(main):001:0> <span class="userinput">wb = ApiClient.all.last; [wb.url_prefix, wb.created_at]</span>
-=> ["https://workbench2.<span class="userinput">uuid_prefix.your.domain</span>/", Sat, 20 Apr 2019 01:23:45 UTC +00:00]
+=> ["https://workbench.example.com/", Sat, 19 Apr 2014 03:35:12 UTC +00:00]
irb(main):002:0> <span class="userinput">include CurrentApiClient</span>
=> true
irb(main):003:0> <span class="userinput">act_as_system_user do wb.update_attributes!(is_trusted: true) end</span>
=> true
</code></pre>
</notextile>
-
-h2. Vocabulary configuration (optional)
-
-To configure the property vocabulary definition, please visit the "Workbench2 Vocabulary Format":{{site.baseurl}}/admin/workbench2-vocabulary.html page in the Admin section.
\ No newline at end of file
diff --git a/doc/install/install-ws.html.textile.liquid b/doc/install/install-ws.html.textile.liquid
index f6a4bb5fa..2a0baa750 100644
--- a/doc/install/install-ws.html.textile.liquid
+++ b/doc/install/install-ws.html.textile.liquid
@@ -11,60 +11,75 @@ SPDX-License-Identifier: CC-BY-SA-3.0
The arvados-ws server provides event notifications to websocket clients. It can be installed anywhere with access to Postgres database and the Arvados API server, typically behind a web proxy that provides SSL support. See the "godoc page":http://godoc.org/github.com/curoverse/arvados/services/ws for additional information.
-By convention, we use the following hostname for the websocket service.
+# "Update config.yml":#update-config
+# "Update nginx configuration":#update-nginx
+# "Install arvados-ws package":#install-packages
+# "Start the service":#start-service
+# "Restart the API server and controller":#restart-api
+# "Confirm working installation":#confirm-working
+
+h2(#configure). Update config.yml
+
+Edit the cluster config at @/etc/arvados/config.yml@ and set @Services.Websocket.ExternalURL@ and @Services.Websocket.InternalURLs at . Replace @zzzzz@ with your cluster id.
<notextile>
-<pre><code>ws.<span class="userinput">uuid_prefix.your.domain</span></code></pre>
+<pre><code> Services:
+ Websocket:
+ InternalURLs:
+ <span class="userinput">"http://ws.ClusterID.example.com:8005"</span>: {}
+ ExternalURL: <span class="userinput">wss://ws.ClusterID.example.com/websocket</span>
+</span></code></pre>
</notextile>
-The above hostname should resolve from anywhere on the internet.
+h2(#update-nginx). Update Nginx configuration
-h2. Install arvados-ws
+The arvados-ws service will be accessible from anywhere on the internet, so we recommend using SSL for transport encryption.
-Typically arvados-ws runs on the same host as the API server.
+Use a text editor to create a new file @/etc/nginx/conf.d/arvados-ws.conf@ with the following configuration. Options that need attention are marked with "TODO".
-On Debian-based systems:
+<notextile><pre>
+upstream arvados-ws {
+ server 127.0.0.1:<span class="userinput">8005</span>;
+}
-<notextile>
-<pre><code>~$ <span class="userinput">sudo apt-get install arvados-ws</span>
-</code></pre>
-</notextile>
+server {
+ listen <span class="userinput">[your public IP address]</span>:443 ssl;
+ server_name ws.<span class="userinput">uuid_prefix.your.domain</span>;
-On Red Hat-based systems:
+ proxy_connect_timeout 90s;
+ proxy_read_timeout 300s;
-<notextile>
-<pre><code>~$ <span class="userinput">sudo yum install arvados-ws</span>
-</code></pre>
-</notextile>
+ ssl on;
+ ssl_certificate <span class="userinput"/>YOUR/PATH/TO/cert.pem</span>;
+ ssl_certificate_key <span class="userinput"/>YOUR/PATH/TO/cert.key</span>;
-Verify that @arvados-ws@ is functional:
+ location / {
+ proxy_pass http://arvados-ws;
+ proxy_set_header Upgrade $http_upgrade;
+ proxy_set_header Connection "upgrade";
+ proxy_set_header Host $host;
+ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+ }
+}
+</pre></notextile>
+
+h2(#install-packages). Install arvados-ws
+
+h3. Centos 7
<notextile>
-<pre><code>~$ <span class="userinput">arvados-ws -h</span>
-Usage of arvados-ws:
- -config path
- path to config file (default "/etc/arvados/config.yml")
- -dump-config
- show current configuration and exit
+<pre><code># <span class="userinput">yum install arvados-ws</span>
</code></pre>
</notextile>
-h3. Update cluster config
-
-Edit the cluster config at @/etc/arvados/config.yml@ and set @Services.Websocket.ExternalURL@ and @Services.Websocket.InternalURLs at . Replace @zzzzz@ with your cluster id.
+h3. Debian and Ubuntu
<notextile>
-<pre><code>Clusters:
- zzzzz:
- Services:
- <span class="userinput">Websocket:
- ExternalURL: wss://ws.uuid_prefix.your.domain/websocket
- InternalURLs:
- "http://localhost:9003": {}
-</span></code></pre>
+<pre><code># <span class="userinput">apt-get --no-install-recommends install arvados-ws</span>
+</code></pre>
</notextile>
-h3. Start the service (option 1: systemd)
+h3. Start the service
If your system does not use systemd, skip this section and follow the "runit instructions":#runit instead.
@@ -98,100 +113,21 @@ Dec 06 11:12:48 zzzzz arvados-ws[8918]: {"error":"pq: password authentication fa
</code></pre>
</notextile>
-Skip ahead to "confirm the service is working":#confirm.
-
-h3(#runit). Start the service (option 2: runit)
+h2(#restart-api). Restart the API server and controller
-Install runit to supervise the arvados-ws daemon. {% include 'install_runit' %}
-
-Create a supervised service.
+After adding the SSO server to the Services section, make sure the cluster config file is up to date on the API server host, and restart the API server and controller processes to ensure the changes are applied.
<notextile>
-<pre><code>~$ <span class="userinput">sudo mkdir /etc/service/arvados-ws</span>
-~$ <span class="userinput">cd /etc/service/arvados-ws</span>
-~$ <span class="userinput">sudo mkdir log log/main</span>
-~$ <span class="userinput">printf '#!/bin/sh\nexec arvados-ws 2>&1\n' | sudo tee run</span>
-~$ <span class="userinput">printf '#!/bin/sh\nexec svlogd main\n' | sudo tee log/run</span>
-~$ <span class="userinput">sudo chmod +x run log/run</span>
-~$ <span class="userinput">sudo sv exit .</span>
-~$ <span class="userinput">cd -</span>
+<pre><code># <span class="userinput">systemctl restart nginx arvados-controller</span>
</code></pre>
</notextile>
-Use @sv stat@ and check the log file to verify the service is running.
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo sv stat /etc/service/arvados-ws</span>
-run: /etc/service/arvados-ws: (pid 12520) 2s; run: log: (pid 12519) 2s
-~$ <span class="userinput">tail /etc/service/arvados-ws/log/main/current</span>
-{"level":"info","msg":"started","time":"2016-12-06T11:56:20.669171449-05:00"}
-{"Listen":":9003","level":"info","msg":"listening","time":"2016-12-06T11:56:20.708847627-05:00"}
-</code></pre>
-</notextile>
-
-h3(#confirm). Confirm the service is working
+h3(#confirm). Confirm working installation
Confirm the service is listening on its assigned port and responding to requests.
<notextile>
-<pre><code>~$ <span class="userinput">curl http://0.0.0.0:<b>9003</b>/status.json</span>
+<pre><code>~$ <span class="userinput">curl http://0.0.0.0:<b>8005</b>/status.json</span>
{"Clients":1}
</code></pre>
</notextile>
-
-h3. Set up a reverse proxy with SSL support
-
-The arvados-ws service will be accessible from anywhere on the internet, so we recommend using SSL for transport encryption.
-
-This is best achieved by putting a reverse proxy with SSL support in front of arvados-ws, running on port 443 and passing requests to arvados-ws on port 9003 (or whatever port you chose in your configuration file).
-
-For example, using Nginx:
-
-<notextile><pre>
-upstream arvados-ws {
- server 127.0.0.1:<span class="userinput">9003</span>;
-}
-
-server {
- listen <span class="userinput">[your public IP address]</span>:443 ssl;
- server_name ws.<span class="userinput">uuid_prefix.your.domain</span>;
-
- proxy_connect_timeout 90s;
- proxy_read_timeout 300s;
-
- ssl on;
- ssl_certificate <span class="userinput"/>YOUR/PATH/TO/cert.pem</span>;
- ssl_certificate_key <span class="userinput"/>YOUR/PATH/TO/cert.key</span>;
-
- location / {
- proxy_pass http://arvados-ws;
- proxy_set_header Upgrade $http_upgrade;
- proxy_set_header Connection "upgrade";
- proxy_set_header Host $host;
- proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
- }
-}
-</pre></notextile>
-
-{% include 'notebox_begin' %}
-If you are upgrading a cluster where Nginx is configured to proxy @ws@ requests to puma, change the @server_name@ value in the old configuration block so it doesn't conflict. When the new configuration is working, delete the old Nginx configuration sections (i.e., the "upstream websockets" block, and the "server" block that references @http://websockets@), and disable/remove the runit or systemd files for the puma server.
-{% include 'notebox_end' %}
-
-h3. Update API server configuration
-
-Restart Nginx to reload the API server configuration.
-
-<notextile>
-<pre><code>$ sudo nginx -s reload</span>
-</code></pre>
-</notextile>
-
-h3. Verify DNS and proxy setup
-
-Use a host elsewhere on the Internet to confirm that your DNS, proxy, and SSL are configured correctly. For @Authorization: Bearer xxxx@ replace @xxxx@ with the value from @ManagementToken@ in @config.yml at .
-
-<notextile>
-<pre><code>$ <span class="userinput">curl -H "Authorization: Bearer xxxx" https://ws.<b>uuid_prefix.your.domain</b>/_health/ping</span>
-{"health":"OK"}
-</code></pre>
-</notextile>
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list