[ARVADOS] updated: 27b534ddd7d2b56ff63cd2ca100fa3119df51a2f
git at public.curoverse.com
git at public.curoverse.com
Fri Oct 31 11:04:17 EDT 2014
Summary of changes:
doc/_config.yml | 11 +-
.../create-standard-objects.html.textile.liquid | 62 +++------
doc/install/index.html.textile.liquid | 21 +--
doc/install/install-api-server.html.textile.liquid | 142 +++++++++++----------
.../install-crunch-dispatch.html.textile.liquid | 8 +-
doc/install/install-keep.html.textile.liquid | 54 --------
doc/install/install-keepproxy.html.textile.liquid | 84 ++++++++++++
doc/install/install-keepstore.html.textile.liquid | 90 +++++++++++++
.../install-manual-overview.html.textile.liquid | 16 +++
...l-manual-prerequisites-ruby.html.textile.liquid | 31 +++++
...nstall-manual-prerequisites.html.textile.liquid | 46 +++++++
.../install-shell-server.html.textile.liquid | 17 +++
doc/install/install-sso.html.textile.liquid | 9 +-
.../install-workbench-app.html.textile.liquid | 106 ++++++++++-----
14 files changed, 466 insertions(+), 231 deletions(-)
delete mode 100644 doc/install/install-keep.html.textile.liquid
create mode 100644 doc/install/install-keepproxy.html.textile.liquid
create mode 100644 doc/install/install-keepstore.html.textile.liquid
create mode 100644 doc/install/install-manual-overview.html.textile.liquid
create mode 100644 doc/install/install-manual-prerequisites-ruby.html.textile.liquid
create mode 100644 doc/install/install-manual-prerequisites.html.textile.liquid
create mode 100644 doc/install/install-shell-server.html.textile.liquid
via 27b534ddd7d2b56ff63cd2ca100fa3119df51a2f (commit)
via 4978c1e740700ec78c26a8046a457496fd9b974c (commit)
via 9ac37f367da0b11971f2ecf8a80a38ae60d43a61 (commit)
via afc70738e8dbbc96e177f1f920b1e8e5f2598fa9 (commit)
via 7c3f2671d43770240a834b9bd5c34ec748acdc1d (commit)
via d14c0c8186517176ecf182cf2555aee8ba4ede6d (commit)
via ed4105d0c6a0d453143849afcea33960afc22117 (commit)
from 28ac1f93305968a913c394dd3d3581c4e290722d (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
commit 27b534ddd7d2b56ff63cd2ca100fa3119df51a2f
Merge: 28ac1f9 4978c1e
Author: Ward Vandewege <ward at curoverse.com>
Date: Fri Oct 31 11:04:03 2014 -0400
Merge branch '4186-install-doc-improvements-2'
refs #4186
commit 4978c1e740700ec78c26a8046a457496fd9b974c
Author: Ward Vandewege <ward at curoverse.com>
Date: Fri Oct 31 11:03:30 2014 -0400
Clean up the installation overview pages a bit more.
refs #4186
diff --git a/doc/install/index.html.textile.liquid b/doc/install/index.html.textile.liquid
index 35805ba..ddbb82e 100644
--- a/doc/install/index.html.textile.liquid
+++ b/doc/install/index.html.textile.liquid
@@ -6,14 +6,6 @@ title: Installation overview
Arvados can be installed in multiple ways. Arvados does not depend on any particular cloud operating stack. Arvados runs on one or more GNU/Linux system(s). Arvados is being developed on Debian and Ubuntu GNU/Linux.
-The simplest way to try out Arvados is to use the Docker-based installation, which installs Arvados in a series of Docker containers.
+The simplest way to try out Arvados is to use the "Docker-based installation":install-docker.html, which installs Arvados in a series of Docker containers.
-For larger scale installations, a manual installation is more appropriate.
-
-h2. Docker
-
-"Installing with Docker":install-docker.html
-
-h2. Manual installation
-
-"Manual Installation":install-manual-overview.html
+For production use or evaluation at scale, a "Manual Installation":install-manual-overview.html is more appropriate.
diff --git a/doc/install/install-manual-overview.html.textile.liquid b/doc/install/install-manual-overview.html.textile.liquid
index 363700a..1ba9451 100644
--- a/doc/install/install-manual-overview.html.textile.liquid
+++ b/doc/install/install-manual-overview.html.textile.liquid
@@ -4,7 +4,9 @@ navsection: installguide
title: Overview
...
-The manual installation guide will walk you through setting up a basic Arvados cluster on a number of (virtual) GNU/Linux systems.
+{% include 'alert_stub' %}
+
+The manual installation guide will walk you through setting up a basic Arvados cluster on a number of (virtual) GNU/Linux systems. This installation method is intended for evaluation or production use at scale.
<div class="alert alert-block alert-info">
<button type="button" class="close" data-dismiss="alert">×</button>
@@ -12,16 +14,3 @@ The manual installation guide will walk you through setting up a basic Arvados c
<p>If you are looking to evaluate Arvados on one machine, we recommend the "Docker installation method":install-docker.html instead.</p>
</div>
-h2. Manual installation
-
-# "Install the Arvados REST API server":install-api-server.html
-# "Create standard objects":create-standard-objects.html
-# "Install the Single Sign On (SSO) server":install-sso.html
-# "Install the Arvados websockets server":install-websockets-server.html
-# "Install the Arvados workbench application":install-workbench-app.html
-# "Install Keepstore":install-keep.html
-# "Install Keepproxy":install-keepproxy.html
-# "Install the Crunch dispatcher":install-crunch-dispatch.html
-# "Install a shell server":install-shell-server.html
-# "Install a compute node":install-compute-node.html
-# Install client libraries (see "SDK Reference":{{site.baseurl}}/sdk/index.html).
commit 9ac37f367da0b11971f2ecf8a80a38ae60d43a61
Merge: afc7073 28ac1f9
Author: Ward Vandewege <ward at curoverse.com>
Date: Fri Oct 31 10:52:24 2014 -0400
Merge branch 'master' into 4186-install-doc-improvements-2
commit afc70738e8dbbc96e177f1f920b1e8e5f2598fa9
Author: Ward Vandewege <ward at curoverse.com>
Date: Fri Oct 31 10:47:45 2014 -0400
Add keepstore installation page.
refs #4186
diff --git a/doc/install/install-keepstore.html.textile.liquid b/doc/install/install-keepstore.html.textile.liquid
new file mode 100644
index 0000000..0c684ea
--- /dev/null
+++ b/doc/install/install-keepstore.html.textile.liquid
@@ -0,0 +1,90 @@
+---
+layout: default
+navsection: installguide
+title: Install Keepstore servers
+...
+
+This installation guide assumes you are on a 64 bit Debian or Ubuntu system.
+
+We are going to install two Keepstore servers. By convention, we use the following hostname pattern:
+
+<div class="offset1">
+table(table table-bordered table-condensed).
+|_Hostname_|
+|keep0. at uuid_prefix@.your.domain|
+|keep1. at uuid_prefix@.your.domain|
+</div>
+
+Because the Keepstore servers are not directly accessible from the internet, these hostnames only need to resolve on the local network.
+
+h2. Install Keepstore
+
+First add the Arvados apt repository, and then install the Keepstore package.
+
+<notextile>
+<pre><code>~$ <span class="userinput">echo "# apt.arvados.org" > /etc/apt/sources.list.d/apt.arvados.org.list</span>
+~$ <span class="userinput">echo "deb http://apt.arvados.org/ wheezy main" >> /etc/apt/sources.list.d/apt.arvados.org.list</span>
+~$ <span class="userinput">/usr/bin/apt-key adv --keyserver pool.sks-keyservers.net --recv 1078ECD7</span>
+~$ <span class="userinput">/usr/bin/apt-get update</span>
+~$ <span class="userinput">/usr/bin/apt-get install keepstore</span>
+</code></pre>
+</notextile>
+
+Verify that Keepstore is functional:
+
+<notextile>
+<pre><code>~$ <span class="userinput">keepstore -h</span>
+2014/10/29 14:23:38 Keep started: pid 6848
+Usage of keepstore:
+ -data-manager-token-file="": File with the API token used by the Data Manager. All DELETE requests or GET /index requests must carry this token.
+ -enforce-permissions=false: Enforce permission signatures on requests.
+ -listen=":25107": Interface on which to listen for requests, in the format ipaddr:port. e.g. -listen=10.0.1.24:8000. Use -listen=:port to listen on all network interfaces.
+ -never-delete=false: If set, nothing will be deleted. HTTP 405 will be returned for valid DELETE requests.
+ -permission-key-file="": File containing the secret key for generating and verifying permission signatures.
+ -permission-ttl=1209600: Expiration time (in seconds) for newly generated permission signatures.
+ -pid="": Path to write pid file
+ -serialize=false: If set, all read and write operations on local Keep volumes will be serialized.
+ -volumes="": Comma-separated list of directories to use for Keep volumes, e.g. -volumes=/var/keep1,/var/keep2. If empty or not supplied, Keep will scan mounted filesystems for volumes with a /keep top-level directory.
+</code></pre>
+</notextile>
+
+If you want access control on your Keepstore server(s), you should provide a permission key. The @-permission-key-file@ argument should contain the path to a file that contains a single line with a long random alphanumeric string. It should be the same as the @blob_signing_key@ that can be set in the "API server":install-api-server.html config/application.yml file.
+
+Prepare one or more volumes for Keepstore to use. Simply create a /keep directory on all the partitions you would like Keepstore to use, and then start Keepstore. For example, using 2 tmpfs volumes:
+
+<notextile>
+<pre><code>~$ <span class="userinput">keepstore</span>
+2014/10/29 11:41:37 Keep started: pid 20736
+2014/10/29 11:41:37 adding Keep volume: /tmp/tmp.vwSCtUCyeH/keep
+2014/10/29 11:41:37 adding Keep volume: /tmp/tmp.Lsn4w8N3Xv/keep
+2014/10/29 11:41:37 Running without a PermissionSecret. Block locators returned by this server will not be signed, and will be rejected by a server that enforces permissions.
+2014/10/29 11:41:37 To fix this, run Keep with --permission-key-file=<path> to define the location of a file containing the permission key.
+
+</code></pre>
+</notextile>
+
+It's recommended to run Keepstore under "runit":https://packages.debian.org/search?keywords=runit or something similar.
+
+Repeat this section for each Keepstore server you are setting up.
+
+h3. Tell the API server about the Keepstore servers
+
+The API server needs to be informed about the presence of your Keepstore servers. For each of the Keepstore servers you have created, please execute the following commands on your <strong>shell server</strong>.
+
+Make sure to update the @service_host@ value to match each of your Keepstore servers.
+
+<notextile>
+<pre><code>~$ <span class="userinput">prefix=`arv --format=uuid user current | cut -d- -f1`</span>
+~$ <span class="userinput">echo "Site prefix is '$prefix'"</span>
+~$ <span class="userinput">read -rd $'\000' keepservice <<EOF; arv keep_service create --keep-service "$keepservice"</span>
+<span class="userinput">{
+ "service_host":"keep0.$prefix.your.domain",
+ "service_port":25107,
+ "service_ssl_flag":false,
+ "service_type":"disk"
+}
+EOF</span>
+</code></pre></notextile>
+
+
+
commit 7c3f2671d43770240a834b9bd5c34ec748acdc1d
Author: Ward Vandewege <ward at curoverse.com>
Date: Thu Oct 30 22:07:11 2014 -0400
Fix typo.
refs #4186
diff --git a/doc/install/install-manual-prerequisites.html.textile.liquid b/doc/install/install-manual-prerequisites.html.textile.liquid
index 06de1dd..e5b28d9 100644
--- a/doc/install/install-manual-prerequisites.html.textile.liquid
+++ b/doc/install/install-manual-prerequisites.html.textile.liquid
@@ -19,7 +19,7 @@ table(table table-bordered table-condensed).
|Arvados compute node|1|
</div>
-The number of Keepstore, shell and compute nodes listed above is a minimum. In a real production installation, you will likely run many more of each of those types of nodes. In such a scenario, you would probably also want to dedicate a node to the Workbenbench server and crunch dispatcher, respectively. For performance reasons, you may want to run the database server on a separate node as well.
+The number of Keepstore, shell and compute nodes listed above is a minimum. In a real production installation, you will likely run many more of each of those types of nodes. In such a scenario, you would probably also want to dedicate a node to the Workbench server and Crunch dispatcher, respectively. For performance reasons, you may want to run the database server on a separate node as well.
h2. A unique identifier
commit d14c0c8186517176ecf182cf2555aee8ba4ede6d
Author: Ward Vandewege <ward at curoverse.com>
Date: Wed Oct 29 12:27:45 2014 -0400
Keepproxy just needs an anonymous token.
refs #4186
diff --git a/doc/install/install-keepproxy.html.textile.liquid b/doc/install/install-keepproxy.html.textile.liquid
index 6725898..646b643 100644
--- a/doc/install/install-keepproxy.html.textile.liquid
+++ b/doc/install/install-keepproxy.html.textile.liquid
@@ -48,33 +48,13 @@ It's recommended to run Keepproxy under "runit":https://packages.debian.org/sear
h3. Create an API token for the Keepproxy server
-The Keepproxy server needs a token to talk to the API server. The token can be associated with the root user. On the <strong>shell server</strong>, use the following command to create the token.
+The Keepproxy server needs a token to talk to the API server.
+
+On the <strong>API server</strong>, use the following command to create the token:
<notextile>
-<pre><code>~$ <span class="userinput">arv api_client_authorization create_system_auth --scopes "[]"</span>
-{
- "href":"/api_client_authorizations/oethieWeKohy4aesahv2moh0Dapheigh9aeNo3uSahg6yaihui",
- "kind":"arvados#apiClientAuthorization",
- "etag":"ieLohYieh5joo3ahxaileChoo",
- "uuid":"oethieWeKohy4aesahv2moh0Dapheigh9aeNo3uSahg6yaihui",
- "owner_uuid":"uuid_prefix-tpzed-000000000000000",
- "created_at":"2014-10-29T15:01:57Z",
- "modified_by_client_uuid":null,
- "modified_by_user_uuid":null,
- "modified_at":null,
- "user_id":1,
- "api_client_id":0,
- "api_token":"fiekieth2luaWe0feePh7yoo6MaifahChaet4ulaitoothais9",
- "created_by_ip_address":"10.1.1.1",
- "default_owner_uuid":null,
- "expires_at":null,
- "last_used_at":null,
- "last_used_by_ip_address":null,
- "scopes":[],
- "_profile":{
- "request_time":0.037659336
- }
-}
+<pre><code>~/arvados/services/api/script$ <span class="userinput">RAILS_ENV=production ./get_anonymous_user_token.rb</span>
+hoShoomoo2bai3Ju1xahg6aeng1siquuaZ1yae2gi2Uhaeng2r
</code></pre></notextile>
The value for the @api_token@ field should be added to Keepproxy's environment as ARVADOS_API_TOKEN. Make sure to also set ARVADOS_API_HOST to @uuid_prefix at .your.domain.
commit ed4105d0c6a0d453143849afcea33960afc22117
Author: Ward Vandewege <ward at curoverse.com>
Date: Wed Oct 29 11:40:12 2014 -0400
Batch of improvements for the manual installation documentation:
* add prerequisites section
* add shell server section
* add keepproxy section
* many other updates throughout
refs #4186
diff --git a/doc/_config.yml b/doc/_config.yml
index 3b31cb0..aa64748 100644
--- a/doc/_config.yml
+++ b/doc/_config.yml
@@ -128,9 +128,16 @@ navbar:
- Docker:
- install/install-docker.html.textile.liquid
- Manual installation:
- - install/install-keep.html.textile.liquid
- - install/install-sso.html.textile.liquid
+ - install/install-manual-overview.html.textile.liquid
+ - install/install-manual-prerequisites.html.textile.liquid
- install/install-api-server.html.textile.liquid
- install/install-workbench-app.html.textile.liquid
+ - install/install-shell-server.html.textile.liquid
- install/create-standard-objects.html.textile.liquid
+ - install/install-keepstore.html.textile.liquid
+ - install/install-keepproxy.html.textile.liquid
- install/install-crunch-dispatch.html.textile.liquid
+ - install/install-compute-node.html.textile.liquid
+ - Software prerequisites:
+ - install/install-manual-prerequisites-ruby.html.textile.liquid
+ - install/install-sso.html.textile.liquid
diff --git a/doc/install/create-standard-objects.html.textile.liquid b/doc/install/create-standard-objects.html.textile.liquid
index d6a091a..92b0ade 100644
--- a/doc/install/create-standard-objects.html.textile.liquid
+++ b/doc/install/create-standard-objects.html.textile.liquid
@@ -6,66 +6,40 @@ title: Create standard objects
...
+Next, we're going to use the Arvados CLI tools on the <strong>shell server</strong> to create some standard objects.
h3. "All users" group
The convention is to add every active user to this group. We give it a distinctive UUID that looks like an IP broadcast address.
-<pre>
-prefix=`arv --format=uuid user current | cut -d- -f1`
-
-echo "Site prefix is '$prefix'"
-# (Make sure it matches your configured 5-character site prefix.)
-
-read -rd $'\000' newgroup <<EOF; arv group create --group "$newgroup"
-{
+<notextile>
+<pre><code>~$ <span class="userinput">prefix=`arv --format=uuid user current | cut -d- -f1`</span>
+~$ <span class="userinput">echo "Site prefix is '$prefix'"</span>
+~$ <span class="userinput">read -rd $'\000' newgroup <<EOF; arv group create --group "$newgroup"</span>
+<span class="userinput">{
"uuid":"$prefix-j7d0g-fffffffffffffff",
"name":"All users"
-}
+}</span>
EOF
-</pre>
+</code></pre></notextile>
h3. "arvados" repository
This will be readable by the "All users" group, and therefore by every active user. This makes it possible for users to run the bundled Crunch scripts by specifying @"script_version":"master","repository":"arvados"@ rather than pulling the Arvados source tree into their own repositories.
-<pre>
-prefix=`arv --format=uuid user current | cut -d- -f1`
-
-echo "Site prefix is '$prefix'"
-# (Make sure it matches your configured 5-character site prefix.)
-
-all_users_group_uuid="$prefix-j7d0g-fffffffffffffff"
-repo_uuid=`arv --format=uuid repository create --repository '{"name":"arvados"}'`
-echo "Arvados repository uuid is '$repo_uuid'"
-
-read -rd $'\000' newlink <<EOF; arv link create --link "$newlink"
-{
+<notextile>
+<pre><code>~$ <span class="userinput">prefix=`arv --format=uuid user current | cut -d- -f1`</span>
+~$ <span class="userinput">echo "Site prefix is '$prefix'"</span>
+~$ <span class="userinput">all_users_group_uuid="$prefix-j7d0g-fffffffffffffff"</span>
+~$ <span class="userinput">repo_uuid=`arv --format=uuid repository create --repository '{"name":"arvados"}'`</span>
+~$ <span class="userinput">echo "Arvados repository uuid is '$repo_uuid'"</span>
+~$ <span class="userinput">read -rd $'\000' newlink <<EOF; arv link create --link "$newlink"</span>
+<span class="userinput">{
"tail_uuid":"$all_users_group_uuid",
"head_uuid":"$repo_uuid",
"link_class":"permission",
"name":"can_read"
}
-EOF
-</pre>
+EOF</span>
+</code></pre></notextile>
-h3. Keep disks
-
-Currently, you need to tell Arvados about Keep services manually. You'll need at least two "disk" services.
-
-Example:
-
-<pre>
-prefix=`arv --format=uuid user current | cut -d- -f1`
-echo "Site prefix is '$prefix'"
-# (Make sure it matches your configured 5-character site prefix.)
-
-read -rd $'\000' keepservice <<EOF; arv keep_service create --keep-service "$keepservice"
-{
- "service_host":"keep0.$prefix.arvadosapi.com",
- "service_port":25107,
- "service_ssl_flag":false,
- "service_type":"disk"
-}
-EOF
-</pre>
diff --git a/doc/install/index.html.textile.liquid b/doc/install/index.html.textile.liquid
index 7cb0fea..35805ba 100644
--- a/doc/install/index.html.textile.liquid
+++ b/doc/install/index.html.textile.liquid
@@ -16,13 +16,4 @@ h2. Docker
h2. Manual installation
-{% include 'alert_stub' %}
-
-# Set up a cluster, or use Amazon
-# "Install Keep":install-keep.html
-# "Install the Single Sign On (SSO) server":install-sso.html
-# "Install the Arvados REST API server":install-api-server.html
-# "Install the Arvados workbench application":install-workbench-app.html
-# "Install the Crunch dispatcher":install-crunch-dispatch.html
-# "Create standard objects":create-standard-objects.html
-# Install client libraries (see "SDK Reference":{{site.baseurl}}/sdk/index.html).
+"Manual Installation":install-manual-overview.html
diff --git a/doc/install/install-api-server.html.textile.liquid b/doc/install/install-api-server.html.textile.liquid
index e1de8c3..f29b2cf 100644
--- a/doc/install/install-api-server.html.textile.liquid
+++ b/doc/install/install-api-server.html.textile.liquid
@@ -4,37 +4,18 @@ navsection: installguide
title: Install the API server
...
-h2. Prerequisites:
+This installation guide assumes you are on a 64 bit Debian or Ubuntu system.
-# A GNU/Linux (virtual) machine
-# A domain name for your api server
-
-h2(#dependencies). Install dependencies
+h2. Install prerequisites
<notextile>
<pre><code>~$ <span class="userinput">sudo apt-get install \
bison build-essential gettext libcurl3 libcurl3-gnutls \
libcurl4-openssl-dev libpcre3-dev libpq-dev libreadline-dev \
- libsqlite3-dev libssl-dev libxslt1.1 postgresql sqlite3 sudo \
- wget zlib1g-dev
+ libssl-dev libxslt1.1 postgresql sudo wget zlib1g-dev
</span></code></pre></notextile>
-h2(#ruby). Install Ruby and bundler
-
-We recommend Ruby >= 2.1.
-
-<notextile>
-<pre><code><span class="userinput">mkdir -p ~/src
-cd ~/src
-wget http://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.2.tar.gz
-tar xzf ruby-2.1.2.tar.gz
-cd ruby-2.1.2
-./configure
-make
-sudo make install
-
-sudo gem install bundler</span>
-</code></pre></notextile>
+Also make sure you have "Ruby and bundler":install-manual-prerequisites-ruby.html installed.
h2. Download the source tree
@@ -45,6 +26,8 @@ h2. Download the source tree
See also: "Downloading the source code":https://arvados.org/projects/arvados/wiki/Download on the Arvados wiki.
+The API server is in @services/api@ in the source tree.
+
h2. Install gem dependencies
<notextile>
@@ -52,25 +35,45 @@ h2. Install gem dependencies
~/arvados/services/api$ <span class="userinput">bundle install</span>
</code></pre></notextile>
+h2. Choose your environment
+
+The API server can be run in @development@ or in @production@ mode. Unless this installation is going to be used for development on the Arvados API server itself, you should run it in @production@ mode.
+
+Copy the example environment file for your environment. For example, if you choose @production@:
+
+<notextile>
+<pre><code>~/arvados/services/api$ <span class="userinput">cp -i config/environments/production.rb.example config/environments/production.rb</span>
+</code></pre></notextile>
+
h2. Configure the API server
-Edit the main configuration:
+First, copy the example configuration file:
<notextile>
<pre><code>~/arvados/services/api$ <span class="userinput">cp -i config/application.yml.example config/application.yml</span>
</code></pre></notextile>
-Choose a unique 5-character alphanumeric string to use as your @uuid_prefix at . An example is given that generates a 5-character string based on a hash of your hostname. The @uuid_prefix@ is a unique identifier for your API server. It also serves as the first part of the hostname for your API server.
+The API server reads the @config/application.yml@ file, as well as the @config/application.defaults.yml@ file. Values in @config/application.yml@ take precedence over the defaults that are defined in @config/application.defaults.yml at . The @config/application.yml.example@ file is not read by the API server and is provided for installation convenience, only.
-For a development site, use your own domain instead of arvadosapi.com.
+Consult @config/application.default.yml@ for a full list of configuration options. Always put your local configuration in @config/application.yml@, never edit @config/application.default.yml at .
-Make sure a clone of the arvados repository exists in @git_repositories_dir@:
+h3(#uuid_prefix). uuid_prefix
+
+It is recommended to explicitly define your @uuid_prefix@ in @config/application.yml@, by setting the 'uuid_prefix' field in the section for your environment.
+
+h3(#git_repositories_dir). git_repositories_dir
+
+This field defaults to @/var/lib/arvados/git at . You can override the value by defining it in @config/application.yml at .
+
+Make sure a clone of the arvados repository exists in @git_repositories_dir at .
<notextile>
-<pre><code>~/arvados/services/api$ <span class="userinput">sudo mkdir -p /var/cache/git</span>
-~/arvados/services/api$ <span class="userinput">sudo git clone --bare ../../.git /var/cache/git/arvados.git</span>
+<pre><code>~/arvados/services/api$ <span class="userinput">sudo mkdir -p /var/lib/arvados/git</span>
+~/arvados/services/api$ <span class="userinput">sudo git clone --bare ../../.git /var/lib/arvados/git/arvados.git</span>
</code></pre></notextile>
+h3. secret_token
+
Generate a new secret token for signing cookies:
<notextile>
@@ -78,17 +81,24 @@ Generate a new secret token for signing cookies:
zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz
</code></pre></notextile>
-If you want access control on your Keep server(s), you should set @blob_signing_key@ to the same value as the permission key you provided to your "Keep server(s)":install-keep.html.
+Then put that value in the @secret_token@ field.
-Put it in @config/application.yml@ in the production or common section:
+h3. blob_signing_key
-<notextile>
-<pre><code><span class="userinput"> secret_token: zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz</span>
-</code></pre>
-</notextile>
+If you want access control on your "Keep":install-keep.html server(s), you should set @blob_signing_key@ to the same value as the permission key you provide to your Keepstore daemon(s).
+
+h3. workbench_address
+
+Fill in the url of your workbench application in in @workbench_address@, for example
+
+ https://workbench.@prefix_uuid@.your.domain
+
+h3. other options
Consult @application.default.yml@ for a full list of configuration options. Always put your local configuration in @application.yml@ instead of editing @application.default.yml at .
+h2. Set up the database
+
Generate a new database password. Nobody ever needs to memorize it or type it, so we'll make a strong one:
<notextile>
@@ -114,19 +124,37 @@ Configure API server to connect to your database by creating and updating @confi
~/arvados/services/api$ <span class="userinput">edit config/database.yml</span>
</code></pre></notextile>
-Create and initialize the database.
+Create and initialize the database. If you are planning a production system, choose the @production@ rails environment, otherwise use @development at .
<notextile>
-<pre><code>~/arvados/services/api$ <span class="userinput">RAILS_ENV=development bundle exec rake db:setup</span>
+<pre><code>~/arvados/services/api$ <span class="userinput">RAILS_ENV=production bundle exec rake db:setup</span>
</code></pre></notextile>
-Set up omniauth:
+Alternatively, if the database user you intend to use for the API server is not allowed to create new databases, you can create the database first and then populate it with rake. Be sure to adjust the database name if you are using the @development@ environment. This sequence of commands is functionally equivalent to the rake db:setup command above.
+
+<notextile>
+<pre><code>~/arvados/services/api$ <span class="userinput">su postgres createdb arvados_production -E UTF8 -O arvados</span>
+~/arvados/services/api$ <span class="userinput">RAILS_ENV=production bundle exec rake db:structure:load</span>
+~/arvados/services/api$ <span class="userinput">RAILS_ENV=production bundle exec rake db:seed</span>
+</code></pre></notextile>
+
+<div class="alert alert-block alert-info">
+ <button type="button" class="close" data-dismiss="alert">×</button>
+ <h4>Note!</h4>
+You can safely ignore the following error message you may see when loading the database structure:
+<notextile>
+<pre><code>ERROR: must be owner of extension plpgsql</code></pre></notextile>
+</div>
+
+h2. Set up omniauth
+
+First copy the omniauth configuration file:
<notextile>
<pre><code>~/arvados/services/api$ <span class="userinput">cp -i config/initializers/omniauth.rb.example config/initializers/omniauth.rb
</code></pre></notextile>
-Edit @config/initializers/omniauth.rb@, and tell your api server to use the Curoverse SSO server for authentication:
+Edit @config/initializers/omniauth.rb@, and tell your api server to use the Curoverse SSO server for authentication. Use the @APP_SECRET@ specified in the snippet below.
<notextile>
<pre><code>APP_ID = 'local_docker_installation'
@@ -141,41 +169,25 @@ CUSTOM_PROVIDER_URL = 'https://auth.curoverse.com'
<p>You can also run your own SSO server. However, the SSO server codebase currently uses OpenID 2.0 to talk to Google's authentication service. Google <a href="https://developers.google.com/accounts/docs/OpenID2">has deprecated that protocol</a>. This means that new clients will not be allowed to talk to Google's authentication services anymore over OpenID 2.0, and they will phase out the use of OpenID 2.0 completely in the coming monts. We are working on upgrading the SSO server codebase to a newer protocol. That work should be complete by the end of November 2014. In the mean time, anyone is free to use the existing Curoverse SSO server for any local Arvados installation.</p>
</div>
-You can now run the development server:
+h2. Start the API server
+
+h3. Development environment
+
+If you plan to run in development mode, you can now run the development server this way:
<notextile>
<pre><code>~/arvados/services/api$ <span class="userinput">bundle exec rails server --port=3030
</code></pre></notextile>
-h3. Apache/Passenger (optional)
+h3. Production environment
-You can use "Passenger":https://www.phusionpassenger.com/ for deployment. Point it to the services/api directory in the source tree.
+We recommend "Passenger":https://www.phusionpassenger.com/ to run the API server in production.
-To enable streaming so users can monitor crunch jobs in real time, add to your Passenger configuration in Apache:
+Point it to the services/api directory in the source tree.
-<notextile>
-<pre><code><span class="userinput">PassengerBufferResponse off</span>
-</code></pre>
-</notextile>
-
-h2(#admin-user). Add an admin user
-
-Point your browser to the API server's login endpoint:
+To enable streaming so users can monitor crunch jobs in real time, make sure to add the following to your Passenger configuration:
<notextile>
-<pre><code><span class="userinput">https://localhost:3030/login</span>
+<pre><code><span class="userinput">PassengerBufferResponse off</span>
</code></pre>
</notextile>
-
-Log in with your google account.
-
-Use the rails console to give yourself admin privileges:
-
-<notextile>
-<pre><code>~/arvados/services/api$ <span class="userinput">bundle exec rails console</span>
-irb(main):001:0> <span class="userinput">Thread.current[:user] = User.all.select(&:identity_url).last</span>
-irb(main):002:0> <span class="userinput">Thread.current[:user].is_admin = true</span>
-irb(main):003:0> <span class="userinput">Thread.current[:user].update_attributes is_admin: true, is_active: true</span>
-irb(main):004:0> <span class="userinput">User.where(is_admin: true).collect &:email</span>
-=> ["root", "<b>your_address at example.com</b>"]
-</code></pre></notextile>
diff --git a/doc/install/install-crunch-dispatch.html.textile.liquid b/doc/install/install-crunch-dispatch.html.textile.liquid
index d0f4414..231d1f4 100644
--- a/doc/install/install-crunch-dispatch.html.textile.liquid
+++ b/doc/install/install-crunch-dispatch.html.textile.liquid
@@ -27,12 +27,6 @@ On compute nodes:
* @pip install --upgrade pyvcf@
-h4. Redis
-
-On controller:
-
-* @apt-get install redis-server@
-
h4. Crunch user account
On compute nodes and controller:
@@ -43,7 +37,7 @@ The crunch user should have the same UID, GID, and home directory on all compute
h4. Repositories
-Crunch scripts must be in Git repositories in @/var/lib/arvados/git/*.git@ (or whatever is configured in @services/api/config/environments/production.rb@).
+Crunch scripts must be in Git repositories in the directory configured as @git_repositories_dir@/*.git (see the "API server installation":install-api-server.html#git_repositories_dir).
Once you have a repository with commits -- and you have read access to the repository -- you should be able to create a new job:
diff --git a/doc/install/install-keep.html.textile.liquid b/doc/install/install-keep.html.textile.liquid
deleted file mode 100644
index 20670f3..0000000
--- a/doc/install/install-keep.html.textile.liquid
+++ /dev/null
@@ -1,54 +0,0 @@
----
-layout: default
-navsection: installguide
-title: Install Keep
-...
-
-This installation guide assumes you are on a 64 bit Debian or Ubuntu system.
-
-First add the Arvados apt repository, and then install the Keep package.
-
-<notextile>
-<pre><code>~$ <span class="userinput">echo "# apt.arvados.org" > /etc/apt/sources.list.d/apt.arvados.org.list</span>
-~$ <span class="userinput">echo "deb http://apt.arvados.org/ wheezy main" >> /etc/apt/sources.list.d/apt.arvados.org.list</span>
-~$ <span class="userinput">/usr/bin/apt-key adv --keyserver pgp.mit.edu --recv 1078ECD7</span>
-~$ <span class="userinput">/usr/bin/apt-get update</span>
-~$ <span class="userinput">/usr/bin/apt-get install keepstore</span>
-</code></pre>
-</notextile>
-
-Verify that Keep is functional:
-
-<notextile>
-<pre><code>~$ <span class="userinput">keepstore -h</span>
-2014/07/24 15:38:27 Keep started: pid 13606
-Usage of keepstore:
- -data-manager-token-file="": File with the API token used by the Data Manager. All DELETE requests or GET /index requests must carry this token.
- -enforce-permissions=false: Enforce permission signatures on requests.
- -listen=":25107": Interface on which to listen for requests, in the format ipaddr:port. e.g. -listen=10.0.1.24:8000. Use -listen=:port to listen on all network interfaces.
- -never-delete=false: If set, nothing will be deleted. HTTP 405 will be returned for valid DELETE requests.
- -permission-key-file="": File containing the secret key for generating and verifying permission signatures.
- -permission-ttl=1209600: Expiration time (in seconds) for newly generated permission signatures.
- -pid="": Path to write pid file
- -serialize=false: If set, all read and write operations on local Keep volumes will be serialized.
- -volumes="": Comma-separated list of directories to use for Keep volumes, e.g. -volumes=/var/keep1,/var/keep2. If empty or not supplied, Keep will scan mounted filesystems for volumes with a /keep top-level directory.
-</code></pre>
-</notextile>
-
-If you want access control on your Keep server(s), you should provide a permission key. The @-permission-key-file@ argument should contain the path to a file that contains a single line with a long random alphanumeric string. It should be the same as the @blob_signing_key@ that can be set in the "API server":install-api-server.html config/application.yml file.
-
-Prepare one or more volumes for Keep to use. Simply create a /keep directory on all the partitions you would like Keep to use, and then start Keep. For example, using 2 tmpfs volumes:
-
-<notextile>
-<pre><code>~$ <span class="userinput">keepstore</span>
-2014/07/24 11:41:37 Keep started: pid 20736
-2014/07/24 11:41:37 adding Keep volume: /tmp/tmp.vwSCtUCyeH/keep
-2014/07/24 11:41:37 adding Keep volume: /tmp/tmp.Lsn4w8N3Xv/keep
-2014/07/24 11:41:37 Running without a PermissionSecret. Block locators returned by this server will not be signed, and will be rejected by a server that enforces permissions.
-2014/07/24 11:41:37 To fix this, run Keep with --permission-key-file=<path> to define the location of a file containing the permission key.
-
-</code></pre>
-</notextile>
-
-It's recommended to run Keep under "runit":https://packages.debian.org/search?keywords=runit or something similar.
-
diff --git a/doc/install/install-keepproxy.html.textile.liquid b/doc/install/install-keepproxy.html.textile.liquid
new file mode 100644
index 0000000..6725898
--- /dev/null
+++ b/doc/install/install-keepproxy.html.textile.liquid
@@ -0,0 +1,104 @@
+---
+layout: default
+navsection: installguide
+title: Install Keepproxy server
+...
+
+This installation guide assumes you are on a 64 bit Debian or Ubuntu system.
+
+The Keepproxy server is a gateway into your Keep storage. Unlike the Keepstore servers, which are only accessible on the local LAN, Keepproxy is designed to provide secure access into Keep from anywhere on the internet.
+
+By convention, we use the following hostname for the Keepproxy:
+
+<div class="offset1">
+table(table table-bordered table-condensed).
+|_Hostname_|
+|keep. at uuid_prefix@.your.domain|
+</div>
+
+This hostname should resolve from anywhere on the internet.
+
+h2. Install Keepproxy
+
+First add the Arvados apt repository, and then install the Keepproxy package.
+
+<notextile>
+<pre><code>~$ <span class="userinput">echo "# apt.arvados.org" > /etc/apt/sources.list.d/apt.arvados.org.list</span>
+~$ <span class="userinput">echo "deb http://apt.arvados.org/ wheezy main" >> /etc/apt/sources.list.d/apt.arvados.org.list</span>
+~$ <span class="userinput">/usr/bin/apt-key adv --keyserver pool.sks-keyservers.net --recv 1078ECD7</span>
+~$ <span class="userinput">/usr/bin/apt-get update</span>
+~$ <span class="userinput">/usr/bin/apt-get install keepproxy</span>
+</code></pre>
+</notextile>
+
+Verify that Keepproxy is functional:
+
+<notextile>
+<pre><code>~$ <span class="userinput">keepproxy -h</span>
+Usage of default:
+ -default-replicas=2: Default number of replicas to write if not specified by the client.
+ -listen=":25107": Interface on which to listen for requests, in the format ipaddr:port. e.g. -listen=10.0.1.24:8000. Use -listen=:port to listen on all network interfaces.
+ -no-get=false: If set, disable GET operations
+ -no-put=false: If set, disable PUT operations
+ -pid="": Path to write pid file
+</code></pre>
+</notextile>
+
+It's recommended to run Keepproxy under "runit":https://packages.debian.org/search?keywords=runit or something similar.
+
+h3. Create an API token for the Keepproxy server
+
+The Keepproxy server needs a token to talk to the API server. The token can be associated with the root user. On the <strong>shell server</strong>, use the following command to create the token.
+
+<notextile>
+<pre><code>~$ <span class="userinput">arv api_client_authorization create_system_auth --scopes "[]"</span>
+{
+ "href":"/api_client_authorizations/oethieWeKohy4aesahv2moh0Dapheigh9aeNo3uSahg6yaihui",
+ "kind":"arvados#apiClientAuthorization",
+ "etag":"ieLohYieh5joo3ahxaileChoo",
+ "uuid":"oethieWeKohy4aesahv2moh0Dapheigh9aeNo3uSahg6yaihui",
+ "owner_uuid":"uuid_prefix-tpzed-000000000000000",
+ "created_at":"2014-10-29T15:01:57Z",
+ "modified_by_client_uuid":null,
+ "modified_by_user_uuid":null,
+ "modified_at":null,
+ "user_id":1,
+ "api_client_id":0,
+ "api_token":"fiekieth2luaWe0feePh7yoo6MaifahChaet4ulaitoothais9",
+ "created_by_ip_address":"10.1.1.1",
+ "default_owner_uuid":null,
+ "expires_at":null,
+ "last_used_at":null,
+ "last_used_by_ip_address":null,
+ "scopes":[],
+ "_profile":{
+ "request_time":0.037659336
+ }
+}
+</code></pre></notextile>
+
+The value for the @api_token@ field should be added to Keepproxy's environment as ARVADOS_API_TOKEN. Make sure to also set ARVADOS_API_HOST to @uuid_prefix at .your.domain.
+
+h3. Set up a reverse proxy with SSL support
+
+Because the Keepproxy is intended for access from anywhere on the internet, it is recommended to use SSL for transport encryption.
+
+This is best achieved by putting a reverse proxy with SSL support in front of Keepproxy. Keepproxy itself runs on port 25107 by default; your reverse proxy can run on port 443 and pass requests to Keepproxy on port 25107.
+
+h3. Tell the API server about the Keepproxy server
+
+The API server needs to be informed about the presence of your Keepproxy server. Please execute the following commands on your <strong>shell server</strong>.
+
+<notextile>
+<pre><code>~$ <span class="userinput">prefix=`arv --format=uuid user current | cut -d- -f1`</span>
+~$ <span class="userinput">echo "Site prefix is '$prefix'"</span>
+~$ <span class="userinput">read -rd $'\000' keepservice <<EOF; arv keep_service create --keep-service "$keepservice"</span>
+<span class="userinput">{
+ "service_host":"keep.$prefix.your.domain",
+ "service_port":443,
+ "service_ssl_flag":true,
+ "service_type":"proxy"
+}
+EOF</span>
+</code></pre></notextile>
+
diff --git a/doc/install/install-manual-overview.html.textile.liquid b/doc/install/install-manual-overview.html.textile.liquid
new file mode 100644
index 0000000..363700a
--- /dev/null
+++ b/doc/install/install-manual-overview.html.textile.liquid
@@ -0,0 +1,27 @@
+---
+layout: default
+navsection: installguide
+title: Overview
+...
+
+The manual installation guide will walk you through setting up a basic Arvados cluster on a number of (virtual) GNU/Linux systems.
+
+<div class="alert alert-block alert-info">
+ <button type="button" class="close" data-dismiss="alert">×</button>
+ <h4>Note</h4>
+ <p>If you are looking to evaluate Arvados on one machine, we recommend the "Docker installation method":install-docker.html instead.</p>
+</div>
+
+h2. Manual installation
+
+# "Install the Arvados REST API server":install-api-server.html
+# "Create standard objects":create-standard-objects.html
+# "Install the Single Sign On (SSO) server":install-sso.html
+# "Install the Arvados websockets server":install-websockets-server.html
+# "Install the Arvados workbench application":install-workbench-app.html
+# "Install Keepstore":install-keep.html
+# "Install Keepproxy":install-keepproxy.html
+# "Install the Crunch dispatcher":install-crunch-dispatch.html
+# "Install a shell server":install-shell-server.html
+# "Install a compute node":install-compute-node.html
+# Install client libraries (see "SDK Reference":{{site.baseurl}}/sdk/index.html).
diff --git a/doc/install/install-manual-prerequisites-ruby.html.textile.liquid b/doc/install/install-manual-prerequisites-ruby.html.textile.liquid
new file mode 100644
index 0000000..0db1e43
--- /dev/null
+++ b/doc/install/install-manual-prerequisites-ruby.html.textile.liquid
@@ -0,0 +1,31 @@
+---
+layout: default
+navsection: installguide
+title: Install Ruby and bundler
+...
+
+We recommend Ruby >= 2.1.
+
+h2(#rvm). Option 1: Install with rvm
+
+<notextile>
+<pre><code>~$ <span class="userinput">\curl -sSL https://get.rvm.io | bash -s stable --ruby=2.1</span>
+~$ <span class="userinput">gem install bundler
+</span></code></pre></notextile>
+
+h2(#fromsource). Option 2: Install from source
+
+<notextile>
+<pre><code><span class="userinput">mkdir -p ~/src
+cd ~/src
+wget http://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.3.tar.gz
+tar xzf ruby-2.1.3.tar.gz
+cd ruby-2.1.3
+./configure
+make
+sudo make install
+
+sudo gem install bundler</span>
+</code></pre></notextile>
+
+
diff --git a/doc/install/install-manual-prerequisites.html.textile.liquid b/doc/install/install-manual-prerequisites.html.textile.liquid
new file mode 100644
index 0000000..06de1dd
--- /dev/null
+++ b/doc/install/install-manual-prerequisites.html.textile.liquid
@@ -0,0 +1,46 @@
+---
+layout: default
+navsection: installguide
+title: Prerequisites
+...
+
+h2. Hardware (or virtual machines)
+
+This guide assumes you have seven systems available in the same network subnet:
+
+<div class="offset1">
+table(table table-bordered table-condensed).
+|_Function_|_Number of nodes_|
+|Arvados REST API, Websockets, Workbench and Crunch dispatcher|1|
+|Arvados SSO server|1|
+|Arvados Keepproxy server|1|
+|Arvados Keepstore servers|2|
+|Arvados shell server|1|
+|Arvados compute node|1|
+</div>
+
+The number of Keepstore, shell and compute nodes listed above is a minimum. In a real production installation, you will likely run many more of each of those types of nodes. In such a scenario, you would probably also want to dedicate a node to the Workbenbench server and crunch dispatcher, respectively. For performance reasons, you may want to run the database server on a separate node as well.
+
+h2. A unique identifier
+
+Each Arvados installation should have a globally unique identifier, which is a unique 5-character alphanumeric string. Here is a snippet of ruby that generates such a string based on the hostname of your computer:
+
+<pre>
+Digest::MD5.hexdigest(`hostname`).to_i(16).to_s(36)[0..4]
+</pre>
+
+You may also use a different method to pick the unique identifier. The unique identifier will be part of the hostname of the services in your Arvados cluster. The rest of this documentation will refer to it as your @uuid_prefix at .
+
+
+h2. SSL certificates
+
+There are four public-facing services that will require an SSL certificate. If you do not have official SSL certificates, you can use self-signed certificates. By convention, we use the following hostname pattern:
+
+<div class="offset1">
+table(table table-bordered table-condensed).
+|_Function_|_Hostname_|
+|Arvados REST API|@uuid_prefix at .your.domain|
+|Arvados Websockets endpoint|ws. at uuid_prefix@.your.domain|
+|Arvados Keepproxy server|keep. at uuid_prefix@.your.domain|
+|Arvados Workbench|workbench. at uuid_prefix@.your.domain|
+</div>
diff --git a/doc/install/install-shell-server.html.textile.liquid b/doc/install/install-shell-server.html.textile.liquid
new file mode 100644
index 0000000..537f1a4
--- /dev/null
+++ b/doc/install/install-shell-server.html.textile.liquid
@@ -0,0 +1,17 @@
+---
+layout: default
+navsection: installguide
+title: Install a shell server
+...
+
+This installation guide assumes you are on a 64 bit Debian or Ubuntu system.
+
+There is nothing inherently special about an Arvados shell server. It is just a GNU/Linux machine with the Arvados SDKs installed. For optimal performance, the Arvados shell server should be on the same LAN as the Arvados cluster, but that is not required.
+
+h2. Install API tokens
+
+Please follow the "API token guide":{{site.baseurl}}/user/reference/api-tokens.html to get API tokens for your user and install them on your shell server. We will use those tokens to test the SDKs as we install them.
+
+h2. Install the SDKs
+
+Install the "Python SDK":{{site.baseurl}}/sdk/python/sdk-python.html and the "Command line SDK":{{site.baseurl}}/sdk/cli/index.html
diff --git a/doc/install/install-sso.html.textile.liquid b/doc/install/install-sso.html.textile.liquid
index 178673a..9cf4c4f 100644
--- a/doc/install/install-sso.html.textile.liquid
+++ b/doc/install/install-sso.html.textile.liquid
@@ -8,14 +8,7 @@ title: Install Single Sign On (SSO) server
h2(#dependencies). Install dependencies
-You need to have ruby 2.1 or higher and the bundler gem installed.
-
-One way to install those dependencies is:
-
-<notextile>
-<pre><code>~$ <span class="userinput">\curl -sSL https://get.rvm.io | bash -s stable --ruby=2.1</span>
-~$ <span class="userinput">gem install bundler
-</span></code></pre></notextile>
+Make sure you have "Ruby and bundler":install-manual-prerequisites-ruby.html installed.
h2(#install). Install SSO server
diff --git a/doc/install/install-workbench-app.html.textile.liquid b/doc/install/install-workbench-app.html.textile.liquid
index ea9e73c..00f33ac 100644
--- a/doc/install/install-workbench-app.html.textile.liquid
+++ b/doc/install/install-workbench-app.html.textile.liquid
@@ -1,27 +1,23 @@
---
layout: default
navsection: installguide
-title: Install the Arvados Workbench application
+title: Install Workbench
...
-h2. Prerequisites
+This installation guide assumes you are on a 64 bit Debian or Ubuntu system.
-# A GNU/linux (virtual) machine (can be shared with the API server)
-# A hostname for your Workbench application
+h2. Install prerequisites
-h2. Install dependencies
-
-If you haven't already installed the API server on the same host:
+<notextile>
+<pre><code>~$ <span class="userinput">sudo apt-get install \
+ bison build-essential gettext libcurl3 libcurl3-gnutls \
+ libcurl4-openssl-dev libpcre3-dev libpq-dev libreadline-dev \
+ libssl-dev libxslt1.1 sudo wget zlib1g-dev graphviz
+</span></code></pre></notextile>
-* Install Ruby 2.1 and Bundler: see the "dependencies" and "Ruby" sections on the "API server installation page":install-api-server.html#dependencies for details.
-* Omit postgresql. Workbench doesn't need its own database.
+Also make sure you have "Ruby and bundler":install-manual-prerequisites-ruby.html installed.
-Install graphviz.
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo apt-get install graphviz</span>
-</code></pre>
-</notextile>
+Workbench doesn't need its own database, so it does not need to have PostgreSQL installed.
h2. Download the source tree
@@ -60,8 +56,30 @@ The validation message from Rubygems was:
Using themes_for_rails (0.5.1) from https://github.com/holtkampw/themes_for_rails (at 1fd2d78)
</code></pre></notextile>
+h2. Choose your environment
+
+The Workbench application can be run in @development@ or in @production@ mode. Unless this installation is going to be used for development on the Workbench applicatoin itself, you should run it in @production@ mode.
+
+Copy the example environment file for your environment. For example, if you choose @production@:
+
+<notextile>
+<pre><code>~/arvados/apps/workbench$ <span class="userinput">cp -i config/environments/production.rb.example config/environments/production.rb</span>
+</code></pre></notextile>
+
h2. Configure the Workbench application
+First, copy the example configuration file:
+
+<notextile>
+<pre><code>~/arvados/apps/workbench$ <span class="userinput">cp -i config/application.yml.example config/application.yml</span>
+</code></pre></notextile>
+
+The Workbench application reads the @config/application.yml@ file, as well as the @config/application.defaults.yml@ file. Values in @config/application.yml@ take precedence over the defaults that are defined in @config/application.defaults.yml at . The @config/application.yml.example@ file is not read by the Workbench application and is provided for installation convenience, only.
+
+Consult @config/application.default.yml@ for a full list of configuration options. Always put your local configuration in @config/application.yml@, never edit @config/application.default.yml at .
+
+h3. secret_token
+
This application needs a secret token. Generate a new secret:
<notextile>
@@ -70,40 +88,55 @@ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
</code></pre>
</notextile>
-Copy @config/application.yml.example@ to @config/application.yml@ and edit it appropriately for your environment.
+Then put that value in the @secret_token@ field.
+
+h3. arvados_login_base and arvados_v1_base
-* Set @secret_token@ to the string you generated with @rake secret at .
-* Point @arvados_login_base@ and @arvados_v1_base@ at your "API server":install-api-server.html, like this:
+Point @arvados_login_base@ and @arvados_v1_base@ at your "API server":install-api-server.html. For example like this:
<notextile>
-<pre><code>arvados_login_base: https://your.host:3030/login
-arvados_v1_base: https://your.host:3030/arvados/v1
+<pre><code>arvados_login_base: https://prefix_uuid.your.domain/login
+arvados_v1_base: https://prefix_uuid.your.domain/arvados/v1
</code></pre>
</notextile>
-* @site_name@ can be any string to identify this Workbench.
-* If the SSL certificate you use for development isn't signed by a CA, make sure @arvados_insecure_https@ is @true at .
+h3. site_name
+
+ at site_name@ can be set to any arbitrary string. It is used to identify this Workbench to people visiting it.
+
+h3. arvados_insecure_https
+
+If the SSL certificate you use for your API server isn't an official certificate signed by a CA, make sure @arvados_insecure_https@ is @true at .
+
+h3. other options
+
+Consult @application.default.yml@ for a full list of configuration options. Always put your local configuration in @application.yml@ instead of editing @application.default.yml at .
Copy @config/piwik.yml.example@ to @config/piwik.yml@ and edit to suit.
-h2. Start a standalone server
+h2. Start the Workbench application
+
+h3. Development environment
-For testing and development, the easiest way to get started is to run the web server that comes with Rails.
+If you plan to run in development mode, you can now run the development server this way:
<notextile>
<pre><code>~/arvados/apps/workbench$ <span class="userinput">bundle exec rails server --port=3031</span>
-</code></pre>
-</notextile>
+</code></pre></notextile>
+
+h3. Production environment
-Point your browser to <notextile><code>http://<b>your.host</b>:3031/</code></notextile>.
+We recommend "Passenger":https://www.phusionpassenger.com/ to run the API server in production.
+
+Point it to the apps/workbench directory in the source tree.
h2. Trusted client setting
Log in to Workbench once to ensure that the Arvados API server has a record of the Workbench client. (It's OK if Workbench says your account hasn't been activated yet. We'll deal with that next.)
-In the API server project root, start the rails console. Locate the ApiClient record for your Workbench installation (typically, while you're setting this up, the @last@ one in the database is the one you want), then set the @is_trusted@ flag for the appropriate client record:
+In the <strong>API server</strong> project root, start the rails console. Locate the ApiClient record for your Workbench installation (typically, while you're setting this up, the @last@ one in the database is the one you want), then set the @is_trusted@ flag for the appropriate client record:
-<notextile><pre><code>~/arvados/services/api$ <span class="userinput">bundle exec rails console</span>
+<notextile><pre><code>~/arvados/services/api$ <span class="userinput">RAILS_ENV=production bundle exec rails console</span>
irb(main):001:0> <span class="userinput">wb = ApiClient.all.last; [wb.url_prefix, wb.created_at]</span>
=> ["https://workbench.example.com/", Sat, 19 Apr 2014 03:35:12 UTC +00:00]
irb(main):002:0> <span class="userinput">include CurrentApiClient</span>
@@ -113,8 +146,17 @@ irb(main):003:0> <span class="userinput">act_as_system_user do wb.update_attr
</code></pre>
</notextile>
-h2. Activate your own account
+h2(#admin-user). Add an admin user
+
+Next, we're going to use the rails console on the <strong>API server</strong> to activate our own account and give yourself admin privileges:
-Unless you already activated your account when installing the API server, the first time you log in to Workbench you will see a message that your account is awaiting activation.
+<notextile>
+<pre><code>~/arvados/services/api$ <span class="userinput">RAILS_ENV=production bundle exec rails console</span>
+irb(main):001:0> <span class="userinput">Thread.current[:user] = User.all.select(&:identity_url).last</span>
+irb(main):002:0> <span class="userinput">Thread.current[:user].is_admin = true</span>
+irb(main):003:0> <span class="userinput">Thread.current[:user].update_attributes is_admin: true, is_active: true</span>
+irb(main):004:0> <span class="userinput">User.where(is_admin: true).collect &:email</span>
+=> ["root", "<b>your_address at example.com</b>"]
+</code></pre></notextile>
-Activate your own account and give yourself administrator privileges by following the instructions in the "'Add an admin user' section of the API server install page":install-api-server.html#admin-user.
+At this point, you should have a working Workbench login with administrator privileges. Revisit your Workbench URL in a browser and reload the page to access it.
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list