[ARVADOS] updated: bbd84f0a5594fdbf21a967572d4c2b33d7987f2b
git at public.curoverse.com
git at public.curoverse.com
Thu Jan 28 19:38:48 EST 2016
Summary of changes:
doc/_includes/_install_git.liquid | 8 +++++++-
doc/_includes/_install_redhat_postgres_auth.liquid | 11 +++++++++++
doc/_includes/_install_ruby_and_bundler.liquid | 2 +-
doc/_includes/_install_runit.liquid | 13 +++++++++++++
doc/install/install-api-server.html.textile.liquid | 12 ++++++------
doc/install/install-arv-git-httpd.html.textile.liquid | 13 ++++---------
doc/install/install-compute-node.html.textile.liquid | 16 +++++-----------
doc/install/install-crunch-dispatch.html.textile.liquid | 11 +++++++++--
doc/install/install-keep-web.html.textile.liquid | 4 +++-
doc/install/install-keepproxy.html.textile.liquid | 4 +++-
doc/install/install-keepstore.html.textile.liquid | 4 +++-
doc/install/install-sso.html.textile.liquid | 15 +++------------
12 files changed, 68 insertions(+), 45 deletions(-)
create mode 100644 doc/_includes/_install_redhat_postgres_auth.liquid
create mode 100644 doc/_includes/_install_runit.liquid
via bbd84f0a5594fdbf21a967572d4c2b33d7987f2b (commit)
via 678e366229d4082d5977c7dadf94224d5d834fb7 (commit)
via 4abdedc4baddd6ed80b180b8820fba19068c1c3a (commit)
via df0f52924d6e843bcdbecd96a9b398b825b61a85 (commit)
via efe49059b1cdfc9f91504bcbdc96656b4b844839 (commit)
via c174caa6acbfb4927631160da728ec4d1f8ad410 (commit)
via 0c3ec9229478a836d6e8fe86369cf4dffdb58183 (commit)
from 00a94def60fcf314e4a8cf525f82ca4f9952ed78 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
commit bbd84f0a5594fdbf21a967572d4c2b33d7987f2b
Merge: 00a94de 678e366
Author: Brett Smith <brett at curoverse.com>
Date: Thu Jan 28 19:38:04 2016 -0500
Merge branch '8005-centos-3rdparty-installs-wip'
Closes #8005, #8135.
commit 678e366229d4082d5977c7dadf94224d5d834fb7
Author: Brett Smith <brett at curoverse.com>
Date: Thu Jan 28 19:27:13 2016 -0500
8005: Add tar Ruby build dependency on CentOS 6.
diff --git a/doc/_includes/_install_ruby_and_bundler.liquid b/doc/_includes/_install_ruby_and_bundler.liquid
index e79cffb..cd1aeaf 100644
--- a/doc/_includes/_install_ruby_and_bundler.liquid
+++ b/doc/_includes/_install_ruby_and_bundler.liquid
@@ -36,7 +36,7 @@ Install prerequisites for CentOS 6:
<pre><code><span class="userinput">sudo yum install \
libyaml-devel glibc-headers autoconf gcc-c++ glibc-devel \
patch readline-devel zlib-devel libffi-devel openssl-devel \
- automake libtool bison sqlite-devel
+ automake libtool bison sqlite-devel tar
</span></code></pre></notextile>
Install prerequisites for Ubuntu 12.04 or 14.04:
commit 4abdedc4baddd6ed80b180b8820fba19068c1c3a
Author: Brett Smith <brett at curoverse.com>
Date: Wed Jan 27 19:02:05 2016 -0500
8005: Install guide uses runit packages on Red Hat.
The runit RPMs only provide /etc/service. The .debs provide /etc/sv
and /etc/service. Our understanding is that /etc/sv is for all
service definitions (akin to /etc/init.d), and /etc/service is for
service definitions that runit should start at boot (akin to
/etc/rcN.d). To provide uniformity, our install guide instructs users
to make /etc/sv if needed, and link it to /etc/service.
This commit could go farther. Today it would be best if all the runit
sections in the install guide followed Tom's modern template used for
arv-git-httpd and arvados-docker-cleaner. However, that will probably
require some creation and testing of log/run scripts, and some
adaptation of the run scripts to fit the template. I wish I could
include those improvements now, but unfortunately I'm out of time, so
they'll have to wait for another day.
diff --git a/doc/_includes/_install_runit.liquid b/doc/_includes/_install_runit.liquid
new file mode 100644
index 0000000..8a1581c
--- /dev/null
+++ b/doc/_includes/_install_runit.liquid
@@ -0,0 +1,13 @@
+On Debian-based systems:
+
+<notextile>
+<pre><code>~$ <span class="userinput">sudo apt-get install runit</span>
+</code></pre>
+</notextile>
+
+On Red Hat-based systems:
+
+<notextile>
+<pre><code>~$ <span class="userinput">sudo yum install runit</span>
+</code></pre>
+</notextile>
diff --git a/doc/install/install-api-server.html.textile.liquid b/doc/install/install-api-server.html.textile.liquid
index 6b9c068..36eeee3 100644
--- a/doc/install/install-api-server.html.textile.liquid
+++ b/doc/install/install-api-server.html.textile.liquid
@@ -203,7 +203,9 @@ For best performance, we recommend you use Nginx as your Web server front-end, w
<ol>
<li><a href="https://www.phusionpassenger.com/library/walkthroughs/deploy/ruby/ownserver/nginx/oss/install_passenger_main.html">Install Nginx and Phusion Passenger</a>.</li>
-<li><p>Puma is already included with the API server's gems. We recommend you run it as a service under <a href="http://smarden.org/runit/">runit</a> or a similar tool. Here's a sample runit script for that:</p>
+<li><p>Install runit to supervise the Puma daemon. {% include 'install_runit' %}</p></li>
+
+<li><p>Install the script below as the run script for the Puma service, modifying it as directed by the comments.</p>
<pre><code>#!/bin/bash
diff --git a/doc/install/install-arv-git-httpd.html.textile.liquid b/doc/install/install-arv-git-httpd.html.textile.liquid
index 146dbe1..dbf67bf 100644
--- a/doc/install/install-arv-git-httpd.html.textile.liquid
+++ b/doc/install/install-arv-git-httpd.html.textile.liquid
@@ -255,19 +255,13 @@ fatal: No REQUEST_METHOD from server
h3. Enable arvados-git-httpd
-On Debian-based systems, install runit:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo apt-get install runit</span>
-</code></pre>
-</notextile>
-
-On Red Hat-based systems, "install runit from source":http://smarden.org/runit/install.html or use an alternative daemon supervisor.
+Install runit to supervise the arvados-git-httpd daemon. {% include 'install_runit' %}
Configure runit to run arvados-git-httpd, making sure to update the API host to match your site:
<notextile>
-<pre><code>~$ <span class="userinput">cd /etc/sv</span>
+<pre><code>~$ <span class="userinput">sudo mkdir -p /etc/sv</span>
+~$ <span class="userinput">cd /etc/sv</span>
/etc/sv$ <span class="userinput">sudo mkdir arvados-git-httpd; cd arvados-git-httpd</span>
/etc/sv/arvados-git-httpd$ <span class="userinput">sudo mkdir log</span>
/etc/sv/arvados-git-httpd$ <span class="userinput">sudo sh -c 'cat >log/run' <<'EOF'
@@ -285,6 +279,7 @@ export PATH="$PATH:/var/lib/arvados/git/bin"
exec chpst -u git:git arvados-git-httpd -address=:9001 -git-command=/var/lib/arvados/git/gitolite/src/gitolite-shell -repo-root=<b>/var/lib/arvados/git</b>/repositories 2>&1
EOF</span>
/etc/sv/arvados-git-httpd$ <span class="userinput">sudo chmod +x run log/run</span>
+/etc/sv/arvados-git-httpd$ <span class="userinput">sudo ln -s "$(pwd)" /etc/service/</span>
</code></pre>
</notextile>
diff --git a/doc/install/install-compute-node.html.textile.liquid b/doc/install/install-compute-node.html.textile.liquid
index d709234..ccca0cc 100644
--- a/doc/install/install-compute-node.html.textile.liquid
+++ b/doc/install/install-compute-node.html.textile.liquid
@@ -82,19 +82,13 @@ The arvados-docker-cleaner program removes least recently used docker images as
This also removes all containers as soon as they exit, as if they were run with @docker run --rm at . If you need to debug or inspect containers after they stop, temporarily stop arvados-docker-cleaner or run it with @--remove-stopped-containers never at .
{% include 'notebox_end' %}
-On Debian-based systems, install runit:
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo apt-get install runit</span>
-</code></pre>
-</notextile>
-
-On Red Hat-based systems, "install runit from source":http://smarden.org/runit/install.html or use an alternative daemon supervisor.
+Install runit to supervise the Docker cleaner daemon. {% include 'install_runit' %}
Configure runit to run the image cleaner using a suitable quota for your compute nodes and workload:
<notextile>
-<pre><code>~$ <span class="userinput">cd /etc/sv</span>
+<pre><code>~$ <span class="userinput">sudo mkdir -p /etc/sv</span>
+~$ <span class="userinput">cd /etc/sv</span>
/etc/sv$ <span class="userinput">sudo mkdir arvados-docker-cleaner; cd arvados-docker-cleaner</span>
/etc/sv/arvados-docker-cleaner$ <span class="userinput">sudo mkdir log log/main</span>
/etc/sv/arvados-docker-cleaner$ <span class="userinput">sudo sh -c 'cat >log/run' <<'EOF'
@@ -106,6 +100,7 @@ EOF</span>
exec python3 -m arvados_docker.cleaner --quota <b>50G</b>
EOF</span>
/etc/sv/arvados-docker-cleaner$ <span class="userinput">sudo chmod +x run log/run</span>
+/etc/sv/arvados-docker-cleaner$ <span class="userinput">sudo ln -s "$(pwd)" /etc/service/</span>
</code></pre>
</notextile>
@@ -182,4 +177,3 @@ And remove your token from the environment:
</code>
</pre>
</notextile>
-
diff --git a/doc/install/install-crunch-dispatch.html.textile.liquid b/doc/install/install-crunch-dispatch.html.textile.liquid
index 2756500..b89d42b 100644
--- a/doc/install/install-crunch-dispatch.html.textile.liquid
+++ b/doc/install/install-crunch-dispatch.html.textile.liquid
@@ -168,7 +168,9 @@ To dispatch Arvados jobs:
* @crunch-job@ needs the installation path of the Perl SDK in its @PERLLIB at .
* @crunch-job@ needs the @ARVADOS_API_HOST@ (and, if necessary, @ARVADOS_API_HOST_INSECURE@) environment variable set.
-We recommend you run @crunch-dispatch.rb@ under "runit":http://smarden.org/runit/ or a similar supervisor. Here's an example runit service script:
+Install runit to monitor the Crunch dispatch daemon. {% include 'install_runit' %}
+
+Install the script below as the run script for the Crunch dispatch service, modifying it as directed by the comments.
<notextile>
<pre><code>#!/bin/sh
diff --git a/doc/install/install-keep-web.html.textile.liquid b/doc/install/install-keep-web.html.textile.liquid
index c7a7b20..16d23e6 100644
--- a/doc/install/install-keep-web.html.textile.liquid
+++ b/doc/install/install-keep-web.html.textile.liquid
@@ -55,7 +55,9 @@ Usage of keep-web:
{% assign railsout = "zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz" %}
If you intend to use Keep-web to serve public data to anonymous clients, configure it with an anonymous token. You can use the same one you used when you set up your Keepproxy server, or use the following command on the <strong>API server</strong> to create another. {% include 'install_rails_command' %}
-We recommend running Keep-web under "runit":https://packages.debian.org/search?keywords=runit or a similar supervisor. The basic command to start Keep-web is:
+Install runit to supervise the Keep-web daemon. {% include 'install_runit' %}
+
+The basic command to start Keep-web in the service run script is:
<notextile>
<pre><code>export ARVADOS_API_HOST=<span class="userinput">uuid_prefix</span>.your.domain
diff --git a/doc/install/install-keepproxy.html.textile.liquid b/doc/install/install-keepproxy.html.textile.liquid
index 14e5ed5..d427274 100644
--- a/doc/install/install-keepproxy.html.textile.liquid
+++ b/doc/install/install-keepproxy.html.textile.liquid
@@ -57,7 +57,9 @@ The Keepproxy server needs a token to talk to the API server. On the <strong>AP
h3. Set up the Keepproxy service
-We recommend you run Keepproxy under "runit":http://smarden.org/runit/ or a similar supervisor. Make sure the launcher sets the envirnoment variables @ARVADOS_API_TOKEN@ (with the token you just generated), @ARVADOS_API_HOST@, and, if needed, @ARVADOS_API_HOST_INSECURE at . The core keepproxy command to run is:
+Install runit to supervise the keepproxy daemon. {% include 'install_runit' %}
+
+The run script for the keepproxy service should set the environment variables @ARVADOS_API_TOKEN@ (with the token you just generated), @ARVADOS_API_HOST@, and, if needed, @ARVADOS_API_HOST_INSECURE at . The core keepproxy command to run is:
<notextile>
<pre><code>ARVADOS_API_TOKEN=<span class="userinput">{{railsout}}</span> ARVADOS_API_HOST=<span class="userinput">uuid_prefix.your.domain</span> exec keepproxy
diff --git a/doc/install/install-keepstore.html.textile.liquid b/doc/install/install-keepstore.html.textile.liquid
index 2e7382b..13dfaf6 100644
--- a/doc/install/install-keepstore.html.textile.liquid
+++ b/doc/install/install-keepstore.html.textile.liquid
@@ -94,7 +94,9 @@ Equivalently:
h3. Run keepstore as a supervised service
-We recommend running Keepstore under "runit":http://smarden.org/runit/ or something similar, using a run script like the following:
+Install runit to supervise the keepstore daemon. {% include 'install_runit' %}
+
+Install this script as the run script for the keepstore service, modifying it as directed below.
<notextile>
<pre><code>#!/bin/sh
commit df0f52924d6e843bcdbecd96a9b398b825b61a85
Author: Brett Smith <brett at curoverse.com>
Date: Wed Jan 27 19:08:33 2016 -0500
8005: Install guide gets SLURM and MUNGE from RPMs.
diff --git a/doc/install/install-crunch-dispatch.html.textile.liquid b/doc/install/install-crunch-dispatch.html.textile.liquid
index 907f0fd..2756500 100644
--- a/doc/install/install-crunch-dispatch.html.textile.liquid
+++ b/doc/install/install-crunch-dispatch.html.textile.liquid
@@ -58,7 +58,12 @@ On Debian-based systems:
</code></pre>
</notextile>
-On Red Hat-based systems, "install SLURM and munge from source following their installation guide":https://computing.llnl.gov/linux/slurm/quickstart_admin.html.
+On Red Hat-based systems:
+
+<notextile>
+<pre><code>~$ <span class="userinput">sudo yum install slurm munge</span>
+</code></pre>
+</notextile>
Now we need to give SLURM a configuration file in @/etc/slurm-llnl/slurm.conf at . Here's an example:
commit efe49059b1cdfc9f91504bcbdc96656b4b844839
Author: Brett Smith <brett at curoverse.com>
Date: Wed Jan 27 18:54:57 2016 -0500
8005: Fix bad Textile markup in compute node install guide.
The switch dashes created strikethrough for much of the notebox.
diff --git a/doc/install/install-compute-node.html.textile.liquid b/doc/install/install-compute-node.html.textile.liquid
index b1e377a..d709234 100644
--- a/doc/install/install-compute-node.html.textile.liquid
+++ b/doc/install/install-compute-node.html.textile.liquid
@@ -79,7 +79,7 @@ h2. Configure the Docker cleaner
The arvados-docker-cleaner program removes least recently used docker images as needed to keep disk usage below a configured limit.
{% include 'notebox_begin' %}
-This also removes all containers as soon as they exit, as if they were run with `docker run --rm`. If you need to debug or inspect containers after they stop, temporarily stop arvados-docker-cleaner or run it with `--remove-stopped-containers never`.
+This also removes all containers as soon as they exit, as if they were run with @docker run --rm at . If you need to debug or inspect containers after they stop, temporarily stop arvados-docker-cleaner or run it with @--remove-stopped-containers never at .
{% include 'notebox_end' %}
On Debian-based systems, install runit:
commit c174caa6acbfb4927631160da728ec4d1f8ad410
Author: Brett Smith <brett at curoverse.com>
Date: Wed Jan 27 15:15:23 2016 -0500
8005: Document installing Git on CentOS 6 from RepoForge.
diff --git a/doc/_includes/_install_git.liquid b/doc/_includes/_install_git.liquid
index 2ca1ce4..60092c1 100644
--- a/doc/_includes/_install_git.liquid
+++ b/doc/_includes/_install_git.liquid
@@ -1,3 +1,9 @@
{% include 'notebox_begin' %}
-Arvados requires git version 1.7.10 or later. If you are using an earlier version of git, please update your git version.
+The Arvados API and Git servers require Git 1.7.10 or later. You can get this version on CentOS 6 from RepoForge. "Install the repository":http://repoforge.org/use/, then run:
+
+<notextile>
+<pre><code>~$ <span class="userinput">sudo yum install --enablerepo=rpmforge-extras git</span>
+</code></pre>
+</notextile>
+
{% include 'notebox_end' %}
commit 0c3ec9229478a836d6e8fe86369cf4dffdb58183
Author: Brett Smith <brett at curoverse.com>
Date: Wed Jan 27 15:00:17 2016 -0500
8005: DRY up PostgreSQL password auth instructions on CentOS 6.
diff --git a/doc/_includes/_install_redhat_postgres_auth.liquid b/doc/_includes/_install_redhat_postgres_auth.liquid
new file mode 100644
index 0000000..35f8b79
--- /dev/null
+++ b/doc/_includes/_install_redhat_postgres_auth.liquid
@@ -0,0 +1,11 @@
+{% include 'notebox_begin' %}
+
+If you are installing on CentOS 6, you will need to modify PostgreSQL's configuration to allow password authentication for local users. The default configuration allows 'ident' authentication only. The following commands will make the configuration change, and restart PostgreSQL for it to take effect.
+
+<notextile>
+<pre><code>~$ <span class="userinput">sudo sed -ri -e 's/^(host +all +all +(127\.0\.0\.1\/32|::1\/128) +)ident$/\1md5/' {{pg_hba_path}}</span>
+~$ <span class="userinput">sudo service {{pg_service}} restart</span>
+</code></pre>
+</notextile>
+
+{% include 'notebox_end' %}
diff --git a/doc/install/install-api-server.html.textile.liquid b/doc/install/install-api-server.html.textile.liquid
index fae06de..6b9c068 100644
--- a/doc/install/install-api-server.html.textile.liquid
+++ b/doc/install/install-api-server.html.textile.liquid
@@ -52,11 +52,9 @@ Enter password for new role: <span class="userinput">paste-password-you-generate
Enter it again: <span class="userinput">paste-password-again</span>
</code></pre></notextile>
-{% include 'notebox_begin' %}
-
-This user setup assumes that your PostgreSQL is configured to accept password authentication. Red Hat systems use ident-based authentication by default. You may need to either adapt the user creation, or reconfigure PostgreSQL (in @pg_hba.conf@) to accept password authentication.
-
-{% include 'notebox_end' %}
+{% assign pg_hba_path = "/opt/rh/postgresql92/root/var/lib/pgsql/data/pg_hba.conf" %}
+{% assign pg_service = "postgresql92-postgresql" %}
+{% include 'install_redhat_postgres_auth' %}
Create the database:
diff --git a/doc/install/install-sso.html.textile.liquid b/doc/install/install-sso.html.textile.liquid
index a306403..aaa6211 100644
--- a/doc/install/install-sso.html.textile.liquid
+++ b/doc/install/install-sso.html.textile.liquid
@@ -94,18 +94,9 @@ On a Red Hat-based system, we also need to initialize the database system:
</code></pre>
</notextile>
-{% include 'notebox_begin' %}
-
-If you are installing on CentOS6, you will need to modify PostgreSQL's configuration to allow password authentication for local users. The default configuration allows 'ident' only. The following commands will make the configuration change, and restart PostgreSQL for it to take effect.
-<br/>
-<notextile>
-<pre><code>~$ <span class="userinput">sudo sed -i -e "s/127.0.0.1\/32 ident/127.0.0.1\/32 md5/" /var/lib/pgsql/data/pg_hba.conf</span>
-~$ <span class="userinput">sudo sed -i -e "s/::1\/128 ident/::1\/128 md5/" /var/lib/pgsql/data/pg_hba.conf</span>
-~$ <span class="userinput">sudo service postgresql restart</span>
-</code></pre>
-</notextile>
-{% include 'notebox_end' %}
-
+{% assign pg_service = "postgresql" %}
+{% assign pg_hba_path = "/var/lib/pgsql/data/pg_hba.conf" %}
+{% include 'install_redhat_postgres_auth' %}
Next, generate a new database password. Nobody ever needs to memorize it or type it, so make a strong one:
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list