[ARVADOS] updated: 01a76d36f6afcd3bb7c8e224881fe98364a47b85
git at public.curoverse.com
git at public.curoverse.com
Tue Aug 4 02:47:14 EDT 2015
Summary of changes:
.../create-standard-objects.html.textile.liquid | 2 +-
doc/install/install-api-server.html.textile.liquid | 13 +++-
.../install-arv-git-httpd.html.textile.liquid | 91 ++++++++++++++++------
services/api/config/application.default.yml | 2 +-
4 files changed, 78 insertions(+), 30 deletions(-)
via 01a76d36f6afcd3bb7c8e224881fe98364a47b85 (commit)
via 30d1623c0fe687acdef334c037c24752c0c61620 (commit)
via c94bc0d5c2ae1514a7b0799477279567fa4df177 (commit)
via 89dcdd013aef473cef6f2b94bfcd2308b60a55d4 (commit)
via 70a9d8683b62a91e7d3ee3e827af6b69b9aa9a94 (commit)
via 4c5925a0eceebcd86a4c59a0123dac49aabffae0 (commit)
via b9a61af34e1f77500d5d0850b4bfd6e11a6f783a (commit)
via 821cb42d42976ec6f750e0b0b191ccc36cbad295 (commit)
via cec08599bc15bccdf1acfebb4f773f139cc45dcb (commit)
via ed2e9fece8c44c5aa464de012b3fce84e7b049eb (commit)
via 0c96fc858f1f4954d997f2065550087dcc7f8bec (commit)
via 2e92762444af21749e1ef232aecb6eeb98b312cd (commit)
from 7e773735e9d547381a77baeeb139696dfa66a7a6 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
commit 01a76d36f6afcd3bb7c8e224881fe98364a47b85
Author: Tom Clegg <tom at curoverse.com>
Date: Tue Aug 4 02:36:29 2015 -0400
6663: More explicit gitolite version advice, and link to home page.
diff --git a/doc/install/install-arv-git-httpd.html.textile.liquid b/doc/install/install-arv-git-httpd.html.textile.liquid
index 9db92e5..6f80be8 100644
--- a/doc/install/install-arv-git-httpd.html.textile.liquid
+++ b/doc/install/install-arv-git-httpd.html.textile.liquid
@@ -100,7 +100,7 @@ git at gitserver:~$ <span class="userinput">rm .ssh/authorized_keys</span>
h2. Install gitolite
-Check "https://github.com/sitaramc/gitolite/tags":https://github.com/sitaramc/gitolite/tags for the latest stable version (_e.g.,_ @v3.6.3@).
+Check "https://github.com/sitaramc/gitolite/tags":https://github.com/sitaramc/gitolite/tags for the latest stable version. This guide was tested with @v3.6.3 at . _Versions below 3.0 are missing some features needed by Arvados, and should not be used._
Download and install the version you selected.
@@ -121,6 +121,8 @@ WARNING: /var/lib/arvados/git/.ssh/authorized_keys missing; creating a new one
</code></pre>
</notextile>
+_If this didn't go well, more detail about installing gitolite, and information about how it works, can be found on the "gitolite home page":http://gitolite.com/._
+
Clone the gitolite-admin repository. The arvados-git-sync.rb script works by editing the files in this working directory and pushing them to gitolite. Here we make sure "git push" won't produce any errors or warnings.
<notextile>
commit 30d1623c0fe687acdef334c037c24752c0c61620
Author: Tom Clegg <tom at curoverse.com>
Date: Tue Aug 4 01:58:06 2015 -0400
6663: Split "install runit" into debian and red hat variants.
diff --git a/doc/install/install-arv-git-httpd.html.textile.liquid b/doc/install/install-arv-git-httpd.html.textile.liquid
index 7ff6527..9db92e5 100644
--- a/doc/install/install-arv-git-httpd.html.textile.liquid
+++ b/doc/install/install-arv-git-httpd.html.textile.liquid
@@ -257,11 +257,19 @@ fatal: No REQUEST_METHOD from server
h3. Enable arvados-git-httpd
-Install "runit":http://smarden.org/runit/ (if it's not already installed) and configure it to run arvados-git-httpd. Update the API host to match your site.
+On Debian-based systems, install runit:
<notextile>
<pre><code>~$ <span class="userinput">sudo apt-get install runit</span>
-~$ <span class="userinput">cd /etc/sv</span>
+</code></pre>
+</notextile>
+
+On Red Hat-based systems, "install runit from source":http://smarden.org/runit/install.html or use an alternative daemon supervisor.
+
+Configure runit to run arvados-git-httpd, making sure to update the API host to match your site:
+
+<notextile>
+<pre><code>~$ <span class="userinput">cd /etc/sv</span>
/etc/sv$ <span class="userinput">sudo mkdir arvados-git-httpd; cd arvados-git-httpd</span>
/etc/sv/arvados-git-httpd$ <span class="userinput">sudo mkdir log</span>
/etc/sv/arvados-git-httpd$ <span class="userinput">sudo sh -c 'cat >log/run' <<'EOF'
commit c94bc0d5c2ae1514a7b0799477279567fa4df177
Author: Tom Clegg <tom at curoverse.com>
Date: Tue Aug 4 01:49:38 2015 -0400
6663: Add "restart nginx" instructions.
diff --git a/doc/install/install-api-server.html.textile.liquid b/doc/install/install-api-server.html.textile.liquid
index 00ccb93..5cdfced 100644
--- a/doc/install/install-api-server.html.textile.liquid
+++ b/doc/install/install-api-server.html.textile.liquid
@@ -324,7 +324,12 @@ server {
</code></pre>
</li>
-<li>Restart Nginx.</li>
+<li><p>Restart Nginx:</p>
+
+<pre><code>~$ <span class="userinput">sudo nginx -s reload</span>
+</code></pre>
+
+</li>
</ol>
</notextile>
diff --git a/doc/install/install-arv-git-httpd.html.textile.liquid b/doc/install/install-arv-git-httpd.html.textile.liquid
index 0abe039..7ff6527 100644
--- a/doc/install/install-arv-git-httpd.html.textile.liquid
+++ b/doc/install/install-arv-git-httpd.html.textile.liquid
@@ -326,3 +326,12 @@ In your API server's @config/application.yml@ file, add the following entry:
</notextile>
Make sure to include the trailing slash.
+
+h2. Restart nginx
+
+Restart nginx to make the nginx and API server configuration changes take effect.
+
+<notextile>
+<pre><code>gitserver:~$ <span class="userinput">sudo nginx -s reload</span>
+</code></pre>
+</notextile>
commit 89dcdd013aef473cef6f2b94bfcd2308b60a55d4
Author: Tom Clegg <tom at curoverse.com>
Date: Tue Aug 4 01:41:45 2015 -0400
6663: Update nginx config to look more like API server's.
diff --git a/doc/install/install-arv-git-httpd.html.textile.liquid b/doc/install/install-arv-git-httpd.html.textile.liquid
index 0b636fe..0abe039 100644
--- a/doc/install/install-arv-git-httpd.html.textile.liquid
+++ b/doc/install/install-arv-git-httpd.html.textile.liquid
@@ -288,22 +288,31 @@ The arvados-git-httpd service will be accessible from anywhere on the internet,
This is best achieved by putting a reverse proxy with SSL support in front of arvados-git-httpd, running on port 443 and passing requests to @arvados-git-httpd@ on port 9001 (or whichever port you used in your run script).
<notextile>
-<pre><code><span class="userinput">http {
- upstream arvados-git-httpd {
- server localhost:9001;
- }
- server {
- listen *:443 ssl;
- server_name git.uuid_prefix.example.com;
- ssl_certificate /root/git.uuid_prefix.example.com.crt;
- ssl_certificate_key /root/git.uuid_prefix.example.com.key;
- location / {
- proxy_pass http://arvados-git-httpd;
- proxy_set_header X-Forwarded-For $remote_addr;
- }
+<pre><code>
+upstream arvados-git-httpd {
+ server 127.0.0.1:<span class="userinput">9001</span>;
+}
+server {
+ listen <span class="userinput">[your public IP address]</span>:443 ssl;
+ server_name git.<span class="userinput">uuid_prefix.your.domain</span>;
+
+ ssl on;
+ ssl_certificate <span class="userinput">/YOUR/PATH/TO/cert.pem</span>;
+ ssl_certificate_key <span class="userinput">/YOUR/PATH/TO/cert.key</span>;
+
+ location / {
+ proxy_pass http://arvados-git-httpd;
+ proxy_redirect off;
+ proxy_connect_timeout 90s;
+ proxy_read_timeout 300s;
+
+ proxy_set_header X-Forwarded-Proto https;
+ proxy_set_header Host $http_host;
+ proxy_set_header X-External-Client $external_client;
+ proxy_set_header X-Real-IP $remote_addr;
+ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
-</span>
</code></pre>
</notextile>
commit 70a9d8683b62a91e7d3ee3e827af6b69b9aa9a94
Author: Tom Clegg <tom at curoverse.com>
Date: Tue Aug 4 01:30:05 2015 -0400
6663: Formatting fixes.
diff --git a/doc/install/install-arv-git-httpd.html.textile.liquid b/doc/install/install-arv-git-httpd.html.textile.liquid
index cb378b7..0b636fe 100644
--- a/doc/install/install-arv-git-httpd.html.textile.liquid
+++ b/doc/install/install-arv-git-httpd.html.textile.liquid
@@ -85,7 +85,7 @@ gitserver:~$ <span class="userinput">sudo chown -R git:git ~git</span>
</code></pre>
</notextile>
-The git user needs its own SSH key. (It must be able to run @ssh git at localhost@ from scripts.)
+The git user needs its own SSH key. (It must be able to run <code>ssh git at localhost</code> from scripts.)
<notextile>
<pre><code>gitserver:~$ <span class="userinput">sudo -u git -i bash</span>
@@ -100,7 +100,7 @@ git at gitserver:~$ <span class="userinput">rm .ssh/authorized_keys</span>
h2. Install gitolite
-Check https://github.com/sitaramc/gitolite/tags for the latest stable version (_e.g.,_ @v3.6.3@).
+Check "https://github.com/sitaramc/gitolite/tags":https://github.com/sitaramc/gitolite/tags for the latest stable version (_e.g.,_ @v3.6.3@).
Download and install the version you selected.
@@ -140,7 +140,7 @@ Everything up-to-date
</code></pre>
</notextile>
-h2. Configure gitolite
+h3. Configure gitolite
Configure gitolite to look up a repository name like @username/reponame.git@ and find the appropriate bare repository storage directory.
@@ -188,7 +188,7 @@ Create a configuration file @/var/www/arvados-api/current/config/arvados-clients
</code></pre>
</notextile>
-h2. Enable the synchronization script
+h3. Enable the synchronization script
The API server package includes a script that retrieves the current set of repository names and permissions from the API, writes them to @arvadosaliases.pl@ in a format usable by gitolite, and triggers gitolite hooks which create new empty repositories if needed. This script should run every 2 to 5 minutes.
commit 4c5925a0eceebcd86a4c59a0123dac49aabffae0
Author: Tom Clegg <tom at curoverse.com>
Date: Tue Aug 4 01:22:15 2015 -0400
6663: Fix API config example for HTTPS, add corresponding SSH example.
diff --git a/doc/install/install-arv-git-httpd.html.textile.liquid b/doc/install/install-arv-git-httpd.html.textile.liquid
index a562115..cb378b7 100644
--- a/doc/install/install-arv-git-httpd.html.textile.liquid
+++ b/doc/install/install-arv-git-httpd.html.textile.liquid
@@ -206,6 +206,17 @@ Otherwise, create @/etc/cron.d/arvados-git-sync@ with the following content:
</code></pre>
</notextile>
+h3. Configure the API server to advertise the correct SSH URLs
+
+In your API server's @config/application.yml@ file, add the following entry:
+
+<notextile>
+<pre><code>git_repo_ssh_base: git at git.<span class="userinput">uuid_prefix.your.domain</span>:
+</code></pre>
+</notextile>
+
+Make sure to include the trailing colon.
+
h2. Install the arvados-git-httpd package
This is needed only for HTTPS access.
@@ -296,11 +307,13 @@ This is best achieved by putting a reverse proxy with SSL support in front of ar
</code></pre>
</notextile>
-h3. Tell the API server about the arvados-git-httpd service
+h3. Configure the API server to advertise the correct HTTPS URLs
In your API server's @config/application.yml@ file, add the following entry:
<notextile>
-<pre><code>git_http_base: git.<span class="userinput">uuid_prefix.your.domain</span>
+<pre><code>git_repo_http_base: https://git.<span class="userinput">uuid_prefix.your.domain</span>/
</code></pre>
</notextile>
+
+Make sure to include the trailing slash.
commit b9a61af34e1f77500d5d0850b4bfd6e11a6f783a
Author: Tom Clegg <tom at curoverse.com>
Date: Tue Aug 4 01:21:59 2015 -0400
6663: gitolite, not arvados-git-sync.rb, creates empty repos.
diff --git a/doc/install/install-arv-git-httpd.html.textile.liquid b/doc/install/install-arv-git-httpd.html.textile.liquid
index d853ac7..a562115 100644
--- a/doc/install/install-arv-git-httpd.html.textile.liquid
+++ b/doc/install/install-arv-git-httpd.html.textile.liquid
@@ -190,7 +190,7 @@ Create a configuration file @/var/www/arvados-api/current/config/arvados-clients
h2. Enable the synchronization script
-The API server package includes a script that retrieves the current set of repository names and permissions from the API, writes names and permissions to @arvadosaliases.pl@ in a format usable by gitolite, and creates new empty repositories if needed. This script should run every 2 to 5 minutes.
+The API server package includes a script that retrieves the current set of repository names and permissions from the API, writes them to @arvadosaliases.pl@ in a format usable by gitolite, and triggers gitolite hooks which create new empty repositories if needed. This script should run every 2 to 5 minutes.
If you are using RVM, create @/etc/cron.d/arvados-git-sync@ with the following content:
commit 821cb42d42976ec6f750e0b0b191ccc36cbad295
Author: Tom Clegg <tom at curoverse.com>
Date: Tue Aug 4 01:08:28 2015 -0400
6663: Clarify "gitolite installed in X" vs. "hosted repos stored in X/repositories".
diff --git a/doc/install/install-arv-git-httpd.html.textile.liquid b/doc/install/install-arv-git-httpd.html.textile.liquid
index 3020aea..d853ac7 100644
--- a/doc/install/install-arv-git-httpd.html.textile.liquid
+++ b/doc/install/install-arv-git-httpd.html.textile.liquid
@@ -72,7 +72,7 @@ On Red Hat-based systems:
h2. Create a "git" user and a storage directory
-Hosted repositories will be stored under @/var/lib/arvados/git/@. If you choose a different location, make sure to update the @git_repositories_dir@ entry in your API server's @config/application.yml@ file, preserving the trailing @repositories/@ part.
+Gitolite and some additional scripts will be installed in @/var/lib/arvados/git@, which means hosted repository data will be stored in @/var/lib/arvados/git/repositories at . If you choose to install gitolite in a different location, make sure to update the @git_repositories_dir@ entry in your API server's @config/application.yml@ file accordingly: for example, if you install gitolite at @/data/gitolite@ then your @git_repositories_dir@ will be @/data/gitolite/repositories at .
A new UNIX account called "git" will own the files. This makes git URLs look familiar to users (<code>git@[...]:username/reponame.git</code>).
commit cec08599bc15bccdf1acfebb4f773f139cc45dcb
Author: Tom Clegg <tom at curoverse.com>
Date: Tue Aug 4 00:59:43 2015 -0400
6663: Clarify port 22 goes to sshd.
diff --git a/doc/install/install-arv-git-httpd.html.textile.liquid b/doc/install/install-arv-git-httpd.html.textile.liquid
index 95ab1c3..3020aea 100644
--- a/doc/install/install-arv-git-httpd.html.textile.liquid
+++ b/doc/install/install-arv-git-httpd.html.textile.liquid
@@ -30,7 +30,7 @@ By convention, we use the following hostname for the git service:
Here, we show how to install the git hosting services *on the same host as your API server.* Using a different host is not yet fully supported. On this page we will refer to it as your git server.
{% include 'notebox_end' %}
-DNS and network configuration should be set up so port 443 reaches your HTTPS proxy, and port 22 reaches your git server.
+DNS and network configuration should be set up so port 443 reaches your HTTPS proxy, and port 22 reaches the OpenSSH service on your git server.
h2. Generate an API token
commit ed2e9fece8c44c5aa464de012b3fce84e7b049eb
Author: Tom Clegg <tom at curoverse.com>
Date: Tue Aug 4 00:47:37 2015 -0400
6663: Clarify default vs. example git_repositories_dir.
diff --git a/doc/install/install-api-server.html.textile.liquid b/doc/install/install-api-server.html.textile.liquid
index 16b3236..00ccb93 100644
--- a/doc/install/install-api-server.html.textile.liquid
+++ b/doc/install/install-api-server.html.textile.liquid
@@ -169,13 +169,15 @@ h3(#git_repositories_dir). git_repositories_dir
The @git_repositories_dir@ setting specifies the directory where user git repositories will be stored.
-Some additional git setup will be done on the "next page":install-arv-git-httpd.html. For now, create an empty directory in the default location:
+The git server setup process is covered on "its own page":install-arv-git-httpd.html. For now, create an empty directory in the default location:
<notextile>
<pre><code>~$ <span class="userinput">sudo mkdir -p /var/lib/arvados/git/repositories</span>
</code></pre></notextile>
-Example @application.yml@:
+If you intend to store your git repositories are stored in a different location, specify that location in @application.yml at .
+
+Default setting in @application.default.yml@:
<notextile>
<pre><code> git_repositories_dir: <span class="userinput">/var/lib/arvados/git/repositories</span>
commit 0c96fc858f1f4954d997f2065550087dcc7f8bec
Author: Tom Clegg <tom at curoverse.com>
Date: Tue Aug 4 00:46:00 2015 -0400
6663: Fix wrong bash prompt in example transcript.
diff --git a/doc/install/create-standard-objects.html.textile.liquid b/doc/install/create-standard-objects.html.textile.liquid
index 8391e28..96a8389 100644
--- a/doc/install/create-standard-objects.html.textile.liquid
+++ b/doc/install/create-standard-objects.html.textile.liquid
@@ -37,7 +37,7 @@ In a couple of minutes, your arvados-git-sync cron job will create an empty repo
<notextile>
<pre><code>~$ <span class="userinput">cd /tmp</span>
/tmp$ <span class="userinput">git clone --bare https://github.com/curoverse/arvados.git</span>
-/tmp/arvados$ <span class="userinput">git --git-dir arvados.git push https://git.<b>uuid_prefix.your.domain</b>/arvados.git '*:*'</span>
+/tmp <span class="userinput">git --git-dir arvados.git push https://git.<b>uuid_prefix.your.domain</b>/arvados.git '*:*'</span>
</code></pre>
</notextile>
commit 2e92762444af21749e1ef232aecb6eeb98b312cd
Author: Tom Clegg <tom at curoverse.com>
Date: Tue Aug 4 00:45:27 2015 -0400
6663: Update default git_repositories_dir to match install guide.
diff --git a/services/api/config/application.default.yml b/services/api/config/application.default.yml
index ba8aeed..51bc4f9 100644
--- a/services/api/config/application.default.yml
+++ b/services/api/config/application.default.yml
@@ -69,7 +69,7 @@ common:
# able to submit crunch jobs. To pass the test suites, put a clone
# of the arvados tree in {git_repositories_dir}/arvados.git or
# {git_repositories_dir}/arvados/.git
- git_repositories_dir: /var/lib/gitolite/repositories
+ git_repositories_dir: /var/lib/arvados/git/repositories
# This is a (bare) repository that stores commits used in jobs. When a job
# runs, the source commits are first fetched into this repository, then this
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list