[ARVADOS] created: 2.1.0-181-g7b009edfb
Git user
git at public.arvados.org
Fri Dec 4 12:40:59 UTC 2020
at 7b009edfb17684630028f2277efa5201f11350fa (commit)
commit 7b009edfb17684630028f2277efa5201f11350fa
Author: Javier Bértoli <jbertoli at curii.com>
Date: Fri Dec 4 09:17:34 2020 -0300
fix(provision): Document CA certificate purpose and installation
refs #17177
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli at curii.com>
diff --git a/doc/install/arvbox.html.textile.liquid b/doc/install/arvbox.html.textile.liquid
index c01ec61fa..3c77ade8d 100644
--- a/doc/install/arvbox.html.textile.liquid
+++ b/doc/install/arvbox.html.textile.liquid
@@ -80,10 +80,23 @@ Arvbox creates root certificate to authorize Arvbox services. Installing the ro
The certificate will be added under the "Arvados testing" organization as "arvbox testing root CA".
-To access your Arvbox instance using command line clients (such as arv-get and arv-put) without security errors, install the certificate into the OS certificate storage (instructions for Debian/Ubuntu):
+To access your Arvbox instance using command line clients (such as arv-get and arv-put) without security errors, install the certificate into the OS certificate storage.
-# copy @arvbox-root-cert.pem@ to @/usr/local/share/ca-certificates/@
-# run @/usr/sbin/update-ca-certificates@
+h3. On Debian/Ubuntu:
+
+<notextile>
+<pre><code>cp arvbox-root-cert.pem /usr/local/share/ca-certificates/
+/usr/sbin/update-ca-certificates
+</code></pre>
+</notextile>
+
+h3. On CentOS:
+
+<notextile>
+<pre><code>cp arvbox-root-cert.pem /etc/pki/ca-trust/source/anchors/
+/usr/bin/update-ca-trust
+</code></pre>
+</notextile>
h2. Configs
diff --git a/doc/install/salt-single-host.html.textile.liquid b/doc/install/salt-single-host.html.textile.liquid
index fb41d59ee..5bed6d05e 100644
--- a/doc/install/salt-single-host.html.textile.liquid
+++ b/doc/install/salt-single-host.html.textile.liquid
@@ -11,7 +11,9 @@ SPDX-License-Identifier: CC-BY-SA-3.0
# "Install Saltstack":#saltstack
# "Single host install using the provision.sh script":#single_host
-# "DNS configuration":#final_steps
+# "Final steps":#final_steps
+## "DNS configuration":#dns_configuration
+## "Install root certificate":#ca_root_certificate
# "Initial user and login":#initial_user
# "Test the installed cluster running a simple workflow":#test_install
@@ -49,7 +51,9 @@ arvados: Failed: 0
</code></pre>
</notextile>
-h2(#final_steps). DNS configuration
+h2(#final_steps). Final configuration steps
+
+h3(#dns_configuration). DNS configuration
After the setup is done, you need to set up your DNS to be able to access the cluster.
@@ -65,6 +69,39 @@ echo "${HOST_IP} api keep keep0 collections download ws workbench workbench2 ${C
</code></pre>
</notextile>
+h3(#ca_root_certificate). Install root certificate
+
+Arvados uses SSL to encrypt communications. Its UI uses AJAX which will silently fail if the certificate is not valid or signed by an unknown Certification Authority.
+
+For this reason, the @arvados-formula@ has a helper state to create a root certificate to authorize Arvados services. The @provision.sh@ script will leave a copy of the generated CA's certificate (@arvados-snakeoil-ca.pem@) in the script's directory so ypu can add it to your workstation.
+
+Installing the root certificate into your web browser will prevent security errors when accessing Arvados services with your web browser.
+
+# Go to the certificate manager in your browser.
+#* In Chrome, this can be found under "Settings → Advanced → Manage Certificates" or by entering @chrome://settings/certificates@ in the URL bar.
+#* In Firefox, this can be found under "Preferences → Privacy & Security" or entering @about:preferences#privacy@ in the URL bar and then choosing "View Certificates...".
+# Select the "Authorities" tab, then press the "Import" button. Choose @arvados-snakeoil-ca.pem@
+
+The certificate will be added under the "Arvados Formula".
+
+To access your Arvados instance using command line clients (such as arv-get and arv-put) without security errors, install the certificate into the OS certificate storage.
+
+* On Debian/Ubuntu:
+
+<notextile>
+<pre><code>cp arvados-root-cert.pem /usr/local/share/ca-certificates/
+/usr/sbin/update-ca-certificates
+</code></pre>
+</notextile>
+
+* On CentOS:
+
+<notextile>
+<pre><code>cp arvados-root-cert.pem /etc/pki/ca-trust/source/anchors/
+/usr/bin/update-ca-trust
+</code></pre>
+</notextile>
+
h2(#initial_user). Initial user and login
At this point you should be able to log into the Arvados cluster.
diff --git a/doc/install/salt-vagrant.html.textile.liquid b/doc/install/salt-vagrant.html.textile.liquid
index d9aa791f0..ed0d5bca6 100644
--- a/doc/install/salt-vagrant.html.textile.liquid
+++ b/doc/install/salt-vagrant.html.textile.liquid
@@ -10,7 +10,9 @@ SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
# "Vagrant":#vagrant
-# "DNS configuration":#final_steps
+# "Final steps":#final_steps
+## "DNS configuration":#dns_configuration
+## "Install root certificate":#ca_root_certificate
# "Initial user and login":#initial_user
# "Test the installed cluster running a simple workflow":#test_install
@@ -37,7 +39,9 @@ If you want to reconfigure the running box, you can just:
</code></pre>
</notextile>
-h2(#final_steps). DNS configuration
+h2(#final_steps). Final configuration steps
+
+h3(#dns_configuration). DNS configuration
After the setup is done, you need to set up your DNS to be able to access the cluster.
@@ -53,6 +57,39 @@ echo "${HOST_IP} api keep keep0 collections download ws workbench workbench2 ${C
</code></pre>
</notextile>
+h3(#ca_root_certificate). Install root certificate
+
+Arvados uses SSL to encrypt communications. Its UI uses AJAX which will silently fail if the certificate is not valid or signed by an unknown Certification Authority.
+
+For this reason, the @arvados-formula@ has a helper state to create a root certificate to authorize Arvados services. The @provision.sh@ script will leave a copy of the generated CA's certificate (@arvados-snakeoil-ca.pem@) in the script's directory so ypu can add it to your workstation.
+
+Installing the root certificate into your web browser will prevent security errors when accessing Arvados services with your web browser.
+
+# Go to the certificate manager in your browser.
+#* In Chrome, this can be found under "Settings → Advanced → Manage Certificates" or by entering @chrome://settings/certificates@ in the URL bar.
+#* In Firefox, this can be found under "Preferences → Privacy & Security" or entering @about:preferences#privacy@ in the URL bar and then choosing "View Certificates...".
+# Select the "Authorities" tab, then press the "Import" button. Choose @arvados-snakeoil-ca.pem@
+
+The certificate will be added under the "Arvados Formula".
+
+To access your Arvados instance using command line clients (such as arv-get and arv-put) without security errors, install the certificate into the OS certificate storage.
+
+* On Debian/Ubuntu:
+
+<notextile>
+<pre><code>cp arvados-root-cert.pem /usr/local/share/ca-certificates/
+/usr/sbin/update-ca-certificates
+</code></pre>
+</notextile>
+
+* On CentOS:
+
+<notextile>
+<pre><code>cp arvados-root-cert.pem /etc/pki/ca-trust/source/anchors/
+/usr/bin/update-ca-trust
+</code></pre>
+</notextile>
+
h2(#initial_user). Initial user and login
At this point you should be able to log into the Arvados cluster.
diff --git a/tools/salt-install/provision.sh b/tools/salt-install/provision.sh
index 9aa5f19b1..7b4fc9da3 100755
--- a/tools/salt-install/provision.sh
+++ b/tools/salt-install/provision.sh
@@ -258,7 +258,7 @@ fi
# END FIXME! #16992 Temporary fix for psql call in arvados-api-server
# Leave a copy of the Arvados CA so the user can copy it where it's required
-echo "Copying the Arvados CA file to the installer dir, so you can import it"
+echo "Copying the Arvados CA certificate to the installer dir, so you can import it"
# If running in a vagrant VM, also add default user to docker group
if [ "x${VAGRANT}" = "xyes" ]; then
cp /etc/ssl/certs/arvados-snakeoil-ca.pem /vagrant
commit 057cf02deb6812d3f8e57843dfe6e03c0bda7198
Author: Javier Bértoli <jbertoli at curii.com>
Date: Thu Dec 3 20:00:46 2020 -0300
fix(provision): Add a CA and sign certificates with it
refs #17177
As discussed [here](https://forum.arvados.org/t/debugging-arvados-deployed-with-salt/58/8)
and [here](https://gitter.im/arvados/community?at=5fc65683496ca3372e3474a3), Arvados needs
certs signed by a known CA to work correctly.
This PR adds a CA and leaves a copy of the certificate in the installer directory.
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli at curii.com>
diff --git a/tools/salt-install/Vagrantfile b/tools/salt-install/Vagrantfile
index ed3466dde..1f587296b 100644
--- a/tools/salt-install/Vagrantfile
+++ b/tools/salt-install/Vagrantfile
@@ -33,6 +33,7 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
arv.vm.provision "shell",
path: "provision.sh",
args: [
+ "--debug",
"--test",
"--vagrant",
"--ssl-port=8443"
diff --git a/tools/salt-install/provision.sh b/tools/salt-install/provision.sh
index a207d0198..9aa5f19b1 100755
--- a/tools/salt-install/provision.sh
+++ b/tools/salt-install/provision.sh
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/bin/bash -x
# Copyright (C) The Arvados Authors. All rights reserved.
#
@@ -139,7 +139,7 @@ file_roots:
base:
- ${S_DIR}
- ${F_DIR}/*
- - ${F_DIR}/*/test/salt/states
+ - ${F_DIR}/*/test/salt/states/examples
pillar_roots:
base:
@@ -154,8 +154,8 @@ mkdir -p ${P_DIR}
cat > ${S_DIR}/top.sls << EOFTSLS
base:
'*':
- - example_single_host_host_entries
- - example_add_snakeoil_certs
+ - single_host.host_entries
+ - single_host.snakeoil_certs
- locale
- nginx.passenger
- postgres
@@ -182,7 +182,6 @@ base:
- postgresql
EOFPSLS
-
# Get the formula and dependencies
cd ${F_DIR} || exit 1
for f in postgres arvados nginx docker locale; do
@@ -258,9 +257,16 @@ if [ "x${RESTORE_PSQL}" = "xyes" ]; then
fi
# END FIXME! #16992 Temporary fix for psql call in arvados-api-server
-# If running in a vagrant VM, add default user to docker group
+# Leave a copy of the Arvados CA so the user can copy it where it's required
+echo "Copying the Arvados CA file to the installer dir, so you can import it"
+# If running in a vagrant VM, also add default user to docker group
if [ "x${VAGRANT}" = "xyes" ]; then
- usermod -a -G docker vagrant
+ cp /etc/ssl/certs/arvados-snakeoil-ca.pem /vagrant
+
+ echo "Adding the vagrant user to the docker group"
+ usermod -a -G docker vagrant
+else
+ cp /etc/ssl/certs/arvados-snakeoil-ca.pem ${SCRIPT_DIR}
fi
# Test that the installation finished correctly
diff --git a/tools/salt-install/single_host/arvados.sls b/tools/salt-install/single_host/arvados.sls
index dffd6575e..a06244270 100644
--- a/tools/salt-install/single_host/arvados.sls
+++ b/tools/salt-install/single_host/arvados.sls
@@ -73,7 +73,7 @@ arvados:
tls:
# certificate: ''
# key: ''
- # required to test with snakeoil certs
+ # required to test with arvados-snakeoil certs
insecure: true
### TOKENS
diff --git a/tools/salt-install/single_host/nginx_controller_configuration.sls b/tools/salt-install/single_host/nginx_controller_configuration.sls
index 7c99d2dea..96fc383d7 100644
--- a/tools/salt-install/single_host/nginx_controller_configuration.sls
+++ b/tools/salt-install/single_host/nginx_controller_configuration.sls
@@ -53,7 +53,7 @@ nginx:
- proxy_set_header: 'X-Forwarded-For $proxy_add_x_forwarded_for'
- proxy_set_header: 'X-External-Client $external_client'
# - include: 'snippets/letsencrypt.conf'
- - include: 'snippets/snakeoil.conf'
+ - include: 'snippets/arvados-snakeoil.conf'
- access_log: /var/log/nginx/__CLUSTER__.__DOMAIN__.access.log combined
- error_log: /var/log/nginx/__CLUSTER__.__DOMAIN__.error.log
- client_max_body_size: 128m
diff --git a/tools/salt-install/single_host/nginx_keepproxy_configuration.sls b/tools/salt-install/single_host/nginx_keepproxy_configuration.sls
index fc4854e5a..61c138474 100644
--- a/tools/salt-install/single_host/nginx_keepproxy_configuration.sls
+++ b/tools/salt-install/single_host/nginx_keepproxy_configuration.sls
@@ -53,6 +53,6 @@ nginx:
- proxy_http_version: '1.1'
- proxy_request_buffering: 'off'
# - include: 'snippets/letsencrypt.conf'
- - include: 'snippets/snakeoil.conf'
+ - include: 'snippets/arvados-snakeoil.conf'
- access_log: /var/log/nginx/keepproxy.__CLUSTER__.__DOMAIN__.access.log combined
- error_log: /var/log/nginx/keepproxy.__CLUSTER__.__DOMAIN__.error.log
diff --git a/tools/salt-install/single_host/nginx_keepweb_configuration.sls b/tools/salt-install/single_host/nginx_keepweb_configuration.sls
index 513c0393e..88083e3c5 100644
--- a/tools/salt-install/single_host/nginx_keepweb_configuration.sls
+++ b/tools/salt-install/single_host/nginx_keepweb_configuration.sls
@@ -53,6 +53,6 @@ nginx:
- proxy_http_version: '1.1'
- proxy_request_buffering: 'off'
# - include: 'snippets/letsencrypt.conf'
- - include: 'snippets/snakeoil.conf'
+ - include: 'snippets/arvados-snakeoil.conf'
- access_log: /var/log/nginx/collections.__CLUSTER__.__DOMAIN__.access.log combined
- error_log: /var/log/nginx/collections.__CLUSTER__.__DOMAIN__.error.log
diff --git a/tools/salt-install/single_host/nginx_webshell_configuration.sls b/tools/salt-install/single_host/nginx_webshell_configuration.sls
index 495de82d2..80e9f57d6 100644
--- a/tools/salt-install/single_host/nginx_webshell_configuration.sls
+++ b/tools/salt-install/single_host/nginx_webshell_configuration.sls
@@ -69,7 +69,7 @@ nginx:
- add_header: "'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type'"
# - include: 'snippets/letsencrypt.conf'
- - include: 'snippets/snakeoil.conf'
+ - include: 'snippets/arvados-snakeoil.conf'
- access_log: /var/log/nginx/webshell.__CLUSTER__.__DOMAIN__.access.log combined
- error_log: /var/log/nginx/webshell.__CLUSTER__.__DOMAIN__.error.log
diff --git a/tools/salt-install/single_host/nginx_websocket_configuration.sls b/tools/salt-install/single_host/nginx_websocket_configuration.sls
index 1848a8737..60d757f89 100644
--- a/tools/salt-install/single_host/nginx_websocket_configuration.sls
+++ b/tools/salt-install/single_host/nginx_websocket_configuration.sls
@@ -54,6 +54,6 @@ nginx:
- proxy_http_version: '1.1'
- proxy_request_buffering: 'off'
# - include: 'snippets/letsencrypt.conf'
- - include: 'snippets/snakeoil.conf'
+ - include: 'snippets/arvados-snakeoil.conf'
- access_log: /var/log/nginx/ws.__CLUSTER__.__DOMAIN__.access.log combined
- error_log: /var/log/nginx/ws.__CLUSTER__.__DOMAIN__.error.log
diff --git a/tools/salt-install/single_host/nginx_workbench2_configuration.sls b/tools/salt-install/single_host/nginx_workbench2_configuration.sls
index 733397adf..4a0190ad1 100644
--- a/tools/salt-install/single_host/nginx_workbench2_configuration.sls
+++ b/tools/salt-install/single_host/nginx_workbench2_configuration.sls
@@ -44,6 +44,6 @@ nginx:
- location /config.json:
- return: {{ "200 '" ~ '{"API_HOST":"__CLUSTER__.__DOMAIN__"}' ~ "'" }}
# - include: 'snippets/letsencrypt.conf'
- - include: 'snippets/snakeoil.conf'
+ - include: 'snippets/arvados-snakeoil.conf'
- access_log: /var/log/nginx/workbench2.__CLUSTER__.__DOMAIN__.access.log combined
- error_log: /var/log/nginx/workbench2.__CLUSTER__.__DOMAIN__.error.log
diff --git a/tools/salt-install/single_host/nginx_workbench_configuration.sls b/tools/salt-install/single_host/nginx_workbench_configuration.sls
index 9a382e777..6a17ee745 100644
--- a/tools/salt-install/single_host/nginx_workbench_configuration.sls
+++ b/tools/salt-install/single_host/nginx_workbench_configuration.sls
@@ -55,7 +55,7 @@ nginx:
- proxy_set_header: 'X-Real-IP $remote_addr'
- proxy_set_header: 'X-Forwarded-For $proxy_add_x_forwarded_for'
# - include: 'snippets/letsencrypt.conf'
- - include: 'snippets/snakeoil.conf'
+ - include: 'snippets/arvados-snakeoil.conf'
- access_log: /var/log/nginx/workbench.__CLUSTER__.__DOMAIN__.access.log combined
- error_log: /var/log/nginx/workbench.__CLUSTER__.__DOMAIN__.error.log
diff --git a/tools/salt-install/tests/run-test.sh b/tools/salt-install/tests/run-test.sh
index cf61d92b5..8d9de6fdf 100755
--- a/tools/salt-install/tests/run-test.sh
+++ b/tools/salt-install/tests/run-test.sh
@@ -7,6 +7,15 @@ export ARVADOS_API_TOKEN=changemesystemroottoken
export ARVADOS_API_HOST=__CLUSTER__.__DOMAIN__:__HOST_SSL_PORT__
export ARVADOS_API_HOST_INSECURE=true
+set -o pipefail
+
+# First, validate that the CA is installed and that we can query it with no errors.
+if ! curl -s -o /dev/null https://workbench.${ARVADOS_API_HOST}/users/welcome?return_to=%2F; then
+ echo "The Arvados CA was not correctly installed. Although some components will work,"
+ echo "others won't. Please verify that the CA cert file was installed correctly and"
+ echo "retry running these tests."
+ exit 1
+fi
# https://doc.arvados.org/v2.0/install/install-jobs-image.html
echo "Creating Arvados Standard Docker Images project"
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list