[ARVADOS] updated: 1.3.0-3284-g244e797ae
Git user
git at public.arvados.org
Wed Oct 14 13:44:08 UTC 2020
Summary of changes:
doc/install/install-using-salt.html.textile.liquid | 72 ++++++++++++++++++----
tools/salt-install/README.md | 20 ++++++
tools/salt-install/Vagrantfile | 8 +--
tools/salt-install/{bin => }/provision.sh | 42 ++++++-------
4 files changed, 106 insertions(+), 36 deletions(-)
create mode 100644 tools/salt-install/README.md
rename tools/salt-install/{bin => }/provision.sh (77%)
via 244e797aed6f895dcdfe72ed90b31f3fe4cf6f85 (commit)
via 94bece8122a20f8c06bb043779725fa319f1e3ca (commit)
from 1021d0b3c6bef69d30871093ce535987f2e6fdfa (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
commit 244e797aed6f895dcdfe72ed90b31f3fe4cf6f85
Author: Javier Bértoli <jbertoli at curii.com>
Date: Wed Oct 14 10:43:32 2020 -0300
feat(docs): salt install documentation
refs #16471 and #16992
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli at curii.com>
diff --git a/doc/install/install-using-salt.html.textile.liquid b/doc/install/install-using-salt.html.textile.liquid
index 294ef89be..d57658ab7 100644
--- a/doc/install/install-using-salt.html.textile.liquid
+++ b/doc/install/install-using-salt.html.textile.liquid
@@ -11,31 +11,41 @@ SPDX-License-Identifier: CC-BY-SA-3.0
# "For the impatient (tl;rl)":#tlrl
# "Introduction":#introduction
+# "Install Saltstack":#saltstack
# "Install dependencies":#dependencies
+# "Install Arvados":#arvados
h2(#tlrl). For the impatient (TL;RL)
-We added a "script"::https://github.com/arvados/arvados/tree/master/tools/salt-install/bin/provision.sh
-and example configuration files under the "tools/salt-install":https://github.com/arvados/arvados/tree/master/tools/salt-install
-directory, which you can use to get all the Arvados components up and running
-in a single node (The whole directory can be used to create a local VM with all
-Arvados' components in it, using "Vagrant":https://www.vagrantup.com/ )
+We added a "script":https://github.com/arvados/arvados/tree/master/tools/salt-install/provision.sh
+under the "tools/salt-install":https://github.com/arvados/arvados/tree/master/tools/salt-install directory.
-Just edit the parameters in the "provision.sh" script and run
+Just edit the @provision.sh@ script, modify the variables at the top of the file to suit your needs
+(usually not required), copy the file to the host where you want to install Arvados and run it as root
<notextile>
<pre><code>
-vagrant up
+bash ./provision.sh
+</code></pre>
</notextile>
-If you want to run these scripts in another host or instance, just copy the
-directory to a temporary directory, edit it and run
+It will install all the main Arvados components to get you up and running.
+
+There is a companion @Vagrantfile@ to install Arvados in a local Vagrant virtual machine. If you have
+"Vagrant":https://www.vagrantup.com/ installed in your computer, you can just get Arvados installed
+running in a local virtual machine with the command
<notextile>
<pre><code>
-bash ./bin/provision.sh
+vagrant up
+</code></pre>
</notextile>
+If you want to run these scripts in another host or instance, just copy the @provision.sh@ file to the
+desired host and run it.
+
+The rest of the document is a description of the individual steps the @provision.sh@ script perform.
+
h2(#introduction). Introduction
To ease the installation of the various Arvados components, we have developed a
@@ -47,7 +57,47 @@ Saltstack is a Python-based, open-source software for event-driven IT
automation, remote task execution, and configuration management. It can be used
in a master/minion setup or master-less.
+h2(#saltstack). Install Saltsack
+
+If you already have a Saltstack environment you can skip this section.
+
+The simplest way to get Salt up and running on a node is to use the bootstrap script they provide:
+
+<notextile>
+<pre><code>
+curl -L https://bootstrap.saltstack.com -o /tmp/bootstrap_salt.sh
+sudo sh /tmp/bootstrap_salt.sh -XUdfP -x python3
+</code></pre>
+</notextile>
+
+For more information check "Saltstack's documentation":https://docs.saltstack.com/en/latest/topics/installation/index.html
+
h2(#dependencies). Install dependencies
-Arvados depends in a few applications (postgresql, nginx) that can also be installed using Salt.
+Arvados depends in a few applications and packages (postgresql, nginx+passenger, ruby) that can also be
+installed using their respective Saltstack formulas.
+
+The formulas we use are:
+
+* "postgres":https://github.com/saltstack-formulas/postgres-formula.git
+* "nginx":https://github.com/saltstack-formulas/nginx-formula.git
+* "docker":https://github.com/saltstack-formulas/docker-formula.git
+* "locale":https://github.com/saltstack-formulas/locale-formula.git
+
+There are example Salt pillar files for each of those formulas in the
+test/salt/pillar/examples":https://github.com/saltstack-formulas/arvados-formula/tree/master/test/salt/pillar/examples
+directory. As they are, they allow you to get all the main Arvados components up and running
+in a single node.
+
+h2(#arvados). Install Arvados
+
+The Arvados formula we maintain is located in the Saltstack's community repository of
+formulas:
+
+* "arvados-formula":https://github.com/saltstack-formulas/arvados-formula.git
+
+and we keep a @development@ version at our own repo
+
+* "arvados-formula develpment":https://github.com/arvados/arvados-formula.git
+This last one might break from time to time, as we try and add new features. Use with caution.
diff --git a/tools/salt-install/Vagrantfile b/tools/salt-install/Vagrantfile
index d91336fd7..36f682761 100644
--- a/tools/salt-install/Vagrantfile
+++ b/tools/salt-install/Vagrantfile
@@ -16,7 +16,7 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
arv.vm.hostname = "arv.local"
# Networking
arv.vm.network "forwarded_port", guest: 8443, host: 8443
- arv.vm.network "forwarded_port", guest: 25101, host: 25101
+ arv.vm.network "forwarded_port", guest: 25100, host: 25100
arv.vm.network "forwarded_port", guest: 9002, host: 9002
arv.vm.network "forwarded_port", guest: 9000, host: 9000
arv.vm.network "forwarded_port", guest: 8900, host: 8900
@@ -25,9 +25,9 @@ Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
arv.vm.network "forwarded_port", guest: 8000, host: 8000
arv.vm.network "forwarded_port", guest: 3001, host: 3001
# config.vm.network "private_network", ip: "192.168.33.10"
- arv.vm.synced_folder "salt_pillars", "/srv/pillars",
- create: true
+ # arv.vm.synced_folder "salt_pillars", "/srv/pillars",
+ # create: true
arv.vm.provision "shell",
- path: "bin/provision.sh"
+ path: "provision.sh"
end
end
diff --git a/tools/salt-install/provision.sh b/tools/salt-install/provision.sh
index 0cae78b35..228b50664 100755
--- a/tools/salt-install/provision.sh
+++ b/tools/salt-install/provision.sh
@@ -10,15 +10,24 @@
#
# vagrant up
-# This could have been done with the Salt vagrant provisioner, but this script
-# can be used in environments other than vagrant.
+##########################################################
+# The 5 letters name you want to give your cluster
+CLUSTER="arva2"
+DOMAIN="arv.local"
+# Which release of Arvados repo you want to use
+RELEASE="production"
+# Which version of Arvados you want to install. Defaults to 'latest'
+# in the desired repo
+# VERSION="2.0.4"
+
+# This is a arvados-formula setting.
# If branch is set, the script will switch to it before running salt
-BRANCH="refactor-config-add-service"
-RELEASE="testing"
+# Usually not needed, only used for testing
+# BRANCH="master"
-CLUSTER="arva2"
-DOMAIN="arv.local"
+##########################################################
+# Usually there's no need to modify things below this line
# Salt's dir
## states
@@ -60,6 +69,7 @@ mkdir -p ${S_DIR}
mkdir -p ${F_DIR}
mkdir -p ${P_DIR}
+# States
cat > ${S_DIR}/top.sls << EOFTSLS
base:
'*':
@@ -71,6 +81,7 @@ base:
- arvados
EOFTSLS
+# Pillars
cat > ${P_DIR}/top.sls << EOFPSLS
base:
'*':
@@ -92,7 +103,7 @@ EOFPSLS
# Get the formula and dependencies
cd ${F_DIR} || exit 1
for f in postgres arvados nginx docker locale; do
- git clone https://github.com/netmanagers/${f}-formula.git
+ git clone https://github.com/saltstack-formulas/${f}-formula.git
done
if [ "x${BRANCH}" != "x" ]; then
@@ -101,7 +112,7 @@ if [ "x${BRANCH}" != "x" ]; then
cd -
fi
-sed "s/example.net/${DOMAIN}/g; s/fixme/${CLUSTER}/g; s/release: development/release: ${RELEASE}/g" \
+sed "s/example.net/${DOMAIN}/g; s/fixme/${CLUSTER}/g; s/release: development/release: ${RELEASE}/g; s/# version: '2.0.4'/version: '${VERSION}'/g" \
${F_DIR}/arvados-formula/test/salt/pillar/arvados_dev.sls > ${P_DIR}/arvados.sls
# Replace cluster and domain name in the example pillars
@@ -110,22 +121,9 @@ for f in ${F_DIR}/arvados-formula/test/salt/pillar/examples/*; do
${f} > ${P_DIR}/$(basename ${f})
done
-# # Copy arvados' pillar.example file to the pillars dir, so it's used
-# sed "s/example.net/${DOMAIN}/g" ${F_DIR}/arvados-formula/pillar.example > ${P_DIR}/arvados.sls
-#
-# # Replace domain name in the example pillars
-# for f in ${F_DIR}/arvados-formula/test/salt/pillar/examples/*; do
-# sed "s/example.net/${DOMAIN}/g" ${f} > ${P_DIR}/$(basename ${f})
-# done
-#
# Let's write a /etc/hosts file that points all the hosts to localhost
echo "127.0.0.2 api keep keep0 collections download ws workbench workbench2 ${CLUSTER}.${DOMAIN} api.${CLUSTER}.${DOMAIN} keep.${CLUSTER}.${DOMAIN} keep0.${CLUSTER}.${DOMAIN} collections.${CLUSTER}.${DOMAIN} download.${CLUSTER}.${DOMAIN} ws.${CLUSTER}.${DOMAIN} workbench.${CLUSTER}.${DOMAIN} workbench2.${CLUSTER}.${DOMAIN}" >> /etc/hosts
-# FIXME! Test to see if arvados-api-server progresses without issues
-cat > /root/.psqlrc << EOF
-\pset pager off
-EOF
-
# Now run the install
salt-call --local state.apply -l debug
commit 94bece8122a20f8c06bb043779725fa319f1e3ca
Author: Javier Bértoli <jbertoli at curii.com>
Date: Mon Oct 12 12:25:16 2020 -0300
fix(salt): choose repo
refs #16471 and #16992
Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli at curii.com>
diff --git a/tools/salt-install/README.md b/tools/salt-install/README.md
new file mode 100644
index 000000000..3175224d0
--- /dev/null
+++ b/tools/salt-install/README.md
@@ -0,0 +1,20 @@
+[comment]: # (Copyright © The Arvados Authors. All rights reserved.)
+[comment]: # ()
+[comment]: # (SPDX-License-Identifier: CC-BY-SA-3.0)
+
+# Arvados install with Saltstack
+
+##### About
+
+This directory holds a small script to install Arvados on a single node, using the
+[Saltstack arvados-formula](https://github.com/saltstack-formulas/arvados-formula)
+in master-less mode.
+
+The fastest way to get it running is to modify the first lines in the `provision.sh`
+script to suit your needs, copy it in the host where you want to install Arvados
+and run it as root.
+
+There's an example `Vagrantfile` also, to install it in a vagrant box if you want
+to try it locally.
+
+For more information, please read https://doc.arvados.org/v2.1/install/install-using-salt.html
diff --git a/tools/salt-install/bin/provision.sh b/tools/salt-install/provision.sh
similarity index 95%
rename from tools/salt-install/bin/provision.sh
rename to tools/salt-install/provision.sh
index d47013018..0cae78b35 100755
--- a/tools/salt-install/bin/provision.sh
+++ b/tools/salt-install/provision.sh
@@ -15,6 +15,7 @@
# If branch is set, the script will switch to it before running salt
BRANCH="refactor-config-add-service"
+RELEASE="testing"
CLUSTER="arva2"
DOMAIN="arv.local"
@@ -79,8 +80,9 @@ base:
- nginx_controller_configuration
- nginx_keepproxy_configuration
- nginx_keepweb_configuration
- - nginx_passenger
+ - nginx_passenger
- nginx_websocket_configuration
+ - nginx_webshell_configuration
- nginx_workbench2_configuration
- nginx_workbench_configuration
- postgresql
@@ -99,7 +101,7 @@ if [ "x${BRANCH}" != "x" ]; then
cd -
fi
-sed "s/example.net/${DOMAIN}/g; s/fixme/${CLUSTER}/g" \
+sed "s/example.net/${DOMAIN}/g; s/fixme/${CLUSTER}/g; s/release: development/release: ${RELEASE}/g" \
${F_DIR}/arvados-formula/test/salt/pillar/arvados_dev.sls > ${P_DIR}/arvados.sls
# Replace cluster and domain name in the example pillars
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list