[ARVADOS] updated: 2.1.0-470-g708544a82

Git user git at public.arvados.org
Wed Mar 31 12:40:50 UTC 2021


Summary of changes:
 tools/terraform/api_instance.tf       |  8 ++++----
 tools/terraform/db_instance.tf        |  5 +++--
 tools/terraform/keepproxy_instance.tf | 16 ++++++++--------
 tools/terraform/keepstore_instance.tf |  4 ++--
 tools/terraform/shell_instance.tf     |  6 +++---
 tools/terraform/workbench_instance.tf |  8 ++++----
 6 files changed, 24 insertions(+), 23 deletions(-)

  discards  375a1363be4b94b9682c0824b762033d58d608eb (commit)
       via  708544a82c4524f7ca7ca899ef272453c754ef97 (commit)

This update added new revisions after undoing existing revisions.  That is
to say, the old revision is not a strict subset of the new revision.  This
situation occurs when you --force push a change and generate a repository
containing something like this:

 * -- * -- B -- O -- O -- O (375a1363be4b94b9682c0824b762033d58d608eb)
            \
             N -- N -- N (708544a82c4524f7ca7ca899ef272453c754ef97)

When this happens we assume that you've already had alert emails for all
of the O revisions, and so we here report only the revisions in the N
branch from the common base, B.

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.


commit 708544a82c4524f7ca7ca899ef272453c754ef97
Author: Javier Bértoli <jbertoli at curii.com>
Date:   Wed Mar 24 10:07:01 2021 -0300

    feat(deployment): add initial terraform code for AWS deployment
    
    refs #17450
    
    Arvados-DCO-1.1-Signed-off-by: Javier Bértoli <jbertoli at curii.com>

diff --git a/doc/install/salt-multi-host.html.textile.liquid b/doc/install/salt-multi-host.html.textile.liquid
index 709c32e2a..0bba34427 100644
--- a/doc/install/salt-multi-host.html.textile.liquid
+++ b/doc/install/salt-multi-host.html.textile.liquid
@@ -9,90 +9,177 @@ Copyright (C) The Arvados Authors. All rights reserved.
 SPDX-License-Identifier: CC-BY-SA-3.0
 {% endcomment %}
 
-# "Install Saltstack":#saltstack
-# "Install dependencies":#dependencies
-# "Install Arvados using Saltstack":#saltstack
-# "DNS configuration":#final_steps
+# "Hosts preparation":#hosts_preparation
+## "Hosts setup using terraform (experimental)":#hosts_setup_using_terraform
+# "Multi host install using the provision.sh script":#multi_host
+# "Choose the desired configuration":#choose_configuration
+## "Multi host / multi hostname":#multi_host_multi_hostnames
+## "Multi host / multiple hostnames (Alternative configuration)":#multi_host_multiple_hostnames
+## "Further customization of the installation (modifying the salt pillars and states)":#further_customization
+# "Run the provision.sh script":#run_provision_script
+# "Final configuration steps":#final_steps
+## "Install the CA root certificate (required in both alternatives)":#ca_root_certificate
+## "DNS configuration (multi host / multiple hostnames)":#multi_host_multiple_hostnames_dns_configuration
 # "Initial user and login":#initial_user
+# "Test the installed cluster running a simple workflow":#test_install
 
-h2(#saltstack). Install Saltstack
+h2(#hosts_preparation). Hosts preparation
 
-If you already have a Saltstack environment you can skip this section.
+In order to run Arvados on a multi-host installation, there are a few requirements that your infrastructure has to fulfill.
 
-The simplest way to get Salt up and running on a node is to use the bootstrap script they provide:
+These instructions explain how to setup a multi-host environment that is suitable for production use of Arvados.
 
+We suggest distributing the Arvados components in the following way, given at least 6 hosts:
+
+# Database server:
+## postgresql server
+# API node:
+## arvados api server
+## arvados controller
+## arvados websocket
+## arvados cloud dispatcher
+# WORKBENCH node:
+## arvados workbench
+## arvados workbench2
+# KEEPPROXY node:
+## arvados keepproxy
+## arvados keepweb
+# KEEPSTOREs (at least 2)
+## arvados keepstore
+# SHELL node (optional):
+## arvados shell
+
+Note that these hosts can be virtual machines in your infrastructure and they don't need to be real machines.
+
+h3(#hosts_setup_using_terraform). Hosts setup using terraform (experimental)
+
+
+
+
+
+
+
+
+
+h2(#multi_host). Multi host install using the provision.sh script
+
+<b>NOTE: The multi host installation is not recommended for production use.</b>
+
+This is a package-based installation method. The Salt scripts are available from the "tools/salt-install":https://github.com/arvados/arvados/tree/master/tools/salt-install directory in the Arvados git repository.
+
+This procedure will install all the main Arvados components to get you up and running in a multi host. The whole installation procedure takes somewhere between 15 to 60 minutes, depending on the host resources and its network bandwidth. As a reference, on a virtual machine with 1 core and 1 GB RAM, it takes ~25 minutes to do the initial install.
+
+We suggest you to use the @provision.sh@ script to deploy Arvados, which is implemented with the @arvados-formula@ in a Saltstack master-less setup. After setting up a few variables in a config file (next step), you'll be ready to run it and get Arvados deployed.
+
+h2(#choose_configuration). Choose the desired configuration
+
+For documentation's sake, we will use the cluster name <i>arva2</i> and the domain <i>arv.local</i>. If you don't change them as required in the next steps, installation won't proceed.
+
+Arvados' multi host installation can be done in two fashions:
+
+* Using a multi hostname, asigning <i>a different port (other than 443) for each user-facing service</i>: This choice is easier to setup, but the user will need to know the port/s for the different services she wants to connect to.
+* Using multiple hostnames on the same IP: this setup involves a few extra steps but each service will have a meaningful hostname so it will make easier to access them later.
+
+Once you decide which of these choices you prefer, copy one the two example configuration files and directory, and edit them to suit your needs.
+
+h3(#multi_host_multi_hostnames). Multi host / multi hostname
 <notextile>
-<pre><code>curl -L https://bootstrap.saltstack.com -o /tmp/bootstrap_salt.sh
-sudo sh /tmp/bootstrap_salt.sh -XUdfP -x python3
+<pre><code>cp local.params.example.multi_host_multi_hostname local.params
+cp -r config_examples/multi_host/multi_hostname local_config_dir
 </code></pre>
 </notextile>
 
-For more information check "Saltstack's documentation":https://docs.saltstack.com/en/latest/topics/installation/index.html
+Edit the variables in the <i>local.params</i> file. Pay attention to the <b>*_PORT, *_TOKEN</b> and <b>*KEY</b> variables.
 
-h2(#dependencies). Install dependencies
+h3(#multi_host_multiple_hostnames). Multi host / multiple hostnames (Alternative configuration)
+<notextile>
+<pre><code>cp local.params.example.multi_host_multiple_hostnames local.params
+cp -r config_examples/multi_host/multiple_hostnames local_config_dir
+</code></pre>
+</notextile>
 
-Arvados depends in a few applications and packages (postgresql, nginx+passenger, ruby) that can also be installed using their respective Saltstack formulas.
+Edit the variables in the <i>local.params</i> file.
 
-The formulas we use are:
+## "Further customization of the installation (modifying the salt pillars and states)":#further_customization
 
-* "postgres":https://github.com/saltstack-formulas/postgres-formula.git
-* "nginx":https://github.com/saltstack-formulas/nginx-formula.git
-* "docker":https://github.com/saltstack-formulas/docker-formula.git
-* "locale":https://github.com/saltstack-formulas/locale-formula.git
-* "letsencrypt":https://github.com/saltstack-formulas/letsencrypt-formula.git
+If you want or need further customization, you can edit the Saltstack pillars and states files. Pay particular attention to the <i>pillars/arvados.sls</i> one. Any extra <i>state</i> file you add under <i>local_config_dir/states</i> will be added to the salt run and applied to the host.
 
-There are example Salt pillar files for each of those formulas in the "arvados-formula's test/salt/pillar/examples":https://github.com/arvados/arvados-formula/tree/master/test/salt/pillar/examples directory. As they are, they allow you to get all the main Arvados components up and running.
+h2(#run_provision_script). Run the provision.sh script
 
-h2(#saltstack). Install Arvados using Saltstack
+When you finished customizing the configuration, you are ready to copy the files to the host (if needed) and run the @provision.sh@ script:
 
-This is a package-based installation method. The Salt scripts are available from the "tools/salt-install":https://github.com/arvados/arvados/tree/master/tools/salt-install directory in the Arvados git repository.
+<notextile>
+<pre><code>scp -r provision.sh local* user at host:
+ssh user at host sudo provision.sh
+</code></pre>
+</notextile>
 
-The Arvados formula we maintain is located in Arvados' Github account and should be considered the canonical place to download its most up-to-date version:
+and wait for it to finish.
 
-* "arvados-formula":https://github.com/arvados/arvados-formula.git
+If everything goes OK, you'll get some final lines stating something like:
 
-As the Saltstack's community keeps a "repository of formulas":https://github.com/saltstack-formulas/ in Github, we also provide
+<notextile>
+<pre><code>arvados: Succeeded: 109 (changed=9)
+arvados: Failed:      0
+</code></pre>
+</notextile>
+
+h2(#final_steps). Final configuration steps
+
+Once the deployment went OK, you'll need to perform a few extra steps in your local browser/host to access the cluster.
 
-* "a copy of the formula":https://github.com/saltstack-formulas/arvados-formula.git
+h3(#ca_root_certificate). Install the CA root certificate (required in both alternatives)
 
-there, and do our best effort to keep it in sync with ours.
+Arvados uses SSL to encrypt communications. Its UI uses AJAX which will silently fail if the certificate is not valid or signed by an unknown Certification Authority.
 
-For those familiar with Saltstack, the process to get Arvados deployed is similar to any other formula:
+For this reason, the @arvados-formula@ has a helper state to create a root certificate to authorize Arvados services. The @provision.sh@ script will leave a copy of the generated CA's certificate (@arvados-snakeoil-ca.pem@) in the script's directory so you can add it to your workstation.
 
-1. Fork/copy the formula to your Salt master host.
-2. Edit the Arvados, nginx, postgres, locale and docker pillars to match your desired configuration.
-3. Run a @state.apply@ to get it deployed.
+Installing the root certificate into your web browser will prevent security errors when accessing Arvados services with your web browser.
 
-h2(#final_steps). DNS configuration
+# Go to the certificate manager in your browser.
+#* In Chrome, this can be found under "Settings → Advanced → Manage Certificates" or by entering @chrome://settings/certificates@ in the URL bar.
+#* In Firefox, this can be found under "Preferences → Privacy & Security" or entering @about:preferences#privacy@ in the URL bar and then choosing "View Certificates...".
+# Select the "Authorities" tab, then press the "Import" button.  Choose @arvados-snakeoil-ca.pem@
 
-After the setup is done, you need to set up your DNS to be able to access the cluster's nodes.
+The certificate will be added under the "Arvados Formula".
 
-The simplest way to do this is to add entries in the @/etc/hosts@ file of every host:
+To access your Arvados instance using command line clients (such as arv-get and arv-put) without security errors, install the certificate into the OS certificate storage.
+
+* On Debian/Ubuntu:
 
 <notextile>
-<pre><code>export CLUSTER="arva2"
-export DOMAIN="arv.local"
+<pre><code>cp arvados-root-cert.pem /usr/local/share/ca-certificates/
+/usr/sbin/update-ca-certificates
+</code></pre>
+</notextile>
+
+* On CentOS:
 
-echo A.B.C.a  api ${CLUSTER}.${DOMAIN} api.${CLUSTER}.${DOMAIN} >> /etc/hosts
-echo A.B.C.b  keep keep.${CLUSTER}.${DOMAIN} >> /etc/hosts
-echo A.B.C.c  keep0 keep0.${CLUSTER}.${DOMAIN} >> /etc/hosts
-echo A.B.C.d  collections collections.${CLUSTER}.${DOMAIN} >> /etc/hosts
-echo A.B.C.e  download download.${CLUSTER}.${DOMAIN} >> /etc/hosts
-echo A.B.C.f  ws ws.${CLUSTER}.${DOMAIN} >> /etc/hosts
-echo A.B.C.g  workbench workbench.${CLUSTER}.${DOMAIN} >> /etc/hosts
-echo A.B.C.h  workbench2 workbench2.${CLUSTER}.${DOMAIN}" >> /etc/hosts
+<notextile>
+<pre><code>cp arvados-root-cert.pem /etc/pki/ca-trust/source/anchors/
+/usr/bin/update-ca-trust
 </code></pre>
 </notextile>
 
-Replacing in each case de @A.B.C.x@ IP with the corresponding IP of the node.
+h3(#multi_host_multiple_hostnames_dns_configuration). DNS configuration (multi host / multiple hostnames)
+
+When using multiple hostnames, after the setup is done, you need to set up your DNS to be able to access the cluster.
 
-If your infrastructure uses another DNS service setup, add the corresponding entries accordingly.
+If you don't have access to the domain's DNS to add the required entries, the simplest way to do it is to edit your @/etc/hosts@ file (as root):
 
-h2(#initial_user). Initial user and login
+<notextile>
+<pre><code>export CLUSTER="arva2"
+export DOMAIN="arv.local"
+export HOST_IP="127.0.0.2"    # This is valid either if installing in your computer directly
+                              # or in a Vagrant VM. If you're installing it on a remote host
+                              # just change the IP to match that of the host.
+echo "${HOST_IP} api keep keep0 collections download ws workbench workbench2 ${CLUSTER}.${DOMAIN} api.${CLUSTER}.${DOMAIN} keep.${CLUSTER}.${DOMAIN} keep0.${CLUSTER}.${DOMAIN} collections.${CLUSTER}.${DOMAIN} download.${CLUSTER}.${DOMAIN} ws.${CLUSTER}.${DOMAIN} workbench.${CLUSTER}.${DOMAIN} workbench2.${CLUSTER}.${DOMAIN}" >> /etc/hosts
+</code></pre>
+</notextile>
 
-At this point you should be able to log into the Arvados cluster.
+h2(#initial_user). Initial user and login 
 
-If you did not change the defaults, the initial URL will be:
+At this point you should be able to log into the Arvados cluster. The initial URL will be:
 
 * https://workbench.arva2.arv.local
 
@@ -102,8 +189,100 @@ or, in general, the url format will be:
 
 By default, the provision script creates an initial user for testing purposes. This user is configured as administrator of the newly created cluster.
 
-Assuming you didn't change the defaults, the initial credentials are:
+Assuming you didn't change these values in the @local.params@ file, the initial credentials are:
 
 * User: 'admin'
 * Password: 'password'
 * Email: 'admin at arva2.arv.local'
+
+h2(#test_install). Test the installed cluster running a simple workflow
+
+The @provision.sh@ script saves a simple example test workflow in the @/tmp/cluster_tests@ directory in the node. If you want to run it, just ssh to the node, change to that directory and run:
+
+<notextile>
+<pre><code>cd /tmp/cluster_tests
+./run-test.sh
+</code></pre>
+</notextile>
+
+It will create a test user (by default, the same one as the admin user), upload a small workflow and run it. If everything goes OK, the output should similar to this (some output was shortened for clarity):
+
+<notextile>
+<pre><code>Creating Arvados Standard Docker Images project
+Arvados project uuid is 'arva2-j7d0g-0prd8cjlk6kfl7y'
+{
+ ...
+ "uuid":"arva2-o0j2j-n4zu4cak5iifq2a",
+ "owner_uuid":"arva2-tpzed-000000000000000",
+ ...
+}
+Uploading arvados/jobs' docker image to the project
+2.1.1: Pulling from arvados/jobs
+8559a31e96f4: Pulling fs layer
+...
+Status: Downloaded newer image for arvados/jobs:2.1.1
+docker.io/arvados/jobs:2.1.1
+2020-11-23 21:43:39 arvados.arv_put[32678] INFO: Creating new cache file at /home/vagrant/.cache/arvados/arv-put/c59256eda1829281424c80f588c7cc4d
+2020-11-23 21:43:46 arvados.arv_put[32678] INFO: Collection saved as 'Docker image arvados jobs:2.1.1 sha256:0dd50'
+arva2-4zz18-1u5pvbld7cvxuy2
+Creating initial user ('admin')
+Setting up user ('admin')
+{
+ "items":[
+  {
+   ...
+   "owner_uuid":"arva2-tpzed-000000000000000",
+   ...
+   "uuid":"arva2-o0j2j-1ownrdne0ok9iox"
+  },
+  {
+   ...
+   "owner_uuid":"arva2-tpzed-000000000000000",
+   ...
+   "uuid":"arva2-o0j2j-1zbeyhcwxc1tvb7"
+  },
+  {
+   ...
+   "email":"admin at arva2.arv.local",
+   ...
+   "owner_uuid":"arva2-tpzed-000000000000000",
+   ...
+   "username":"admin",
+   "uuid":"arva2-tpzed-3wrm93zmzpshrq2",
+   ...
+  }
+ ],
+ "kind":"arvados#HashList"
+}
+Activating user 'admin'
+{
+ ...
+ "email":"admin at arva2.arv.local",
+ ...
+ "username":"admin",
+ "uuid":"arva2-tpzed-3wrm93zmzpshrq2",
+ ...
+}
+Running test CWL workflow
+INFO /usr/bin/cwl-runner 2.1.1, arvados-python-client 2.1.1, cwltool 3.0.20200807132242
+INFO Resolved 'hasher-workflow.cwl' to 'file:///tmp/cluster_tests/hasher-workflow.cwl'
+...
+INFO Using cluster arva2 (https://arva2.arv.local:8443/)
+INFO Upload local files: "test.txt"
+INFO Uploaded to ea34d971b71d5536b4f6b7d6c69dc7f6+50 (arva2-4zz18-c8uvwqdry4r8jao)
+INFO Using collection cache size 256 MiB
+INFO [container hasher-workflow.cwl] submitted container_request arva2-xvhdp-v1bkywd58gyocwm
+INFO [container hasher-workflow.cwl] arva2-xvhdp-v1bkywd58gyocwm is Final
+INFO Overall process status is success
+INFO Final output collection d6c69a88147dde9d52a418d50ef788df+123
+{
+    "hasher_out": {
+        "basename": "hasher3.md5sum.txt",
+        "class": "File",
+        "location": "keep:d6c69a88147dde9d52a418d50ef788df+123/hasher3.md5sum.txt",
+        "size": 95
+    }
+}
+INFO Final process status is success
+</code></pre>
+</notextile>
diff --git a/doc/install/salt-multi-host.html.textile.liquid b/doc/install/terraform-multi-host.html.textile.liquid
similarity index 99%
copy from doc/install/salt-multi-host.html.textile.liquid
copy to doc/install/terraform-multi-host.html.textile.liquid
index 709c32e2a..c7f6fe554 100644
--- a/doc/install/salt-multi-host.html.textile.liquid
+++ b/doc/install/terraform-multi-host.html.textile.liquid
@@ -1,7 +1,7 @@
 ---
 layout: default
 navsection: installguide
-title: Multi host Arvados
+title: Terraform code to create the 
 ...
 {% comment %}
 Copyright (C) The Arvados Authors. All rights reserved.
diff --git a/tools/terraform/_provider.tf b/tools/terraform/_provider.tf
new file mode 100644
index 000000000..ce3eb03ec
--- /dev/null
+++ b/tools/terraform/_provider.tf
@@ -0,0 +1,16 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+provider "aws" {
+  region  = var.aws_region
+  profile = var.aws_profile
+}
+
+terraform {
+  required_providers {
+    aws = {
+      version = "~> 3.31"
+    }
+  }
+}
diff --git a/tools/terraform/_user_data.sh b/tools/terraform/_user_data.sh
new file mode 100644
index 000000000..634420e3b
--- /dev/null
+++ b/tools/terraform/_user_data.sh
@@ -0,0 +1,20 @@
+#!/bin/sh
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+apt update
+apt install -y curl xfsprogs
+
+# Find the unformatted disks with lsblk, getting those with no format ($2)
+# and which name has no number (for xv*) or 'p?' (for nmve*)
+UNFORMATTED_DISK="/dev/$(lsblk -o NAME,FSTYPE -dsn | awk '/xv.*[0-9].*/ || /nvme.*p.*/ { next; } $2 == "" {print $1}')"
+if ! grep -q '/data' /etc/fstab && [ "$${UNFORMATTED_DISK}" != "/dev/" ]; then
+  mkdir -p /data
+  mkfs.xfs -f $${UNFORMATTED_DISK} || exit 1
+  BLKID=$(blkid |grep xfs|awk '{print $2}')
+
+  echo "# Added by curii_run_once script" >> /etc/fstab
+  echo "$${BLKID} /data xfs auto 0 0" >> /etc/fstab
+  mount  /data || exit 1
+fi
diff --git a/tools/terraform/api_iam_role.tf b/tools/terraform/api_iam_role.tf
new file mode 100644
index 000000000..118ac8c2f
--- /dev/null
+++ b/tools/terraform/api_iam_role.tf
@@ -0,0 +1,27 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+# Assume role for the instance
+resource "aws_iam_role" "api_iam_role" {
+    name = "${var.cluster}-api-iam-role"
+    assume_role_policy = templatefile("${path.module}/iam_policy_assume_role.json", {})
+}
+
+# Associate the dispatcher policy to the role
+resource "aws_iam_role_policy_attachment" "api_dispatcher_policies_attachment" {
+    role       = aws_iam_role.api_iam_role.name
+    policy_arn = aws_iam_policy.dispatcher_iam_policy.arn
+}
+
+# Associate letsencrypt modification policy to the role
+resource "aws_iam_role_policy_attachment" "api_letsencrypt_route53_policies_attachment" {
+    role       = aws_iam_role.api_iam_role.name
+    policy_arn = aws_iam_policy.letsencrypt_route53_iam_policy.arn
+}
+
+# Add the role to the instance profile
+resource "aws_iam_instance_profile" "api_instance_profile" {
+  name  = "api_instance_profile"
+  role = "${var.cluster}-api-iam-role"
+}
diff --git a/tools/terraform/api_iam_role_outputs.tf b/tools/terraform/api_iam_role_outputs.tf
new file mode 100644
index 000000000..33fa9f8f4
--- /dev/null
+++ b/tools/terraform/api_iam_role_outputs.tf
@@ -0,0 +1,22 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "dispatcher_iam_policy_id" {
+  value = aws_iam_policy.dispatcher_iam_policy.id
+}
+output "dispatcher_iam_policy_arn" {
+  value = aws_iam_policy.dispatcher_iam_policy.arn
+}
+output "letsencrypt_route53_iam_policy_id" {
+  value = aws_iam_policy.letsencrypt_route53_iam_policy.id
+}
+output "letsencrypt_route53_iam_policy_arn" {
+  value = aws_iam_policy.letsencrypt_route53_iam_policy.arn
+}
+output "api_iam_role_arn" {
+  value = aws_iam_role.api_iam_role.arn
+}
+output "api_iam_role_id" {
+  value = aws_iam_role.api_iam_role.id
+}
diff --git a/tools/terraform/api_instance.tf b/tools/terraform/api_instance.tf
new file mode 100644
index 000000000..285cf614b
--- /dev/null
+++ b/tools/terraform/api_instance.tf
@@ -0,0 +1,96 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+module "api" {
+  source                 = "terraform-aws-modules/ec2-instance/aws"
+  version                = "~> 2.17.0"
+
+  name                   = "${var.cluster}-api"
+  instance_count         = 1
+
+  iam_instance_profile   = "api_instance_profile"
+  ami                    = try(var.instance_ami["api"], var.instance_ami["default"])
+  instance_type          = try(var.instance_type["api"], var.instance_type["default"])
+  key_name               = var.key_name
+  monitoring             = true
+
+  tags                    = merge({"Name": "${var.cluster}-api",
+                                   "OsType": "LINUX"}, local.resource_tags)
+  volume_tags             = merge({"Name": "${var.cluster}-api"}, local.resource_tags)
+
+  network_interface       = [{
+    device_index = 0,
+    network_interface_id = aws_network_interface.api.id,
+  }]
+
+  # associate_public_ip_address = false
+  ebs_optimized           = true
+  user_data               = templatefile("_user_data.sh", {})
+
+  root_block_device           = [{
+    encrypted             = true,
+    kms_key_id            = var.kms_key_id,
+    volume_size           = var.root_bd_size,
+    delete_on_termination = true,
+  }]
+  ebs_block_device            = [{
+    encrypted             = true,
+    kms_key_id            = var.kms_key_id,
+    volume_size           = try(var.data_bd_size["api"], var.data_bd_size["default"])
+    delete_on_termination = true,
+    device_name           = "xvdh",
+  }]
+}
+
+resource "aws_eip" "cluster_api_public_ip" {
+  vpc      = true
+  instance = module.api.id[0]
+  network_interface = aws_network_interface.api.id
+  tags     = merge({"Name": "${var.cluster}-api-ip"},local.resource_tags)
+}
+
+resource "aws_network_interface" "api" {
+  subnet_id       = var.manage_vpc ? module.vpc.0.public_subnets[0] : var.public_subnets_ids[0]
+  # private_ips     = [cidrhost(var.vpc_subnet_cidrs[0], var.host_number["api"])]
+  security_groups = [
+                     local.ssh_sg,
+                     local.http_sg,
+                     local.https_sg,
+                    ]
+  tags           = merge({"Name": "${var.cluster}-api"}, local.resource_tags)
+}
+
+## Public A RRs
+module "api_route53_public_records_A" {
+  source         = "./modules/aws/route53/records/a"
+  zone_id        = module.r53_zone_public.id
+
+  zone_records_A = {
+    (var.r53_domain_name) = {
+      ttl     = "300",
+      records = [aws_eip.cluster_api_public_ip.public_ip]
+    },
+    "ws" = {
+      ttl     = "300",
+      records = [aws_eip.cluster_api_public_ip.public_ip]
+    }
+  }
+}
+
+## Private A RRs
+module "api_route53_private_records_A" {
+  source         = "./modules/aws/route53/records/a"
+  zone_id        = module.r53_zone_private.id
+
+  zone_records_A = {
+    (var.r53_domain_name) = {
+      ttl     = "300",
+      records = aws_network_interface.api.private_ips
+    },
+    "ws" = {
+      ttl     = "300",
+      records = aws_network_interface.api.private_ips
+    }
+  }
+}
diff --git a/tools/terraform/api_instance_outputs.tf b/tools/terraform/api_instance_outputs.tf
new file mode 100644
index 000000000..ba6dc6b70
--- /dev/null
+++ b/tools/terraform/api_instance_outputs.tf
@@ -0,0 +1,19 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "api_id" {
+  value = module.api.id
+}
+output "api_private_dns_names" {
+  value = aws_network_interface.api.private_dns_name
+}
+output "api_private_ip" {
+  value = module.api.private_ip
+}
+output "api_private_eni_id" {
+  value = aws_network_interface.api.id
+}
+output "api_public_ip" {
+  value = aws_eip.cluster_api_public_ip.public_ip
+}
diff --git a/tools/terraform/db_instance.tf b/tools/terraform/db_instance.tf
new file mode 100644
index 000000000..0377f484f
--- /dev/null
+++ b/tools/terraform/db_instance.tf
@@ -0,0 +1,70 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+module "database" {
+  source                 = "terraform-aws-modules/ec2-instance/aws"
+  version                = "~> 2.17.0"
+
+  name                   = "${var.cluster}-database"
+  instance_count         = 1
+
+  ami                    = try(var.instance_ami["database"], var.instance_ami["default"])
+  instance_type          = try(var.instance_type["database"], var.instance_type["default"])
+  key_name               = var.key_name
+  monitoring             = true
+
+  tags                   = merge({"Name": "${var.cluster}-database",
+                                  "OsType": "LINUX"}, local.resource_tags)
+  volume_tags            = merge({"Name": "${var.cluster}-database"}, local.resource_tags)
+
+  network_interface      = [{
+    device_index = 0,
+    network_interface_id = aws_network_interface.database.id,
+  }]
+
+  ebs_optimized          = true
+  user_data               = templatefile("_user_data.sh", {})
+
+  root_block_device           = [{
+    encrypted             = true,
+    kms_key_id            = var.kms_key_id,
+    volume_size           = var.root_bd_size,
+    delete_on_termination = true,
+  }]
+  ebs_block_device            = [{
+    encrypted             = true,
+    kms_key_id            = var.kms_key_id,
+    volume_size           = try(var.data_bd_size["database"], var.data_bd_size["default"])
+    delete_on_termination = true,
+    device_name           = "xvdh",
+  }]
+}
+
+resource "aws_network_interface" "database" {
+  subnet_id       = var.manage_vpc ? module.vpc.0.private_subnets[0] : var.private_subnets_ids[0]
+  # private_ips     = [cidrhost(var.vpc_subnet_cidrs[0], var.host_number["database"])]
+  security_groups = [
+                     local.ssh_sg,
+                     local.postgresql_sg,
+                    ]
+  tags            = merge({"Name": "${var.cluster}-database"}, local.resource_tags)
+}
+
+## Private A RRs
+module "database_route53_private_records_A" {
+  source         = "./modules/aws/route53/records/a"
+  zone_id        = module.r53_zone_private.id
+
+  zone_records_A = {
+    "database" = {
+      ttl     = "300",
+      records = aws_network_interface.database.private_ips
+
+    },
+    "db" = {
+      ttl     = "300",
+      records = aws_network_interface.database.private_ips
+    }
+  }
+}
diff --git a/tools/terraform/db_instance_outputs.tf b/tools/terraform/db_instance_outputs.tf
new file mode 100644
index 000000000..e03cdcaa6
--- /dev/null
+++ b/tools/terraform/db_instance_outputs.tf
@@ -0,0 +1,16 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "database_id" {
+  value = module.database.id
+}
+output "database_private_dns_names" {
+  value = module.database.private_dns
+}
+output "database_private_ip" {
+  value = module.database.private_ip
+}
+output "database_private_eni_id" {
+  value = aws_network_interface.database.id
+}
diff --git a/tools/terraform/dispatcher_iam_policy.json b/tools/terraform/dispatcher_iam_policy.json
new file mode 100644
index 000000000..7022c6579
--- /dev/null
+++ b/tools/terraform/dispatcher_iam_policy.json
@@ -0,0 +1,14 @@
+{
+  "Version": "2012-10-17",
+  "Id": "${cluster}-api-dispatcher-role policy",
+  "Statement": [
+    {
+      "Effect": "Allow",
+      "Action": [
+        "ec2:*",
+        "kms:*"
+      ],
+      "Resource": "*"
+    }
+  ]
+}
diff --git a/tools/terraform/dispatcher_iam_policy.tf b/tools/terraform/dispatcher_iam_policy.tf
new file mode 100644
index 000000000..c8202842c
--- /dev/null
+++ b/tools/terraform/dispatcher_iam_policy.tf
@@ -0,0 +1,11 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+resource "aws_iam_policy" "dispatcher_iam_policy" {
+  name  = "${var.cluster}-dispatcher-iam-role-policy"
+  description = "Policy to allow API to launch compute instances"
+  policy = templatefile("${path.module}/dispatcher_iam_policy.json", {
+    "cluster" = var.cluster
+  })
+}
diff --git a/tools/terraform/general_outputs.tf b/tools/terraform/general_outputs.tf
new file mode 100644
index 000000000..a46092dcf
--- /dev/null
+++ b/tools/terraform/general_outputs.tf
@@ -0,0 +1,23 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+### GENERAL
+output "aws_region" {
+  value = var.aws_region
+}
+output "aws_profile" {
+  value = var.aws_profile
+}
+output "environment" {
+  value = var.environment
+}
+output "namespace" {
+  value = var.namespace
+}
+output "resource_tags" {
+  value = local.resource_tags
+}
+output "ami" {
+  value = var.ami
+}
diff --git a/tools/terraform/iam_policy_assume_role.json b/tools/terraform/iam_policy_assume_role.json
new file mode 100644
index 000000000..e3b695d98
--- /dev/null
+++ b/tools/terraform/iam_policy_assume_role.json
@@ -0,0 +1,13 @@
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Action": "sts:AssumeRole",
+      "Principal": {
+        "Service": "ec2.amazonaws.com"
+      },
+      "Effect": "Allow",
+      "Sid": ""
+    }
+  ]
+}
diff --git a/tools/terraform/keepproxy_iam_role.tf b/tools/terraform/keepproxy_iam_role.tf
new file mode 100644
index 000000000..838d1f5ea
--- /dev/null
+++ b/tools/terraform/keepproxy_iam_role.tf
@@ -0,0 +1,21 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+# Assume role for the instance
+resource "aws_iam_role" "keepproxy_iam_role" {
+    name = "${var.cluster}-keepproxy-iam-role"
+    assume_role_policy = templatefile("${path.module}/iam_policy_assume_role.json", {})
+}
+
+# Associate letsencrypt modification policy to the role
+resource "aws_iam_role_policy_attachment" "keepproxy_letsencrypt_route53_policies_attachment" {
+    role       = aws_iam_role.keepproxy_iam_role.name
+    policy_arn = aws_iam_policy.letsencrypt_route53_iam_policy.arn
+}
+
+# Add the role to the instance profile
+resource "aws_iam_instance_profile" "keepproxy_instance_profile" {
+  name  = "keepproxy_instance_profile"
+  role = "${var.cluster}-keepproxy-iam-role"
+}
diff --git a/tools/terraform/keepproxy_iam_role_outputs.tf b/tools/terraform/keepproxy_iam_role_outputs.tf
new file mode 100644
index 000000000..dc20142c6
--- /dev/null
+++ b/tools/terraform/keepproxy_iam_role_outputs.tf
@@ -0,0 +1,10 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "keepproxy_iam_role_arn" {
+  value = aws_iam_role.keepproxy_iam_role.arn
+}
+output "keepproxy_iam_role_id" {
+  value = aws_iam_role.keepproxy_iam_role.id
+}
diff --git a/tools/terraform/keepproxy_instance.tf b/tools/terraform/keepproxy_instance.tf
new file mode 100644
index 000000000..8fab4bb10
--- /dev/null
+++ b/tools/terraform/keepproxy_instance.tf
@@ -0,0 +1,112 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+module "keepproxy" {
+  source                 = "terraform-aws-modules/ec2-instance/aws"
+  version                = "~> 2.16"
+
+  name                   = "${var.cluster}-keepproxy"
+  instance_count         = 1
+
+  iam_instance_profile   = "keepproxy_instance_profile"
+  ami                    = try(var.instance_ami["keepproxy"], var.instance_ami["default"])
+  instance_type          = try(var.instance_type["keepproxy"], var.instance_type["default"])
+  key_name               = var.key_name
+  monitoring             = true
+
+  tags                   = merge({"Name": "${var.cluster}-keepproxy",
+                                  "OsType": "LINUX"}, local.resource_tags)
+  volume_tags            = merge({"Name": "${var.cluster}-keepproxy"}, local.resource_tags)
+
+  network_interface      = [{
+    device_index = 0,
+    network_interface_id = aws_network_interface.keepproxy.id,
+  }]
+
+  # associate_public_ip_address = false
+  ebs_optimized           = true
+  user_data               = templatefile("_user_data.sh", {})
+
+  root_block_device           = [{
+    encrypted             = true,
+    kms_key_id            = var.kms_key_id,
+    volume_size           = var.root_bd_size,
+    delete_on_termination = true,
+  }]
+  ebs_block_device            = [{
+    encrypted             = true,
+    kms_key_id            = var.kms_key_id,
+    volume_size           = try(var.data_bd_size["keeproxy"], var.data_bd_size["default"])
+    delete_on_termination = true,
+    device_name           = "xvdh",
+  }]
+}
+
+resource "aws_eip" "cluster_keepproxy_public_ip" {
+  vpc      = true
+  instance = module.keepproxy.id[0]
+  network_interface = aws_network_interface.keepproxy.id
+  tags     = merge({"Name": "${var.cluster}-keepproxy-ip"},local.resource_tags)
+}
+
+resource "aws_network_interface" "keepproxy" {
+  subnet_id       = var.manage_vpc ? module.vpc.0.public_subnets[0] : var.public_subnets_ids[0]
+  # private_ips     = [cidrhost(var.vpc_subnet_cidrs[0], var.host_number["keepproxy"])]
+  security_groups = [
+                     local.ssh_sg,
+                     local.http_sg,
+                     local.https_sg,
+                    ]
+  tags           = merge({"Name": "${var.cluster}-keepproxy"}, local.resource_tags)
+}
+
+## Public A RRs
+module "keepproxy_route53_public_records_A" {
+  source         = "./modules/aws/route53/records/a"
+  zone_id        = module.r53_zone_public.id
+
+  zone_records_A = {
+    "keep" = {
+      ttl     = "300",
+      records = [aws_eip.cluster_keepproxy_public_ip.public_ip]
+    },
+    "collections" = {
+      ttl     = "300",
+      records = [aws_eip.cluster_keepproxy_public_ip.public_ip]
+    },
+    "*.collections" = {
+      ttl     = "300",
+      records = [aws_eip.cluster_keepproxy_public_ip.public_ip]
+    },
+    "download" = {
+      ttl     = "300",
+      records = [aws_eip.cluster_keepproxy_public_ip.public_ip]
+    }
+  }
+}
+
+## Private A RRs
+module "keepproxy_route53_private_records_A" {
+  source         = "./modules/aws/route53/records/a"
+  zone_id        = module.r53_zone_private.id
+
+  zone_records_A = {
+    "keep" = {
+      ttl     = "300",
+      records = aws_network_interface.keepproxy.private_ips
+    },
+    "collections" = {
+      ttl     = "300",
+      records = aws_network_interface.keepproxy.private_ips
+    },
+    "*.collections" = {
+      ttl     = "300",
+      records = aws_network_interface.keepproxy.private_ips
+    },
+    "download" = {
+      ttl     = "300",
+      records = aws_network_interface.keepproxy.private_ips
+    }
+  }
+}
diff --git a/tools/terraform/keepproxy_instance_outputs.tf b/tools/terraform/keepproxy_instance_outputs.tf
new file mode 100644
index 000000000..4578a90fd
--- /dev/null
+++ b/tools/terraform/keepproxy_instance_outputs.tf
@@ -0,0 +1,19 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "keepproxy_id" {
+  value = module.keepproxy.id
+}
+output "keepproxy_private_dns_names" {
+  value = module.keepproxy.private_dns
+}
+output "keepproxy_private_ip" {
+  value = module.keepproxy.private_ip
+}
+output "keepproxy_private_eni_id" {
+  value = aws_network_interface.keepproxy.id
+}
+output "keepproxy_public_ip" {
+  value = aws_eip.cluster_keepproxy_public_ip.public_ip
+}
diff --git a/tools/terraform/keepstore_iam_policy.json b/tools/terraform/keepstore_iam_policy.json
new file mode 100644
index 000000000..9f7dd202d
--- /dev/null
+++ b/tools/terraform/keepstore_iam_policy.json
@@ -0,0 +1,19 @@
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Effect": "Allow",
+      "Action": ["s3:ListBucket"],
+      "Resource": ["${bucket_arn}"]
+    },
+    {
+      "Effect": "Allow",
+      "Action": [
+        "s3:PutObject",
+        "s3:GetObject",
+        "s3:DeleteObject"
+      ],
+      "Resource": ["${bucket_arn}/*"]
+    }
+  ]
+}
diff --git a/tools/terraform/keepstore_iam_policy.tf b/tools/terraform/keepstore_iam_policy.tf
new file mode 100644
index 000000000..6a9d873ed
--- /dev/null
+++ b/tools/terraform/keepstore_iam_policy.tf
@@ -0,0 +1,13 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+# IAM policy to access the bucket
+resource "aws_iam_policy" "keepstore_iam_policy" {
+  count = var.keepstore_count
+  name  = "${var.cluster}-keepstore-${format("%02d", count.index)}-iam-role-policy"
+  description = "Policy to allow writing to the S3 bucket ${var.cluster}-nyw5e-${format("%016d", count.index)}-volume-policy"
+  policy = templatefile("${path.module}/keepstore_iam_policy.json", {
+    "bucket_arn" = aws_s3_bucket.keepstore.*.arn[count.index]
+  })
+}
diff --git a/tools/terraform/keepstore_iam_policy_s3_bucket.json b/tools/terraform/keepstore_iam_policy_s3_bucket.json
new file mode 100644
index 000000000..83c1ddaeb
--- /dev/null
+++ b/tools/terraform/keepstore_iam_policy_s3_bucket.json
@@ -0,0 +1,32 @@
+{
+  "Version": "2012-10-17",
+  "Id": "${id}",
+  "Statement": [
+    {
+      "Sid": "BucketAllow",
+      "Effect": "Allow",
+      "Principal": {
+        "AWS":
+          ${jsonencode([ for arn in access_arns : "${arn}" ], )}
+      },
+      "Action": [
+        "s3:ListBucket"
+      ],
+      "Resource": "${bucket_arn}"
+    },
+    {
+      "Sid": "BucketAllowObjects",
+      "Effect": "Allow",
+      "Principal": {
+        "AWS":
+          ${jsonencode([ for arn in access_arns : "${arn}" ], )}
+      },
+      "Action": [
+        "s3:PutObject",
+        "s3:GetObject",
+        "s3:DeleteObject"
+      ],
+      "Resource": "${bucket_arn}/*"
+    }
+  ]
+}
diff --git a/tools/terraform/keepstore_iam_role.tf b/tools/terraform/keepstore_iam_role.tf
new file mode 100644
index 000000000..512abbbed
--- /dev/null
+++ b/tools/terraform/keepstore_iam_role.tf
@@ -0,0 +1,24 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+# Assume role for the instance
+resource "aws_iam_role" "keepstore_iam_assume_role" {
+  count = var.keepstore_count
+  name = "${var.cluster}-keepstore-${format("%02d", count.index)}-iam-role"
+  assume_role_policy = file("${path.module}/iam_policy_assume_role.json")
+}
+
+# Associate the access bucket policy to the role
+resource "aws_iam_role_policy_attachment" "keepstore_policies_attachment" {
+  count = var.keepstore_count
+  role       = aws_iam_role.keepstore_iam_assume_role.*.name[count.index]
+  policy_arn = aws_iam_policy.keepstore_iam_policy.*.arn[count.index]
+}
+
+# Add the role to the instance profile
+resource "aws_iam_instance_profile" "keepstore_instance_profile" {
+  count = var.keepstore_count
+  name  = "keepstore-${format("%02d", count.index)}_instance_profile"
+  role = "${var.cluster}-keepstore-${format("%02d", count.index)}-iam-role"
+}
diff --git a/tools/terraform/keepstore_iam_role_outputs.tf b/tools/terraform/keepstore_iam_role_outputs.tf
new file mode 100644
index 000000000..a1265d8e6
--- /dev/null
+++ b/tools/terraform/keepstore_iam_role_outputs.tf
@@ -0,0 +1,16 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "keepstore_iam_policy_id" {
+  value = aws_iam_policy.keepstore_iam_policy.*.id
+}
+output "keepstore_iam_policy_arn" {
+  value = aws_iam_policy.keepstore_iam_policy.*.arn
+}
+output "keepstore_iam_assume_role_arn" {
+  value = aws_iam_role.keepstore_iam_assume_role.*.arn
+}
+output "keepstore_iam_assume_role_id" {
+  value = aws_iam_role.keepstore_iam_assume_role.*.id
+}
diff --git a/tools/terraform/keepstore_instance.tf b/tools/terraform/keepstore_instance.tf
new file mode 100644
index 000000000..8e61fefc6
--- /dev/null
+++ b/tools/terraform/keepstore_instance.tf
@@ -0,0 +1,85 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+module "keepstore" {
+  count                = var.keepstore_count
+  source               = "terraform-aws-modules/ec2-instance/aws"
+  version              = "~> 2.17.0"
+
+  name                 = "${var.cluster}-keepstore-${format("%02d", count.index)}"
+  instance_count       = 1
+
+  ami                  = try(var.instance_ami["keepstore"], var.instance_ami["default"])
+  instance_type        = try(var.instance_type["keepstore"], var.instance_type["default"])
+
+  iam_instance_profile = "keepstore-${format("%02d", count.index)}_instance_profile"
+  key_name             = var.key_name
+  monitoring           = true
+
+  tags                 = merge({"Name": "${var.cluster}-keepstore-${format("%02d", count.index)}",
+                               "OsType": "LINUX"}, local.resource_tags)
+  volume_tags          = merge({"Name": "${var.cluster}-keepstore-${format("%02d", count.index)}"},
+                               local.resource_tags)
+
+  network_interface    = [{
+    device_index         = 0,
+    network_interface_id = aws_network_interface.keepstore.*.id[count.index],
+  }]
+
+  # associate_public_ip_address = false
+  ebs_optimized        = true
+  user_data               = templatefile("_user_data.sh", {})
+
+  root_block_device    = [{
+    encrypted             = true,
+    kms_key_id            = var.kms_key_id,
+    volume_size           = var.root_bd_size,
+    delete_on_termination = true,
+  }]
+  ebs_block_device     = [{
+    encrypted             = true,
+    kms_key_id            = var.kms_key_id,
+    volume_size           = try(var.data_bd_size["keepstore"], var.data_bd_size["default"])
+    delete_on_termination = true,
+    device_name           = "xvdh",
+  }]
+}
+
+# resource "aws_eip" "cluster_keepstore_public_ip" {
+#   count     = var.keepstore_count
+#   vpc      = true
+#   instance = module.keepstore.*.id[count.index][0]
+#   network_interface = aws_network_interface.keepstore.*.id[count.index]
+#   tags     = merge({"Name": "${var.cluster}-keepstore-${format("%02d", count.index)}-ip"},local.resource_tags)
+# }
+
+resource "aws_network_interface" "keepstore" {
+  count            = var.keepstore_count
+  subnet_id        = var.manage_vpc ? module.vpc.0.private_subnets[0] : var.private_subnets_ids[0]
+  # private_ips     = [cidrhost(var.vpc_subnet_cidrs[0], var.host_number["keepstore"])]
+  security_groups = [
+                     local.ssh_sg,
+                     local.keepstore_sg,
+                    ]
+  tags      = merge({"Name": "${var.cluster}-keepstore-${format("%02d", count.index)}"},
+                    local.resource_tags)
+}
+
+### FIXME! Needs improvement
+## Private A RRs
+module "keepstore_route53_private_records_A" {
+  source         = "./modules/aws/route53/records/a"
+  zone_id        = module.r53_zone_private.id
+
+  zone_records_A = {
+    "keep0" = {
+      ttl     = "300",
+      records = aws_network_interface.keepstore.0.private_ips
+    },
+    "keep1" = {
+      ttl     = "300",
+      records = aws_network_interface.keepstore.1.private_ips
+    },
+  }
+}
diff --git a/tools/terraform/keepstore_instance_outputs.tf b/tools/terraform/keepstore_instance_outputs.tf
new file mode 100644
index 000000000..ac76a5ee3
--- /dev/null
+++ b/tools/terraform/keepstore_instance_outputs.tf
@@ -0,0 +1,16 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "keepstore_id" {
+  value = module.keepstore.*.id
+}
+output "keepstore_private_dns_names" {
+  value = module.keepstore.*.private_dns
+}
+output "keepstore_private_ip" {
+  value = module.keepstore.*.private_ip
+}
+output "keepstore_private_eni_id" {
+  value = aws_network_interface.keepstore.*.id
+}
diff --git a/tools/terraform/keepstore_s3_bucket.tf b/tools/terraform/keepstore_s3_bucket.tf
new file mode 100644
index 000000000..5aad658bd
--- /dev/null
+++ b/tools/terraform/keepstore_s3_bucket.tf
@@ -0,0 +1,35 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+resource "aws_s3_bucket" "keepstore" {
+  count  = var.keepstore_count
+  bucket = "${var.cluster}-nyw5e-${format("%016d", count.index)}-volume"
+  acl    = "private"
+
+  website {
+    index_document = "index.html"
+    error_document = "error.html"
+  }
+
+  server_side_encryption_configuration {
+    rule {
+      apply_server_side_encryption_by_default {
+        sse_algorithm = "AES256"
+      }
+    }
+  }
+  tags   = merge({"Name": "${var.cluster}-nyw5e-${format("%016d", count.index)}-bucket"},
+                 local.resource_tags)
+}
+
+resource "aws_s3_bucket_policy" "keepstore_bucket_policy" {
+  count  = var.keepstore_count
+  bucket = aws_s3_bucket.keepstore.*.id[count.index]
+
+  policy = templatefile("${path.module}/keepstore_iam_policy_s3_bucket.json", {
+    id               = "${var.cluster}-nyw5e-${format("%016d", count.index)}-volume-policy",
+    bucket_arn       = aws_s3_bucket.keepstore.*.arn[count.index]
+    access_arns      = [aws_iam_role.keepstore_iam_assume_role.*.arn[count.index]]
+  })
+}
diff --git a/tools/terraform/keepstore_s3_bucket_outputs.tf b/tools/terraform/keepstore_s3_bucket_outputs.tf
new file mode 100644
index 000000000..b816d22e4
--- /dev/null
+++ b/tools/terraform/keepstore_s3_bucket_outputs.tf
@@ -0,0 +1,13 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "keepstore_bucket_arn" {
+  value = aws_s3_bucket.keepstore.*.arn
+}
+output "keepstore_bucket_id" {
+  value = aws_s3_bucket.keepstore.*.id 
+}
+output "keepstore_bucket_policy_id" {
+  value = aws_s3_bucket_policy.keepstore_bucket_policy.*.id
+}
diff --git a/tools/terraform/keypair.tf b/tools/terraform/keypair.tf
new file mode 100644
index 000000000..c9a6636eb
--- /dev/null
+++ b/tools/terraform/keypair.tf
@@ -0,0 +1,11 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+resource "aws_key_pair" "keypair" {
+  count      = var.enable_key_pair == true ? 1 : 0
+
+  key_name   = var.key_name
+  public_key = var.key_public_key == "" ? file(var.key_path) : var.key_public_key
+  tags       = local.resource_tags
+}
diff --git a/tools/terraform/keypair_outputs.tf b/tools/terraform/keypair_outputs.tf
new file mode 100644
index 000000000..92d188f77
--- /dev/null
+++ b/tools/terraform/keypair_outputs.tf
@@ -0,0 +1,8 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "keypair_name" {
+  description = "Name of the SSH keypair applied to the instances."
+  value       = var.key_name
+}
diff --git a/tools/terraform/letsencrypt_route53_iam_policy.json b/tools/terraform/letsencrypt_route53_iam_policy.json
new file mode 100644
index 000000000..282a63fd9
--- /dev/null
+++ b/tools/terraform/letsencrypt_route53_iam_policy.json
@@ -0,0 +1,25 @@
+{
+    "Version": "2012-10-17",
+    "Id": "${cluster}_letsencrypt_route53_policy",
+    "Statement": [
+        {
+            "Effect": "Allow",
+            "Action": [
+                "route53:ListHostedZones",
+                "route53:GetChange"
+            ],
+            "Resource": [
+                "*"
+            ]
+        },
+        {
+            "Effect" : "Allow",
+            "Action" : [
+                "route53:ChangeResourceRecordSets"
+            ],
+            "Resource" : [
+                "arn:aws:route53:::hostedzone/${zone_id}"
+            ]
+        }
+    ]
+}
diff --git a/tools/terraform/letsencrypt_route53_iam_policy.tf b/tools/terraform/letsencrypt_route53_iam_policy.tf
new file mode 100644
index 000000000..c1b7fdcf2
--- /dev/null
+++ b/tools/terraform/letsencrypt_route53_iam_policy.tf
@@ -0,0 +1,12 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+resource "aws_iam_policy" "letsencrypt_route53_iam_policy" {
+  name  = "${var.cluster}-letsencrypt_route53-iam-role-policy"
+  description = "Policy to allow API to add records to the public zone"
+  policy = templatefile("${path.module}/letsencrypt_route53_iam_policy.json", {
+    "cluster" = var.cluster
+    "zone_id" = module.r53_zone_public.id
+  })
+}
diff --git a/tools/terraform/locals.tf b/tools/terraform/locals.tf
new file mode 100644
index 000000000..c40dae11d
--- /dev/null
+++ b/tools/terraform/locals.tf
@@ -0,0 +1,20 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+locals {
+  resource_tags = merge(
+                        { "Terraform" = "true" },
+                        { "Environment" = var.environment },
+                        { "Namespace" = var.namespace },
+                        { "Cluster" = var.cluster },
+                        var.tags,
+                       )
+  ssh_sg         = var.manage_security_groups ? module.arvados_ssh_sg.0.this_security_group_id : try(var.vpc_security_group_ids["ssh"], var.vpc_security_group_ids["default"])
+  http_sg        = var.manage_security_groups ? module.arvados_http_sg.0.this_security_group_id : try(var.vpc_security_group_ids["http"], var.vpc_security_group_ids["default"])
+  https_sg       = var.manage_security_groups ? module.arvados_https_sg.0.this_security_group_id : try(var.vpc_security_group_ids["https"], var.vpc_security_group_ids["default"])
+  webshell_sg    = var.manage_security_groups ? module.arvados_webshell_sg.0.this_security_group_id : try(var.vpc_security_group_ids["webshell"], var.vpc_security_group_ids["default"])
+  postgresql_sg  = var.manage_security_groups ? module.arvados_postgresql_sg.0.this_security_group_id : try(var.vpc_security_group_ids["postgresql"], var.vpc_security_group_ids["default"])
+  keepstore_sg   = var.manage_security_groups ? module.arvados_keepstore_sg.0.this_security_group_id : try(var.vpc_security_group_ids["keepstore"], var.vpc_security_group_ids["default"])
+
+}
diff --git a/tools/terraform/modules/aws/route53/records/a/main.tf b/tools/terraform/modules/aws/route53/records/a/main.tf
new file mode 100644
index 000000000..3f71b39b4
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/a/main.tf
@@ -0,0 +1,15 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+resource "aws_route53_record" "a_record" {
+  for_each = var.zone_records_A
+
+  zone_id         = var.zone_id
+  name            = lookup(each.value, "name", each.key)
+  type            = "A"
+  ttl             = lookup(each.value, "ttl", 600)
+
+  records         = lookup(each.value, "records", null)
+  allow_overwrite = lookup(each.value, "allow_overwrite", null)
+}
diff --git a/tools/terraform/modules/aws/route53/records/a/outputs.tf b/tools/terraform/modules/aws/route53/records/a/outputs.tf
new file mode 100644
index 000000000..9ae9cdf8b
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/a/outputs.tf
@@ -0,0 +1,7 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "a_records" {
+  value = aws_route53_record.a_record
+}
diff --git a/tools/terraform/modules/aws/route53/records/a/variables.tf b/tools/terraform/modules/aws/route53/records/a/variables.tf
new file mode 100644
index 000000000..d42acc150
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/a/variables.tf
@@ -0,0 +1,17 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+variable "zone_id"  {
+  type = string
+}
+# Sadly, terraform can't iterate over maps of mixed values without lots
+# of juggling with forced parameters in object definitions and
+# other similarly nasty workarounds, so trying to keep it simple, we'll
+# declare modules for each type of RRs, so code is simpler.
+# More info here https://github.com/hashicorp/terraform/issues/19898
+variable "zone_records_A" {
+  description = "Map of A RRs to add to the zone to create"
+  type        = map
+  default     = {}
+}
diff --git a/tools/terraform/modules/aws/route53/records/aaaa/main.tf b/tools/terraform/modules/aws/route53/records/aaaa/main.tf
new file mode 100644
index 000000000..cdd3e0d49
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/aaaa/main.tf
@@ -0,0 +1,15 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+resource "aws_route53_record" "aaaa_record" {
+  for_each = var.zone_records_AAAA
+
+  zone_id         = var.zone_id
+  name            = lookup(each.value, "name", each.key)
+  type            = "AAAA"
+  ttl             = lookup(each.value, "ttl", 600)
+
+  records         = lookup(each.value, "records", null)
+  allow_overwrite = lookup(each.value, "allow_overwrite", null)
+}
diff --git a/tools/terraform/modules/aws/route53/records/aaaa/outputs.tf b/tools/terraform/modules/aws/route53/records/aaaa/outputs.tf
new file mode 100644
index 000000000..683183a96
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/aaaa/outputs.tf
@@ -0,0 +1,7 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "aaaa_records" {
+  value = aws_route53_record.aaaa_record
+}
diff --git a/tools/terraform/modules/aws/route53/records/aaaa/variables.tf b/tools/terraform/modules/aws/route53/records/aaaa/variables.tf
new file mode 100644
index 000000000..43a838c6b
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/aaaa/variables.tf
@@ -0,0 +1,17 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+variable "zone_id"  {
+  type = string
+}
+# Sadly, terraform can't iterate over maps of mixed values without lots
+# of juggling with forced parameters in object definitions and
+# other similarly nasty workarounds, so trying to keep it simple, we'll
+# declare modules for each type of RRs, so code is simpler.
+# More info here https://github.com/hashicorp/terraform/issues/19898
+variable "zone_records_AAAA" {
+  description = "Map of AAAA RRs to add to the zone to create"
+  type        = map
+  default     = {}
+}
diff --git a/tools/terraform/modules/aws/route53/records/alias/main.tf b/tools/terraform/modules/aws/route53/records/alias/main.tf
new file mode 100644
index 000000000..92834a3c4
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/alias/main.tf
@@ -0,0 +1,18 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+resource "aws_route53_record" "alias_record" {
+  for_each = var.zone_records_ALIAS
+
+  zone_id         = var.zone_id
+  name            = lookup(each.value, "name", each.key)
+  type            = lookup(each.value, "type", "A")
+
+  alias {
+    name                   = each.value.alias.name
+    zone_id                = each.value.alias.zone_id
+    evaluate_target_health = lookup(each.value.alias, "evaluate_target_health", false)
+  }
+  allow_overwrite = lookup(each.value, "allow_overwrite", null)
+}
diff --git a/tools/terraform/modules/aws/route53/records/alias/outputs.tf b/tools/terraform/modules/aws/route53/records/alias/outputs.tf
new file mode 100644
index 000000000..59b605456
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/alias/outputs.tf
@@ -0,0 +1,7 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "alias_records" {
+ value = aws_route53_record.alias_record
+}
diff --git a/tools/terraform/modules/aws/route53/records/alias/variables.tf b/tools/terraform/modules/aws/route53/records/alias/variables.tf
new file mode 100644
index 000000000..e9381e622
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/alias/variables.tf
@@ -0,0 +1,17 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+variable "zone_id"  {
+  type = string
+}
+# Sadly, terraform can't iterate over maps of mixed values without lots
+# of juggling with forced parameters in object definitions and
+# other similarly nasty workarounds, so trying to keep it simple, we'll
+# declare modules for each type of RRs, so code is simpler.
+# More info here https://github.com/hashicorp/terraform/issues/19898
+variable "zone_records_ALIAS" {
+  description = "Map of ALIAS RRs to add to the zone to create"
+  type        = map
+  default     = {}
+}
diff --git a/tools/terraform/modules/aws/route53/records/cname/main.tf b/tools/terraform/modules/aws/route53/records/cname/main.tf
new file mode 100644
index 000000000..9eef47a6c
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/cname/main.tf
@@ -0,0 +1,15 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+resource "aws_route53_record" "cname_record" {
+  for_each = var.zone_records_CNAME
+
+  zone_id         = var.zone_id
+  name            = lookup(each.value, "name", each.key)
+  type            = "CNAME"
+  ttl             = lookup(each.value, "ttl", 600)
+
+  records         = each.value.records
+  allow_overwrite = lookup(each.value, "allow_overwrite", null)
+}
diff --git a/tools/terraform/modules/aws/route53/records/cname/outputs.tf b/tools/terraform/modules/aws/route53/records/cname/outputs.tf
new file mode 100644
index 000000000..db7757eee
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/cname/outputs.tf
@@ -0,0 +1,7 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "cname_records" {
+  value = aws_route53_record.cname_record
+}
diff --git a/tools/terraform/modules/aws/route53/records/cname/variables.tf b/tools/terraform/modules/aws/route53/records/cname/variables.tf
new file mode 100644
index 000000000..6d053dc08
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/cname/variables.tf
@@ -0,0 +1,17 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+variable "zone_id"  {
+  type = string
+}
+# Sadly, terraform can't iterate over maps of mixed values without lots
+# of juggling with forced parameters in object definitions and
+# other similarly nasty workarounds, so trying to keep it simple, we'll
+# declare modules for each type of RRs, so code is simpler.
+# More info here https://github.com/hashicorp/terraform/issues/19898
+variable "zone_records_CNAME" {
+  description = "Map of CNAME RRs to add to the zone to create"
+  type        = map
+  default     = {}
+}
diff --git a/tools/terraform/modules/aws/route53/records/mx/main.tf b/tools/terraform/modules/aws/route53/records/mx/main.tf
new file mode 100644
index 000000000..71ced88ba
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/mx/main.tf
@@ -0,0 +1,15 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+resource "aws_route53_record" "mx_record" {
+  for_each = var.zone_records_MX
+
+  zone_id         = var.zone_id
+  name            = lookup(each.value, "name", each.key)
+  type            = "MX"
+  ttl             = lookup(each.value, "ttl", 600)
+
+  records         = lookup(each.value, "records", null)
+  allow_overwrite = lookup(each.value, "allow_overwrite", null)
+}
diff --git a/tools/terraform/modules/aws/route53/records/mx/outputs.tf b/tools/terraform/modules/aws/route53/records/mx/outputs.tf
new file mode 100644
index 000000000..f1c7e62f0
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/mx/outputs.tf
@@ -0,0 +1,7 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "mx_records" {
+  value = aws_route53_record.mx_record
+}
diff --git a/tools/terraform/modules/aws/route53/records/mx/variables.tf b/tools/terraform/modules/aws/route53/records/mx/variables.tf
new file mode 100644
index 000000000..afdb5d1c2
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/mx/variables.tf
@@ -0,0 +1,17 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+variable "zone_id"  {
+  type = string
+}
+# Sadly, terraform can't iterate over maps of mixed values without lots
+# of juggling with forced parameters in object definitions and
+# other similarly nasty workarounds, so trying to keep it simple, we'll
+# declare modules for each type of RRs, so code is simpler.
+# More info here https://github.com/hashicorp/terraform/issues/19898
+variable "zone_records_MX" {
+  description = "Map of MX RRs to add to the zone to create"
+  type        = map
+  default     = {}
+}
diff --git a/tools/terraform/modules/aws/route53/records/ns/main.tf b/tools/terraform/modules/aws/route53/records/ns/main.tf
new file mode 100644
index 000000000..6670fc29a
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/ns/main.tf
@@ -0,0 +1,15 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+resource "aws_route53_record" "ns_record" {
+  for_each = var.zone_records_NS
+
+  zone_id         = var.zone_id
+  name            = lookup(each.value, "name", each.key)
+  type            = "NS"
+  ttl             = lookup(each.value, "ttl", 600)
+
+  records         = lookup(each.value, "records", null)
+  allow_overwrite = lookup(each.value, "allow_overwrite", null)
+}
diff --git a/tools/terraform/modules/aws/route53/records/ns/outputs.tf b/tools/terraform/modules/aws/route53/records/ns/outputs.tf
new file mode 100644
index 000000000..4e900e2b3
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/ns/outputs.tf
@@ -0,0 +1,7 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "ns_records" {
+  value = aws_route53_record.ns_record
+}
diff --git a/tools/terraform/modules/aws/route53/records/ns/variables.tf b/tools/terraform/modules/aws/route53/records/ns/variables.tf
new file mode 100644
index 000000000..c993ffcc2
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/ns/variables.tf
@@ -0,0 +1,17 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+variable "zone_id"  {
+  type = string
+}
+# Sadly, terraform can't iterate over maps of mixed values without lots
+# of juggling with forced parameters in object definitions and
+# other similarly nasty workarounds, so trying to keep it simple, we'll
+# declare modules for each type of RRs, so code is simpler.
+# More info here https://github.com/hashicorp/terraform/issues/19898
+variable "zone_records_NS" {
+  description = "Map of NS RRs to add to the zone to create"
+  type        = map
+  default     = {}
+}
diff --git a/tools/terraform/modules/aws/route53/records/txt/main.tf b/tools/terraform/modules/aws/route53/records/txt/main.tf
new file mode 100644
index 000000000..91a3969bc
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/txt/main.tf
@@ -0,0 +1,15 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+resource "aws_route53_record" "txt_record" {
+  for_each = var.zone_records_TXT
+
+  zone_id         = var.zone_id
+  name            = lookup(each.value, "name", each.key)
+  type            = "TXT"
+  ttl             = lookup(each.value, "ttl", 600)
+
+  records         = lookup(each.value, "records", null)
+  allow_overwrite = lookup(each.value, "allow_overwrite", null)
+}
diff --git a/tools/terraform/modules/aws/route53/records/txt/outputs.tf b/tools/terraform/modules/aws/route53/records/txt/outputs.tf
new file mode 100644
index 000000000..fed21f21c
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/txt/outputs.tf
@@ -0,0 +1,7 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "txt_records" {
+  value = aws_route53_record.txt_record
+}
diff --git a/tools/terraform/modules/aws/route53/records/txt/variables.tf b/tools/terraform/modules/aws/route53/records/txt/variables.tf
new file mode 100644
index 000000000..17dacdd8e
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/records/txt/variables.tf
@@ -0,0 +1,17 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+variable "zone_id"  {
+  type = string
+}
+# Sadly, terraform can't iterate over maps of mixed values without lots
+# of juggling with forced parameters in object definitions and
+# other similarly nasty workarounds, so trying to keep it simple, we'll
+# declare modules for each type of RRs, so code is simpler.
+# More info here https://github.com/hashicorp/terraform/issues/19898
+variable "zone_records_TXT" {
+  description = "Map of TXT RRs to add to the zone to create"
+  type        = map
+  default     = {}
+}
diff --git a/tools/terraform/modules/aws/route53/zone/private/main.tf b/tools/terraform/modules/aws/route53/zone/private/main.tf
new file mode 100644
index 000000000..8aa125e06
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/zone/private/main.tf
@@ -0,0 +1,17 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+resource "aws_route53_zone" "this" {
+name          = var.zone_name
+  vpc {
+    vpc_id = lookup(var.zone_config, "vpc_id")
+  }
+
+  comment       = lookup(var.zone_config, "comment", null)
+  force_destroy = lookup(var.zone_config, "force_destroy", null)
+  tags          = merge(
+                        var.tags,
+                        {"ZoneScope" = "private"}
+                       )
+}
diff --git a/tools/terraform/modules/aws/route53/zone/private/outputs.tf b/tools/terraform/modules/aws/route53/zone/private/outputs.tf
new file mode 100644
index 000000000..9765daaa0
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/zone/private/outputs.tf
@@ -0,0 +1,13 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "id" {
+  value = aws_route53_zone.this.id
+}
+output "name" {
+  value = aws_route53_zone.this.name
+}
+output "name_servers" {
+  value = aws_route53_zone.this.name_servers
+}
diff --git a/tools/terraform/modules/aws/route53/zone/private/variables.tf b/tools/terraform/modules/aws/route53/zone/private/variables.tf
new file mode 100644
index 000000000..7afb16eb5
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/zone/private/variables.tf
@@ -0,0 +1,17 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+variable "zone_name"  {
+  description = "Public zone to create"
+  type        = string
+}
+variable "zone_config" {
+  description = "Zone's config parameters"
+  type    = map
+  default = {}
+}
+variable "tags"  {
+  type = map 
+  default = {}
+}
diff --git a/tools/terraform/modules/aws/route53/zone/public/main.tf b/tools/terraform/modules/aws/route53/zone/public/main.tf
new file mode 100644
index 000000000..9697ea217
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/zone/public/main.tf
@@ -0,0 +1,13 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+resource "aws_route53_zone" "this" {
+  name          = var.zone_name
+  comment       = lookup(var.zone_config, "comment", null)
+  force_destroy = lookup(var.zone_config, "force_destroy", null)
+  tags          = merge(
+                        var.tags,
+                        {"ZoneScope" = "public"}
+                       )
+}
diff --git a/tools/terraform/modules/aws/route53/zone/public/outputs.tf b/tools/terraform/modules/aws/route53/zone/public/outputs.tf
new file mode 100644
index 000000000..9765daaa0
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/zone/public/outputs.tf
@@ -0,0 +1,13 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "id" {
+  value = aws_route53_zone.this.id
+}
+output "name" {
+  value = aws_route53_zone.this.name
+}
+output "name_servers" {
+  value = aws_route53_zone.this.name_servers
+}
diff --git a/tools/terraform/modules/aws/route53/zone/public/variables.tf b/tools/terraform/modules/aws/route53/zone/public/variables.tf
new file mode 100644
index 000000000..7afb16eb5
--- /dev/null
+++ b/tools/terraform/modules/aws/route53/zone/public/variables.tf
@@ -0,0 +1,17 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+variable "zone_name"  {
+  description = "Public zone to create"
+  type        = string
+}
+variable "zone_config" {
+  description = "Zone's config parameters"
+  type    = map
+  default = {}
+}
+variable "tags"  {
+  type = map 
+  default = {}
+}
diff --git a/tools/terraform/route53.tf b/tools/terraform/route53.tf
new file mode 100644
index 000000000..97763d5ce
--- /dev/null
+++ b/tools/terraform/route53.tf
@@ -0,0 +1,28 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+## ZONE definition
+module "r53_zone_public" {
+  source      = "./modules/aws/route53/zone/public"
+  zone_name   = var.r53_domain_name
+  tags        = merge(
+                      {"Name"    = var.r53_domain_name,
+                       "Cluster" = var.cluster,
+                      },
+                      local.resource_tags,
+                     )
+}
+module "r53_zone_private" {
+  source      = "./modules/aws/route53/zone/private"
+  zone_name   = var.r53_domain_name
+  zone_config = {
+    vpc_id = var.manage_vpc ? module.vpc.*.vpc_id[0] : var.vpc_id
+  }
+  tags        = merge(
+                      {"Name"    = var.r53_domain_name,
+                       "Cluster" = var.cluster,
+                      },
+                      local.resource_tags,
+                     )
+}
diff --git a/tools/terraform/route53_outputs.tf b/tools/terraform/route53_outputs.tf
new file mode 100644
index 000000000..fad655867
--- /dev/null
+++ b/tools/terraform/route53_outputs.tf
@@ -0,0 +1,22 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "vpc_route53_private_zone_id" {
+  value = module.r53_zone_private.id
+}
+output "vpc_route53_private_zone_name" {
+  value = module.r53_zone_private.name
+}
+output "vpc_route53_private_name_servers" {
+  value = module.r53_zone_private.name_servers
+}
+output "vpc_route53_public_zone_id" {
+  value = module.r53_zone_public.id
+}
+output "vpc_route53_public_zone_name" {
+  value = module.r53_zone_public.name
+}
+output "vpc_route53_public_name_servers" {
+  value = module.r53_zone_public.name_servers
+}
diff --git a/tools/terraform/security_groups.tf b/tools/terraform/security_groups.tf
new file mode 100644
index 000000000..4c24e5ef4
--- /dev/null
+++ b/tools/terraform/security_groups.tf
@@ -0,0 +1,112 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+module "arvados_ssh_sg" {
+  count               = var.manage_security_groups ? 1 : 0
+  source              = "terraform-aws-modules/security-group/aws//modules/ssh"
+
+  name                = "${var.cluster}_ssh_sg"
+  description         = "SSH to Arvados VPC"
+  vpc_id              = var.manage_vpc ? module.vpc.*.vpc_id[0] : var.vpc_id
+
+  ingress_cidr_blocks = concat(
+                               var.allowed_full_access_cidrs,
+                               var.allowed_ssh_access_cidrs,
+                               var.private_subnets,
+                               var.public_subnets
+                              )
+  tags                = merge({"Name": "${var.cluster}-ssh-sg"},
+                              local.resource_tags)
+}
+module "arvados_http_sg" {
+  count               = var.manage_security_groups ? 1 : 0
+  source              = "terraform-aws-modules/security-group/aws//modules/http-80"
+  name                = "${var.cluster}_http_80_sg"
+  description         = "HTTP security group"
+  vpc_id              = var.manage_vpc ? module.vpc.*.vpc_id[0] : var.vpc_id
+  ingress_cidr_blocks = concat(
+                               var.allowed_full_access_cidrs,
+                               var.allowed_http_access_cidrs,
+                               var.private_subnets,
+                               var.public_subnets
+                              )
+  tags                = merge({"Name": "${var.cluster}-http-sg"},
+                              local.resource_tags)
+}
+module "arvados_https_sg" {
+  count               = var.manage_security_groups ? 1 : 0
+  source              = "terraform-aws-modules/security-group/aws//modules/https-443"
+  name                = "${var.cluster}_https_443_sg"
+  description         = "HTTPs security group"
+  vpc_id              = var.manage_vpc ? module.vpc.*.vpc_id[0] : var.vpc_id
+  ingress_cidr_blocks = concat(
+                               var.allowed_full_access_cidrs,
+                               var.allowed_http_access_cidrs,
+                               var.private_subnets,
+                               var.public_subnets
+                              )
+  tags                = merge({"Name": "${var.cluster}-https-sg"},
+                               local.resource_tags)
+}
+module "arvados_webshell_sg" {
+  count               = var.manage_security_groups ? 1 : 0
+  source              = "terraform-aws-modules/security-group/aws"
+
+  name                = "${var.cluster}_webshell_sg"
+  description         = "Arvados access to webshell server"
+  vpc_id              = var.manage_vpc ? module.vpc.*.vpc_id[0] : var.vpc_id
+
+  ingress_cidr_blocks = concat(
+                                var.allowed_full_access_cidrs,
+                                var.allowed_ssh_access_cidrs,
+                                var.private_subnets,
+                        )
+  ingress_with_cidr_blocks = [
+    {
+      from_port   = 4200
+      to_port     = 4200
+      protocol    = "tcp"
+      description = "Webshell port"
+      cidr_blocks = var.cluster_cidr
+    },
+  ]
+  tags                = merge({"Name": "${var.cluster}-webshell-sg"},
+                              local.resource_tags)
+}
+module "arvados_postgresql_sg" {
+  count               = var.manage_security_groups ? 1 : 0
+  source              = "terraform-aws-modules/security-group/aws//modules/postgresql"
+
+  name                = "${var.cluster}_postgresql_sg"
+  description         = "Arvados postgresql security group"
+  vpc_id              = var.manage_vpc ? module.vpc.*.vpc_id[0] : var.vpc_id
+
+  ingress_cidr_blocks = concat(
+                               var.allowed_full_access_cidrs,
+                               var.private_subnets,
+                              )
+
+  tags                = merge({"Name": "${var.cluster}-postgresql-sg"},
+                              local.resource_tags)
+}
+module "arvados_keepstore_sg" {
+  count               = var.manage_security_groups ? 1 : 0
+  source = "terraform-aws-modules/security-group/aws"
+
+  name        = "keepstore_sg"
+  description = "Arvados security group for the keepstore service"
+  vpc_id              = var.manage_vpc ? module.vpc.*.vpc_id[0] : var.vpc_id
+
+  ingress_cidr_blocks      = [var.vpc_cidr]
+  ingress_with_cidr_blocks = [
+    {
+      from_port   = 25107
+      to_port     = 25107
+      protocol    = "tcp"
+      description = "Keepstore port"
+      cidr_blocks = var.cluster_cidr
+    },
+  ]
+  tags        = merge({"Name": "keepstore-sg"},local.resource_tags)
+}
diff --git a/tools/terraform/security_groups_outputs.tf b/tools/terraform/security_groups_outputs.tf
new file mode 100644
index 000000000..c0f0f9d49
--- /dev/null
+++ b/tools/terraform/security_groups_outputs.tf
@@ -0,0 +1,28 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "arvados_ssh_security_group_id" {
+  description = "The ID of the arvados SSH security group"
+  value       = var.manage_security_groups ? module.arvados_ssh_sg.0.this_security_group_id : var.vpc_security_group_ids["ssh"]
+}
+output "arvados_webshell_security_group_id" {
+  description = "The ID of the arvados Webshell security group"
+  value       = var.manage_security_groups ? module.arvados_webshell_sg.0.this_security_group_id : var.vpc_security_group_ids["webshell"]
+}
+output "arvados_http_security_group_id" {
+  description = "The ID of the arvados HTTP security group"
+  value       = var.manage_security_groups ? module.arvados_http_sg.0.this_security_group_id : var.vpc_security_group_ids["http"]
+}
+output "arvados_https_security_group_id" {
+  description = "The ID of the arvados HTTPS security group"
+  value       = var.manage_security_groups ? module.arvados_https_sg.0.this_security_group_id : var.vpc_security_group_ids["https"]
+}
+output "arvados_postgresql_security_group_id" {
+  description = "The ID of the arvados Postgresql security group"
+  value       = var.manage_security_groups ? module.arvados_postgresql_sg.0.this_security_group_id : var.vpc_security_group_ids["postgresql"]
+}
+output "arvados_keepstore_security_group_id" {
+  description = "The ID of the arvados Keepstore security group"
+  value       = var.manage_security_groups ? module.arvados_keepstore_sg.0.this_security_group_id : var.vpc_security_group_ids["keepstore"]
+}
diff --git a/tools/terraform/shell_iam_role.tf b/tools/terraform/shell_iam_role.tf
new file mode 100644
index 000000000..7e11adf73
--- /dev/null
+++ b/tools/terraform/shell_iam_role.tf
@@ -0,0 +1,21 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+# Assume role for the instance
+resource "aws_iam_role" "shell_iam_role" {
+    name = "${var.cluster}-shell-iam-role"
+    assume_role_policy = templatefile("${path.module}/iam_policy_assume_role.json", {})
+}
+
+# Associate letsencrypt modification policy to the role
+resource "aws_iam_role_policy_attachment" "shell_letsencrypt_route53_policies_attachment" {
+    role       = aws_iam_role.shell_iam_role.name
+    policy_arn = aws_iam_policy.letsencrypt_route53_iam_policy.arn
+}
+
+# Add the role to the instance profile
+resource "aws_iam_instance_profile" "shell_instance_profile" {
+  name  = "shell_instance_profile"
+  role = "${var.cluster}-shell-iam-role"
+}
diff --git a/tools/terraform/shell_iam_role_outputs.tf b/tools/terraform/shell_iam_role_outputs.tf
new file mode 100644
index 000000000..23e364d5d
--- /dev/null
+++ b/tools/terraform/shell_iam_role_outputs.tf
@@ -0,0 +1,10 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "shell_iam_role_arn" {
+  value = aws_iam_role.shell_iam_role.arn
+}
+output "shell_iam_role_id" {
+  value = aws_iam_role.shell_iam_role.id
+}
diff --git a/tools/terraform/shell_instance.tf b/tools/terraform/shell_instance.tf
new file mode 100644
index 000000000..e7774612f
--- /dev/null
+++ b/tools/terraform/shell_instance.tf
@@ -0,0 +1,91 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+module "shell" {
+  source                 = "terraform-aws-modules/ec2-instance/aws"
+  version                = "~> 2.17.0"
+
+  name                   = "${var.cluster}-shell"
+  instance_count         = 1
+
+  iam_instance_profile   = "shell_instance_profile"
+  ami                    = try(var.instance_ami["shell"], var.instance_ami["default"])
+  instance_type          = try(var.instance_type["shell"], var.instance_type["default"])
+  key_name               = var.key_name
+  monitoring             = true
+
+  tags                   = merge({"Name": "${var.cluster}-shell",
+                                  "OsType": "LINUX"}, local.resource_tags)
+  volume_tags            = merge({"Name": "${var.cluster}-shell"}, local.resource_tags)
+
+  network_interface      = [{
+    device_index = 0,
+    network_interface_id = aws_network_interface.shell.id,
+  }]
+
+  # associate_public_ip_address = false
+  ebs_optimized               = true
+  user_data               = templatefile("_user_data.sh", {})
+
+  root_block_device           = [{
+    encrypted             = true,
+    kms_key_id            = var.kms_key_id,
+    volume_size           = var.root_bd_size,
+    delete_on_termination = true,
+  }]
+  ebs_block_device            = [{
+    encrypted             = true,
+    kms_key_id            = var.kms_key_id,
+    volume_size           = try(var.data_bd_size["shell"], var.data_bd_size["default"])
+    delete_on_termination = true,
+    device_name           = "xvdh",
+  }]
+}
+
+resource "aws_eip" "cluster_shell_public_ip" {
+  vpc      = true
+  instance = module.shell.id[0]
+  network_interface = aws_network_interface.shell.id
+  tags     = merge({"Name": "${var.cluster}-shell-ip"},local.resource_tags)
+}
+
+resource "aws_network_interface" "shell" {
+  subnet_id       = var.manage_vpc ? module.vpc.0.public_subnets[0] : var.public_subnets_ids[0]
+  # private_ips     = [cidrhost(var.vpc_subnet_cidrs[0], var.host_number["shell"])]
+  security_groups = [
+                     local.ssh_sg,
+                     local.webshell_sg,
+                    ]
+  tags           = merge({"Name": "${var.cluster}-shell"}, local.resource_tags)
+}
+
+## Public A RRs
+module "shell_route53_public_records_A" {
+  source         = "./modules/aws/route53/records/a"
+  zone_id        = module.r53_zone_public.id
+
+  zone_records_A = {
+    "shell" = {
+      ttl     = "300",
+      records = [aws_eip.cluster_shell_public_ip.public_ip]
+    },
+  }
+}
+
+## Private A RRs
+module "shell_route53_private_records_A" {
+  source         = "./modules/aws/route53/records/a"
+  zone_id        = module.r53_zone_private.id
+
+  zone_records_A = {
+    "shell" = {
+      ttl     = "300",
+      records = aws_network_interface.shell.private_ips
+    },
+    "webshell" = {
+      ttl     = "300",
+      records = aws_network_interface.shell.private_ips
+    },
+  }
+}
diff --git a/tools/terraform/shell_instance_outputs.tf b/tools/terraform/shell_instance_outputs.tf
new file mode 100644
index 000000000..6be74d481
--- /dev/null
+++ b/tools/terraform/shell_instance_outputs.tf
@@ -0,0 +1,19 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "shell_id" {
+  value = module.shell.id
+}
+output "shell_private_dns_names" {
+  value = module.shell.private_dns
+}
+output "shell_private_ip" {
+  value = module.shell.private_ip
+}
+output "shell_private_eni_id" {
+  value = aws_network_interface.shell.id
+}
+output "shell_public_ip" {
+  value = aws_eip.cluster_shell_public_ip.public_ip
+}
diff --git a/tools/terraform/terraform.tfvars b/tools/terraform/terraform.tfvars
new file mode 100644
index 000000000..46ea86971
--- /dev/null
+++ b/tools/terraform/terraform.tfvars
@@ -0,0 +1,84 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+### GENERAL
+aws_profile              = "profile-to-use"
+aws_region               = "us-east-1"
+environment              = "production"
+namespace                = "3rd-party deploy test"
+
+### KEYPAIR
+key_name                 = "keyname"
+key_path                 = "~/.ssh/id_rsa.pub"
+
+cluster                  = "vwxyz"
+r53_domain_name          = "vwxyz.arvados.test"
+
+# VPC
+# If you have/want to use a VPC already defined, set this value to false
+# and uncomment and provide values for the following variables
+
+# manage_vpc               = true
+# vpc_id                   = "vpc-12345678901234567"
+# private_subnets_ids      = []
+# compute_subnets_ids      = []
+# public_subnets_ids       = []
+
+cluster_cidr             = "10.0.0.0/16"
+azs                      = ["us-east-1a"]
+private_subnets          = ["10.0.255.0/24"]
+compute_subnets          = ["10.0.254.0/24"]
+public_subnets           = ["10.0.0.0/24"]
+enable_nat_gateway       = true
+enable_vpn_gateway       = false
+single_nat_gateway       = true
+one_nat_gateway_per_az   = false
+enable_dhcp_options      = true
+
+instance_type = {
+  "default"   = "m5a.large",
+  # "api"       = "m5a.large",
+  # "shell"     = "m5a.large",
+  # "keepproxy" = "m5a.large",
+  # "keepstore" = "m5a.large",
+  # "workbench" = "m5a.large",
+  # "database"  = "m5a.large",
+}
+instance_ami = {
+  "default"   = "ami-07d02ee1eeb0c996c",
+  # "api"       = "ami-07d02ee1eeb0c996c",
+  # "shell"     = "ami-07d02ee1eeb0c996c",
+  # "keepstore" = "ami-07d02ee1eeb0c996c",
+  # "keepproxy" = "ami-07d02ee1eeb0c996c",
+  # "workbench" = "ami-07d02ee1eeb0c996c",
+  # "database"  = "ami-07d02ee1eeb0c996c",
+}
+
+data_bd_size = {
+  "default"   = 50,
+  # "api"       = 50,
+  # "shell"     = 50,
+  # "keepproxy" = 50,
+  # "keepstore" = 50,
+  # "workbench" = 50,
+   "database"  = 250,
+}
+# KEEPSTORE/s
+keepstore_count = 2
+
+# SECURITY
+# CIDRs allowed unrestricted access to the instances
+# allowed_access_cidrs = "0.0.0.0/0"
+
+# If you have/want to use already defined security groups, set this value to false
+# and uncomment and provide values for the following variables
+# vpc_security_group_ids = {
+#   "default"    = "sg-01111111111111110",
+#   "ssh"        = "sg-01234567890123456",
+#   "http"       = "sg-12345678901234567",
+#   "https"      = "sg-23456789012345678",
+#   "webshell"   = "sg-34567890123456789",
+#   "postgresql" = "sg-45678901234567890",
+#   "keepstore"  = "sg-56789012345678901",
+# }
diff --git a/tools/terraform/variables.tf b/tools/terraform/variables.tf
new file mode 100644
index 000000000..902ff4563
--- /dev/null
+++ b/tools/terraform/variables.tf
@@ -0,0 +1,243 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+### GENERAL
+variable "aws_region" {
+  description = "The AWS region where to deploy the cluster"
+  type        = string
+  default     = ""
+}
+variable "aws_profile" {
+  description = "The AWS profile to use"
+  type        = string
+  default     = ""
+}
+variable "ami" {
+  description = "The AMI to use when launching instances"
+  # This is Debian buster in us-east-1
+  type        = string
+  default     = "ami-07d02ee1eeb0c996c"
+}
+variable "namespace" {
+  description = "A descriptive name for the resources' namespace"
+  type        = string
+  default     = ""
+}
+variable "environment" {
+  description = "A descriptive name for the cluster's environment"
+  type        = string
+  default     = ""
+}
+variable "tags" {
+  description = "Tags to add to all the environment resources, beside the default ones (Environment, Namespace, Terraform)"
+  type        = map
+  default     = {}
+}
+
+### KEYPAIRS
+variable "key_path" {
+  description = "Where the keypair is localted (e.g. `~/.ssh/id_rsa.pub`)"
+  type        = string
+  default     = "" 
+}
+variable "key_name" {
+  description = "Name to give to the keypair"
+  type        = string
+  default     = ""
+}
+variable "key_public_key" {
+  description = "Public key value if you didn't provide a path to the key"
+  type        = string
+  default     = ""
+}
+variable "enable_key_pair" {
+  description = "A boolean flag to enable/disable key pair"
+  type        = bool
+  default     = true
+}
+
+### VPC
+variable "cluster" {
+  type    = string
+}
+variable "cluster_cidr" {
+  type = string
+  default = ""
+}
+variable "azs" {
+  type = list(string)
+  default = []
+}
+variable "private_subnets" {
+  type = list(string)
+  default = []
+}
+variable "public_subnets" {
+  type = list(string)
+  default = []
+}
+variable "compute_subnets" {
+  type = list(string)
+  default = []
+}
+variable "private_subnets_ids" {
+  type = list(string)
+  default = []
+}
+variable "public_subnets_ids" {
+  type = list(string)
+  default = []
+}
+variable "compute_subnets_ids" {
+  type = list(string)
+  default = []
+}
+variable "enable_nat_gateway" {
+  type = bool
+  default = true
+}
+variable "enable_vpn_gateway" {
+  type = bool
+  default = false
+}
+variable "single_nat_gateway" {
+  type = bool
+  default = true
+}
+variable "one_nat_gateway_per_az" {
+  type = bool
+  default = false
+}
+variable "route53_force_destroy" {
+  description = "Destroy R53 zone when VPC is destroyed"
+  type = bool
+  default = false
+}
+variable "enable_dhcp_options" {
+  type = bool
+  default = false
+}
+variable "allowed_http_access_cidrs" {
+  description = "CIDRs that will have HTTP/HTTPs access to the cluster's VPC instances"
+  type        = list(string)
+  default     = ["0.0.0.0/0"]
+}
+variable "allowed_ssh_access_cidrs" {
+  description = "CIDRs that will have ssh access to the cluster's VPC instances"
+  type        = list(string)
+  default     = []
+}
+variable "allowed_full_access_cidrs" {
+  description = "CIDRs that will have full access to the cluster's VPC instances"
+  type        = list(string)
+  default     = []
+}
+
+variable "kms_key_id" {
+  default = ""
+}
+
+variable "key_pair" {
+  default = ""
+}
+
+variable "instance_ami" {
+  type    = map(string)
+  default = {}
+}
+
+variable "instance_type" {
+  type    = map(string)
+  default = {}
+}
+
+variable "arvados_volume_name" {
+  type    = string
+  default = ""
+}
+
+variable "keepstore_count" {
+  type    = string
+  default = "1"
+}
+
+variable "root_bd_size" {
+  type    = number
+  default = 50
+}
+
+variable "data_bd_size" {
+  type    = map(number)
+  default = {}
+}
+
+variable "db_identifier" {
+  type    = string
+  default = ""
+}
+
+variable "db_engine_version" {
+  type    = string
+  default = "11.8"
+}
+
+variable "db_instance_class" {
+  type    = string
+  default = "db.t2.large"
+}
+
+variable "db_allocated_storage" {
+  type    = number
+  default = 512
+}
+
+variable "db_storage_encrypted" {
+  type    = bool
+  default = true
+}
+
+variable "db_permissions_boundary" {
+  type    = string
+  default = ""
+}
+
+variable "manage_vpc" {
+  description = "If you want to manage/create the VPC where the cluster will be deployed with terraform"
+  type    = bool
+  default = true
+}
+
+variable "vpc_id" {
+  description = "If you are not managing the vpc with terraform, then you need to provide the VPC ID"
+  type    = string
+  default = ""
+}
+
+variable "manage_security_groups" {
+  description = "If you want to manage/create the security groups with terraform"
+  type    = bool
+  default = true
+}
+
+variable "vpc_security_group_ids" {
+  description = "If you are not managing the security groups, then you need to provide them"
+  type    = map(string)
+  default = {}
+}
+
+variable "vpc_cidr" {
+  type    = string
+  default = ""
+
+}
+variable "vpc_subnet_cidrs" {
+  type    = list(string)
+  default = []
+}
+variable "r53_domain_name" {
+  description = "Domain which will be appended to the cluster name"
+  type    = string
+  default = ""
+}
+
diff --git a/tools/terraform/vpc.tf b/tools/terraform/vpc.tf
new file mode 100644
index 000000000..539c24eb9
--- /dev/null
+++ b/tools/terraform/vpc.tf
@@ -0,0 +1,32 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+module "vpc" {
+  count                    = var.manage_vpc ? 1 : 0
+  source                   = "terraform-aws-modules/vpc/aws"
+  version                  = "2.77.0"
+
+  name                     = var.cluster
+  cidr                     = var.cluster_cidr
+
+  azs                      = var.azs
+  # we'll need internet access in the compute nodes, so the compute_subnets will
+  # go to the private_subnets
+  private_subnets          = concat(var.private_subnets, var.compute_subnets)
+  public_subnets           = var.public_subnets
+
+  enable_dns_hostnames     = true
+  enable_dns_support       = true
+
+  enable_s3_endpoint       = true
+
+  enable_nat_gateway       = var.enable_nat_gateway
+  enable_vpn_gateway       = var.enable_vpn_gateway
+  single_nat_gateway       = var.single_nat_gateway
+  one_nat_gateway_per_az   = var.one_nat_gateway_per_az
+
+  enable_dhcp_options      = var.enable_dhcp_options
+  dhcp_options_domain_name = var.r53_domain_name
+  tags                     = local.resource_tags
+}
diff --git a/tools/terraform/vpc_outputs.tf b/tools/terraform/vpc_outputs.tf
new file mode 100644
index 000000000..a2641199e
--- /dev/null
+++ b/tools/terraform/vpc_outputs.tf
@@ -0,0 +1,40 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "vpc_id" {
+  value = var.manage_vpc ? module.vpc.0.vpc_id : var.vpc_id
+}
+output "cluster" {
+  value = var.cluster
+}
+output "vpc_name" {
+  value = var.manage_vpc ? module.vpc.0.name : var.cluster
+}
+output "vpc_cidr" {
+  value = var.cluster_cidr
+}
+output "vpc_azs" {
+  value = var.manage_vpc ? module.vpc.0.azs : var.azs
+}
+output "vpc_private_subnets_ids" {
+  value = var.manage_vpc ? module.vpc.0.private_subnets : concat(var.private_subnets_ids, var.compute_subnets_ids)
+}
+output "vpc_compute_subnets_ids" {
+  value = var.manage_vpc ? [module.vpc.0.private_subnets[1]] : var.compute_subnets_ids
+}
+output "vpc_public_subnets_ids" {
+  value = var.manage_vpc ? module.vpc.0.public_subnets : var.public_subnets_ids
+}
+output "vpc_nat_public_ips" {
+  value = var.manage_vpc ? module.vpc.0.nat_public_ips : null
+}
+output "vpc_private_subnets_cidr_blocks" {
+  value = var.manage_vpc ? module.vpc.0.private_subnets_cidr_blocks : concat(var.private_subnets, var.compute_subnets)
+}
+output "vpc_compute_subnets_cidr_blocks" {
+  value = var.manage_vpc ? [module.vpc.0.private_subnets_cidr_blocks[1]] : var.compute_subnets
+}
+output "vpc_public_subnets_cidr_blocks" {
+  value = var.manage_vpc ? module.vpc.0.public_subnets_cidr_blocks : var.public_subnets
+}
diff --git a/tools/terraform/workbench_iam_role.tf b/tools/terraform/workbench_iam_role.tf
new file mode 100644
index 000000000..0774b9ca7
--- /dev/null
+++ b/tools/terraform/workbench_iam_role.tf
@@ -0,0 +1,21 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+# Assume role for the instance
+resource "aws_iam_role" "workbench_iam_role" {
+    name = "${var.cluster}-workbench-iam-role"
+    assume_role_policy = templatefile("${path.module}/iam_policy_assume_role.json", {})
+}
+
+# Associate letsencrypt modification policy to the role
+resource "aws_iam_role_policy_attachment" "workbench_letsencrypt_route53_policies_attachment" {
+    role       = aws_iam_role.workbench_iam_role.name
+    policy_arn = aws_iam_policy.letsencrypt_route53_iam_policy.arn
+}
+
+# Add the role to the instance profile
+resource "aws_iam_instance_profile" "workbench_instance_profile" {
+  name  = "workbench_instance_profile"
+  role = "${var.cluster}-workbench-iam-role"
+}
diff --git a/tools/terraform/workbench_iam_role_outputs.tf b/tools/terraform/workbench_iam_role_outputs.tf
new file mode 100644
index 000000000..d8770705a
--- /dev/null
+++ b/tools/terraform/workbench_iam_role_outputs.tf
@@ -0,0 +1,10 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "workbench_iam_role_arn" {
+  value = aws_iam_role.workbench_iam_role.arn
+}
+output "workbench_iam_role_id" {
+  value = aws_iam_role.workbench_iam_role.id
+}
diff --git a/tools/terraform/workbench_instance.tf b/tools/terraform/workbench_instance.tf
new file mode 100644
index 000000000..31e147ee4
--- /dev/null
+++ b/tools/terraform/workbench_instance.tf
@@ -0,0 +1,96 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+module "workbench" {
+  source                 = "terraform-aws-modules/ec2-instance/aws"
+  version                = "~> 2.17.0"
+
+  name                   = "${var.cluster}-workbench"
+  instance_count         = 1
+
+  iam_instance_profile   = "workbench_instance_profile"
+  ami                    = try(var.instance_ami["workbench"], var.instance_ami["default"])
+  instance_type          = try(var.instance_type["workbench"], var.instance_type["default"])
+  key_name               = var.key_name
+  monitoring             = true
+
+  tags                   = merge({"Name": "${var.cluster}-workbench",
+                                  "OsType": "LINUX"}, local.resource_tags)
+  volume_tags            = merge({"Name": "${var.cluster}-workbench"}, local.resource_tags)
+
+  network_interface      = [{
+    device_index = 0,
+    network_interface_id = aws_network_interface.workbench.id,
+  }]
+
+  # associate_public_ip_address = false
+  ebs_optimized          = true
+  user_data               = templatefile("_user_data.sh", {})
+
+  root_block_device           = [{
+    encrypted             = true,
+    kms_key_id            = var.kms_key_id,
+    volume_size           = var.root_bd_size,
+    delete_on_termination = true,
+  }]
+  ebs_block_device            = [{
+    encrypted             = true,
+    kms_key_id            = var.kms_key_id,
+    volume_size           = try(var.data_bd_size["workbench"], var.data_bd_size["default"])
+    delete_on_termination = true,
+    device_name           = "xvdh",
+  }]
+}
+
+resource "aws_eip" "cluster_workbench_public_ip" {
+  vpc      = true
+  instance = module.workbench.id[0]
+  network_interface = aws_network_interface.workbench.id
+  tags     = merge({"Name": "${var.cluster}-workbench-ip"},local.resource_tags)
+}
+
+resource "aws_network_interface" "workbench" {
+  subnet_id       = var.manage_vpc ? module.vpc.0.public_subnets[0] : var.public_subnets_ids[0]
+  # private_ips     = [cidrhost(var.vpc_subnet_cidrs[0], var.host_number["workbench"])]
+  security_groups = [
+                     local.ssh_sg,
+                     local.http_sg,
+                     local.https_sg,
+                    ]
+  tags           = merge({"Name": "${var.cluster}-workbench"}, local.resource_tags)
+}
+
+## Public A RRs
+module "workbench_route53_public_records_A" {
+  source         = "./modules/aws/route53/records/a"
+  zone_id        = module.r53_zone_public.id
+
+  zone_records_A = {
+    "workbench" = {
+      ttl     = "300",
+      records = [aws_eip.cluster_workbench_public_ip.public_ip]
+    },
+    "workbench2" = {
+      ttl     = "300",
+      records = [aws_eip.cluster_workbench_public_ip.public_ip]
+    },
+  }
+}
+
+## Private A RRs
+module "workbench_route53_private_records_A" {
+  source         = "./modules/aws/route53/records/a"
+  zone_id        = module.r53_zone_private.id
+
+  zone_records_A = {
+    "workbench" = {
+      ttl     = "300",
+      records = aws_network_interface.workbench.private_ips
+    },
+    "workbench2" = {
+      ttl     = "300",
+      records = aws_network_interface.workbench.private_ips
+    },
+  }
+}
diff --git a/tools/terraform/workbench_instance_outputs.tf b/tools/terraform/workbench_instance_outputs.tf
new file mode 100644
index 000000000..82e2b9717
--- /dev/null
+++ b/tools/terraform/workbench_instance_outputs.tf
@@ -0,0 +1,19 @@
+# Copyright (C) The Arvados Authors. All rights reserved.
+#
+# SPDX-License-Identifier: Apache-2.0
+
+output "workbench_id" {
+  value = module.workbench.id
+}
+output "workbench_private_dns_names" {
+  value = module.workbench.private_dns
+}
+output "workbench_private_ip" {
+  value = module.workbench.private_ip
+}
+output "workbench_private_eni_id" {
+  value = aws_network_interface.workbench.id
+}
+output "workbench_public_ip" {
+  value = aws_eip.cluster_workbench_public_ip.public_ip
+}

-----------------------------------------------------------------------


hooks/post-receive
-- 




More information about the arvados-commits mailing list