[ARVADOS] created: 1.3.0-2738-g649a637ca
Git user
git at public.arvados.org
Mon Jun 29 17:58:35 UTC 2020
at 649a637ca3546b775bef2d650ec60798f94b3d8e (commit)
commit 649a637ca3546b775bef2d650ec60798f94b3d8e
Author: Peter Amstutz <peter.amstutz at curii.com>
Date: Mon Jun 29 13:58:16 2020 -0400
15964: Add a line about storing Dockerfiles in source control
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz at curii.com>
diff --git a/doc/user/topics/arv-docker.html.textile.liquid b/doc/user/topics/arv-docker.html.textile.liquid
index f039a6c3e..d371400f5 100644
--- a/doc/user/topics/arv-docker.html.textile.liquid
+++ b/doc/user/topics/arv-docker.html.textile.liquid
@@ -45,6 +45,8 @@ The "RUN" command is executed inside the container and can be any shell command
You can also visit the "Docker tutorial":https://docs.docker.com/get-started/part2/ for more information and examples.
+You should add your Dockerfiles to the same source control repository as the Workflows that use them.
+
h3. Create a new image
We're now ready to create a new Docker image. Use @docker build@ to create a new image from the Dockerfile.
commit d8dc80d74aeb66cb9e736a3cb9525f17c4b22a8e
Author: Peter Amstutz <peter.amstutz at curii.com>
Date: Mon Jun 29 13:56:11 2020 -0400
15964: Rewrite docker tutorial to reflect current best practices
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz at curii.com>
diff --git a/doc/user/topics/arv-docker.html.textile.liquid b/doc/user/topics/arv-docker.html.textile.liquid
index e9e845026..f039a6c3e 100644
--- a/doc/user/topics/arv-docker.html.textile.liquid
+++ b/doc/user/topics/arv-docker.html.textile.liquid
@@ -1,7 +1,7 @@
---
layout: default
navsection: userguide
-title: "Customizing Crunch environment using Docker"
+title: "Working with Docker images"
...
{% comment %}
Copyright (C) The Arvados Authors. All rights reserved.
@@ -9,130 +9,64 @@ Copyright (C) The Arvados Authors. All rights reserved.
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-This page describes how to customize the runtime environment (e.g., the programs, libraries, and other dependencies needed to run a job) that a crunch script will be run in using "Docker.":https://www.docker.com/ Docker is a tool for building and running containers that isolate applications from other applications running on the same node. For detailed information about Docker, see the "Docker User Guide.":https://docs.docker.com/userguide/
+This page describes how to set up the runtime environment (e.g., the programs, libraries, and other dependencies needed to run a job) that a workflow step will be run in using "Docker.":https://www.docker.com/ Docker is a tool for building and running containers that isolate applications from other applications running on the same node. For detailed information about Docker, see the "Docker User Guide.":https://docs.docker.com/userguide/
-This page will demonstrate how to:
+This page will describe:
-# Fetch the arvados/jobs Docker image
-# Manually install additional software into the container
-# Create a new custom image
-# Upload that image to Arvados for use by Crunch jobs
-# Share your image with others
+# Create a custom image using a Dockerfile
+# Uploading an image to Arvados
+# Sources of pre-built bioinformatics Docker images
{% include 'tutorial_expectations_workstation' %}
You also need ensure that "Docker is installed,":https://docs.docker.com/installation/ the Docker daemon is running, and you have permission to access Docker. You can test this by running @docker version at . If you receive a permission denied error, your user account may need to be added to the @docker@ group. If you have root access, you can add yourself to the @docker@ group using @$ sudo addgroup $USER docker@ then log out and log back in again; otherwise consult your local sysadmin.
-h2. Fetch a starting image
+h2. Create a custom image using a Dockerfile
-The easiest way to begin is to start from the "arvados/jobs" image which already has the Arvados SDK installed along with other configuration required for use with Crunch.
+This example shows how to create a Docker image and add the R package.
-Download the latest "arvados/jobs" image from the Docker registry:
+First, create new directory called @docker-example@, in that directory create a file called @Dockerfile at .
<notextile>
-<pre><code>$ <span class="userinput">docker pull arvados/jobs:latest</span>
-Pulling repository arvados/jobs
-3132168f2acb: Download complete
-a42b7f2c59b6: Download complete
-e5afdf26a7ae: Download complete
-5cae48636278: Download complete
-7a4f91b70558: Download complete
-a04a275c1fd6: Download complete
-c433ff206a22: Download complete
-b2e539b45f96: Download complete
-073b2581c6be: Download complete
-593915af19dc: Download complete
-32260b35005e: Download complete
-6e5b860c1cde: Download complete
-95f0bfb43d4d: Download complete
-c7fd77eedb96: Download complete
-0d7685aafd00: Download complete
+<pre><code>
+$ mkdir docker-example-r-base
+$ cd docker-example-r-base
</code></pre>
</notextile>
-h2. Install new packages
-
-Next, enter the container using @docker run@, providing the arvados/jobs image and the program you want to run (in this case the bash shell).
-
<notextile>
-<pre><code>$ <span class="userinput">docker run --interactive --tty --user root arvados/jobs /bin/bash</span>
-root at fbf1d0f529d5:/#
+<pre><code>
+FROM ubuntu:bionic
+RUN apt-get update && apt-get -yq --no-install-recommends install r-base-core
</code></pre>
</notextile>
-Next, update the package list using @apt-get update at .
+The "RUN" command is executed inside the container and can be any shell command line. You are not limited to installing Debian packages. You may compile programs or libraries from source and install them, edit systemwide configuration files, use other package managers such as @pip@ or @gem@, and perform any other customization necessary to run your program.
-<notextile>
-<pre><code>root at fbf1d0f529d5:/# apt-get update
-Get:2 http://apt.arvados.org stretch-dev InRelease [3260 B]
-Get:1 http://security-cdn.debian.org/debian-security stretch/updates InRelease [94.3 kB]
-Ign:3 http://cdn-fastly.deb.debian.org/debian stretch InRelease
-Get:4 http://cdn-fastly.deb.debian.org/debian stretch-updates InRelease [91.0 kB]
-Get:5 http://apt.arvados.org stretch-dev/main amd64 Packages [208 kB]
-Get:6 http://cdn-fastly.deb.debian.org/debian stretch Release [118 kB]
-Get:7 http://security-cdn.debian.org/debian-security stretch/updates/main amd64 Packages [499 kB]
-Get:8 http://cdn-fastly.deb.debian.org/debian stretch Release.gpg [2434 B]
-Get:9 http://cdn-fastly.deb.debian.org/debian stretch-updates/main amd64 Packages.diff/Index [10.6 kB]
-Get:10 http://cdn-fastly.deb.debian.org/debian stretch-updates/main amd64 Packages 2019-07-08-0821.07.pdiff [445 B]
-Get:10 http://cdn-fastly.deb.debian.org/debian stretch-updates/main amd64 Packages 2019-07-08-0821.07.pdiff [445 B]
-Fetched 1026 kB in 0s (1384 kB/s)
-Reading package lists... Done
-</code></pre>
-</notextile>
+You can also visit the "Docker tutorial":https://docs.docker.com/get-started/part2/ for more information and examples.
+
+h3. Create a new image
-In this example, we will install the "R" statistical language Debian package "r-base-core". Use @apt-get install@:
+We're now ready to create a new Docker image. Use @docker build@ to create a new image from the Dockerfile.
<notextile>
-<pre><code>root at fbf1d0f529d5:/# <span class="userinput">apt-get install r-base-core</span>
-Reading package lists... Done
-Building dependency tree
-Reading state information... Done
-The following additional packages will be installed:
-[...]
-done.
+<pre><code>
+docker-example-r-base$ docker build -t docker-example-r-base .
</code></pre>
</notextile>
+h3. Verify image
+
Now we can verify that "R" is installed:
<notextile>
-<pre><code>root at fbf1d0f529d5:/# <span class="userinput">R</span>
+<pre><code>$ docker run -ti docker-example-r-base
+root at 57ec8f8b2663:/# R
-R version 3.3.3 (2017-03-06) -- "Another Canoe"
-Copyright (C) 2017 The R Foundation for Statistical Computing
+R version 3.4.4 (2018-03-15) -- "Someone to Lean On"
+Copyright (C) 2018 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
-R is free software and comes with ABSOLUTELY NO WARRANTY.
-You are welcome to redistribute it under certain conditions.
-Type 'license()' or 'licence()' for distribution details.
-
-R is a collaborative project with many contributors.
-Type 'contributors()' for more information and
-'citation()' on how to cite R or R packages in publications.
-
-Type 'demo()' for some demos, 'help()' for on-line help, or
-'help.start()' for an HTML browser interface to help.
-Type 'q()' to quit R.
-
->
-</code></pre>
-</notextile>
-
-Note that you are not limited to installing Debian packages. You may compile programs or libraries from source and install them, edit systemwide configuration files, use other package managers such as @pip@ or @gem@, and perform any other customization necessary to run your program.
-
-h2. Create a new image
-
-We're now ready to create a new Docker image. First, quit the container, then use @docker commit@ to create a new image from the stopped container. The container id can be found in the default hostname of the container displayed in the prompt, in this case @fbf1d0f529d5@:
-
-<notextile>
-<pre><code>root at fbf1d0f529d5:/# <span class="userinput">exit</span>
-$ <span class="userinput">docker commit fbf1d0f529d5 arvados/jobs-with-r</span>
-sha256:2818853ff9f9af5d7f77979803baac9c4710790ad2b84c1a754b02728fdff205
-$ <span class="userinput">docker images</span>
-$ docker images |head
-REPOSITORY TAG IMAGE ID CREATED SIZE
-arvados/jobs-with-r latest 2818853ff9f9 9 seconds ago 703.1 MB
-arvados/jobs latest 12b9f859d48c 4 days ago 362 MB
</code></pre>
</notextile>
@@ -141,13 +75,16 @@ h2. Upload your image
Finally, we are ready to upload the new Docker image to Arvados. Use @arv-keepdocker@ with the image repository name to upload the image. Without arguments, @arv-keepdocker@ will print out the list of Docker images in Arvados that are available to you.
<notextile>
-<pre><code>$ <span class="userinput">arv-keepdocker arvados/jobs-with-r</span>
-703M / 703M 100.0%
-Collection saved as 'Docker image arvados/jobs-with-r:latest 2818853ff9f9'
-qr1hi-4zz18-abcdefghijklmno
-$ <span class="userinput">arv-keepdocker</span>
+<pre><code>$ <span class="userinput">arv-keepdocker docker-example-r-base</span>
+2020-06-29 13:48:19 arvados.arv_put[769] INFO: Creating new cache file at /home/peter/.cache/arvados/arv-put/39ddb51ebf6c5fcb3d713b5969466967
+206M / 206M 100.0% 2020-06-29 13:48:21 arvados.arv_put[769] INFO:
+
+2020-06-29 13:48:21 arvados.arv_put[769] INFO: Collection saved as 'Docker image docker-example-r-base:latest sha256:edd10'
+x20vs-4zz18-0tayximqcyb6uf8
+
+$ <span class="userinput">arv-keepdocker images</span>
REPOSITORY TAG IMAGE ID COLLECTION CREATED
-arvados/jobs-with-r latest 2818853ff9f9 qr1hi-4zz18-abcdefghijklmno Tue Jan 17 20:35:53 2017
+docker-example-r-base latest sha256:edd10 x20vs-4zz18-0tayximqcyb6uf8 Mon Jun 29 17:46:16 2020
</code></pre>
</notextile>
@@ -156,14 +93,24 @@ You are now able to specify the runtime environment for your program using @Dock
<pre>
hints:
DockerRequirement:
- dockerPull: arvados/jobs-with-r
+ dockerPull: docker-example-r-base
</pre>
-h2. Share Docker images
+h3. Uploading Docker images to a shared project
-Docker images are subject to normal Arvados permissions. If wish to share your Docker image with others (or wish to share a pipeline template that uses your Docker image) you will need to use @arv-keepdocker@ with the @--project-uuid@ option to upload the image to a shared project.
+Docker images are subject to normal Arvados permissions. If wish to share your Docker image with others you should use @arv-keepdocker@ with the @--project-uuid@ option to add the image to a shared project and ensure that metadata is set correctly.
<notextile>
-<pre><code>$ <span class="userinput">arv-keepdocker arvados/jobs-with-r --project-uuid qr1hi-j7d0g-xxxxxxxxxxxxxxx</span>
+<pre><code>$ <span class="userinput">arv-keepdocker docker-example-r-base --project-uuid qr1hi-j7d0g-xxxxxxxxxxxxxxx</span>
</code></pre>
</notextile>
+
+h2. Sources of pre-built images
+
+In addition to creating your own contianers, there are a number of resources where you can find bioinformatics tools already wrapped in container images:
+
+"BioContainers":https://biocontainers.pro/
+
+"Dockstore":https://dockstore.org/
+
+"Docker Hub":https://hub.docker.com/
commit 459ee5684c245b8cc960a4c0a0cc2825e85ec0e5
Author: Peter Amstutz <peter.amstutz at curii.com>
Date: Fri Jun 26 17:41:48 2020 -0400
15964: Removing mentions of qr1hi from docs. Cleaning up user guide.
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz at curii.com>
diff --git a/doc/_config.yml b/doc/_config.yml
index 3b59cbca4..d1fc43154 100644
--- a/doc/_config.yml
+++ b/doc/_config.yml
@@ -28,26 +28,25 @@ navbar:
- user/getting_started/workbench.html.textile.liquid
- user/tutorials/tutorial-workflow-workbench.html.textile.liquid
- user/composer/composer.html.textile.liquid
+ - Working at the Command Line:
+ - user/getting_started/setup-cli.html.textile.liquid
+ - user/reference/api-tokens.html.textile.liquid
+ - user/getting_started/check-environment.html.textile.liquid
- Access an Arvados virtual machine:
- user/getting_started/vm-login-with-webshell.html.textile.liquid
- user/getting_started/ssh-access-unix.html.textile.liquid
- user/getting_started/ssh-access-windows.html.textile.liquid
- - user/getting_started/check-environment.html.textile.liquid
- - user/reference/api-tokens.html.textile.liquid
- Working with data sets:
- user/tutorials/tutorial-keep.html.textile.liquid
- user/tutorials/tutorial-keep-get.html.textile.liquid
- user/tutorials/tutorial-keep-mount-gnu-linux.html.textile.liquid
- user/tutorials/tutorial-keep-mount-os-x.html.textile.liquid
- user/tutorials/tutorial-keep-mount-windows.html.textile.liquid
- - user/topics/keep.html.textile.liquid
- user/tutorials/tutorial-keep-collection-lifecycle.html.textile.liquid
- user/topics/arv-copy.html.textile.liquid
- - user/topics/storage-classes.html.textile.liquid
- user/topics/collection-versioning.html.textile.liquid
- - Working with git repositories:
- - user/tutorials/add-new-repository.html.textile.liquid
- - user/tutorials/git-arvados-guide.html.textile.liquid
+ - user/topics/storage-classes.html.textile.liquid
+ - user/topics/keep.html.textile.liquid
- Running workflows at the command line:
- user/cwl/cwl-runner.html.textile.liquid
- user/cwl/cwl-run-options.html.textile.liquid
@@ -59,6 +58,9 @@ navbar:
- user/cwl/cwl-extensions.html.textile.liquid
- user/cwl/cwl-versions.html.textile.liquid
- user/topics/arv-docker.html.textile.liquid
+ - Working with git repositories:
+ - user/tutorials/add-new-repository.html.textile.liquid
+ - user/tutorials/git-arvados-guide.html.textile.liquid
- Reference:
- user/topics/link-accounts.html.textile.liquid
- user/reference/cookbook.html.textile.liquid
diff --git a/doc/_includes/_ssh_addkey.liquid b/doc/_includes/_ssh_addkey.liquid
index 7a8a992b6..de0da6a76 100644
--- a/doc/_includes/_ssh_addkey.liquid
+++ b/doc/_includes/_ssh_addkey.liquid
@@ -18,6 +18,10 @@ Paste your public key into the text area labeled *Public Key*, and click on the
h1(#login). Using SSH to log into an Arvados VM
-To see a list of virtual machines that you have access to and determine the name and login information, click on the dropdown menu icon <span class="fa fa-lg fa-user"></span> <span class="caret"></span> in the upper right corner of the top navigation menu to access the user settings menu and click on the menu item *Virtual machines* to go to the Virtual machines page. This page lists the virtual machines you can access. The *Host name* column lists the name of each available VM. The *Login name* column will have a list of comma separated values of the form @you at . In this guide the hostname will be *_shell_* and the login will be *_you_*. Replace these with your hostname and login name as appropriate.
+To see a list of virtual machines that you have access to, click on the dropdown menu icon <span class="fa fa-lg fa-user"></span> <span class="caret"></span> in the upper right corner of the top navigation menu to access the user settings menu, then click on the menu item *Virtual machines* to go to the Virtual machines page.
+This page lists the virtual machines you can access. The *Host name* column lists the name of each available VM. The *Login name* column lists your login name on that VM. The *Command line* column provides a sample @ssh@ command line.
+At the bottom of the page there may be additional instructions for connecting your specific Arvados instance. If so, follow your site-specific instructions. If there are no site-specific instructions, you can probably connect directly with @ssh at .
+
+The following are generic instructions. In the examples the login will be *_you_* and the hostname will be *_shell.ClusterID.example.com_* and . Replace these with your login name and hostname as appropriate.
diff --git a/doc/_includes/_tutorial_cluster_name.liquid b/doc/_includes/_tutorial_cluster_name.liquid
index 3833abbf2..1de5fa2fa 100644
--- a/doc/_includes/_tutorial_cluster_name.liquid
+++ b/doc/_includes/_tutorial_cluster_name.liquid
@@ -5,5 +5,5 @@ SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
{% include 'notebox_begin' %}
-This tutorial assumes you are using the playground Arvados instance, @qr1hi at . If you are using a different instance, replace @qr1hi@ with your instance. See "Accessing Arvados Workbench":{{site.baseurl}}/user/getting_started/workbench.html for more details.
+This tutorial assumes you are using the playground Arvados instance, @pirca at . If you are using a different instance, replace @pirca@ with your instance. See "Accessing Arvados Workbench":{{site.baseurl}}/user/getting_started/workbench.html for more details.
{% include 'notebox_end' %}
diff --git a/doc/_includes/_tutorial_expectations.liquid b/doc/_includes/_tutorial_expectations.liquid
index 6c4fbeb1f..09b18f0d4 100644
--- a/doc/_includes/_tutorial_expectations.liquid
+++ b/doc/_includes/_tutorial_expectations.liquid
@@ -5,5 +5,5 @@ SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
{% include 'notebox_begin' %}
-This tutorial assumes that you are logged into an Arvados VM instance (instructions for "Webshell":{{site.baseurl}}/user/getting_started/vm-login-with-webshell.html or "Unix":{{site.baseurl}}/user/getting_started/ssh-access-unix.html#login or "Windows":{{site.baseurl}}/user/getting_started/ssh-access-windows.html#login) or you have installed the Arvados "FUSE Driver":{{site.baseurl}}/sdk/python/arvados-fuse.html and "Python SDK":{{site.baseurl}}/sdk/python/sdk-python.html on your workstation and have a "working environment.":{{site.baseurl}}/user/getting_started/check-environment.html
+This tutorial assumes that you have access to the "Arvados command line tools":/user/getting_started/setup-cli.html and have set the "API token":{{site.baseurl}}/user/reference/api-tokens.html and confirmed a "working environment.":{{site.baseurl}}/user/getting_started/check-environment.html .
{% include 'notebox_end' %}
diff --git a/doc/_includes/_tutorial_hello_cwl.liquid b/doc/_includes/_tutorial_hello_cwl.liquid
new file mode 100644
index 000000000..ae1ec80ab
--- /dev/null
+++ b/doc/_includes/_tutorial_hello_cwl.liquid
@@ -0,0 +1,11 @@
+#!/usr/bin/env cwl-runner
+{% comment %}
+Copyright (C) The Arvados Authors. All rights reserved.
+
+SPDX-License-Identifier: CC-BY-SA-3.0
+{% endcomment %}
+cwlVersion: v1.0
+class: CommandLineTool
+inputs: []
+outputs: []
+arguments: ["echo", "hello world!"]
diff --git a/doc/architecture/federation.html.textile.liquid b/doc/architecture/federation.html.textile.liquid
index e2b80de70..2ec2c93fb 100644
--- a/doc/architecture/federation.html.textile.liquid
+++ b/doc/architecture/federation.html.textile.liquid
@@ -26,7 +26,7 @@ Clusters are identified by a five-digit alphanumeric id (numbers and lowercase l
Cluster identifiers are mapped API server hosts one of two ways:
-* Through DNS resolution, under the @arvadosapi.com@ domain. For example, the API server for the cluster @qr1hi@ can be found at @qr1hi.arvadosapi.com at . To register a cluster id for free under @arvadosapi.com@, contact "info at curii.com":mailto:info at curii.com
+* Through DNS resolution, under the @arvadosapi.com@ domain. For example, the API server for the cluster @pirca@ can be found at @pirca.arvadosapi.com at . To register a cluster id for free under @arvadosapi.com@, contact "info at curii.com":mailto:info at curii.com
* Through explicit configuration:
The @RemoteClusters@ section of @/etc/arvados/config.yml@ (for arvados-controller)
diff --git a/doc/examples/pipeline_templates/gatk-exome-fq-snp.json b/doc/examples/pipeline_templates/gatk-exome-fq-snp.json
deleted file mode 100644
index 481dda38c..000000000
--- a/doc/examples/pipeline_templates/gatk-exome-fq-snp.json
+++ /dev/null
@@ -1,175 +0,0 @@
-{
- "name":"GATK / exome PE fastq to snp",
- "components":{
- "extract-reference":{
- "repository":"arvados",
- "script_version":"e820bd1c6890f93ea1a84ffd5730bbf0e3d8e153",
- "script":"file-select",
- "script_parameters":{
- "names":[
- "human_g1k_v37.fasta.gz",
- "human_g1k_v37.fasta.fai.gz",
- "human_g1k_v37.dict.gz"
- ],
- "input":"d237a90bae3870b3b033aea1e99de4a9+10820+K at qr1hi"
- },
- "output_name":false
- },
- "bwa-index":{
- "repository":"arvados",
- "script_version":"e820bd1c6890f93ea1a84ffd5730bbf0e3d8e153",
- "script":"bwa-index",
- "script_parameters":{
- "input":{
- "output_of":"extract-reference"
- },
- "bwa_tbz":{
- "value":"8b6e2c4916133e1d859c9e812861ce13+70",
- "required":true
- }
- },
- "output_name":false
- },
- "bwa-aln":{
- "repository":"arvados",
- "script_version":"e820bd1c6890f93ea1a84ffd5730bbf0e3d8e153",
- "script":"bwa-aln",
- "script_parameters":{
- "input":{
- "dataclass":"Collection",
- "required":"true"
- },
- "reference_index":{
- "output_of":"bwa-index"
- },
- "samtools_tgz":{
- "value":"c777e23cf13e5d5906abfdc08d84bfdb+74",
- "required":true
- },
- "bwa_tbz":{
- "value":"8b6e2c4916133e1d859c9e812861ce13+70",
- "required":true
- }
- },
- "runtime_constraints":{
- "max_tasks_per_node":1
- },
- "output_name":false
- },
- "picard-gatk2-prep":{
- "repository":"arvados",
- "script_version":"e820bd1c6890f93ea1a84ffd5730bbf0e3d8e153",
- "script":"picard-gatk2-prep",
- "script_parameters":{
- "input":{
- "output_of":"bwa-aln"
- },
- "reference":{
- "output_of":"extract-reference"
- },
- "picard_zip":{
- "value":"687f74675c6a0e925dec619cc2bec25f+77",
- "required":true
- }
- },
- "runtime_constraints":{
- "max_tasks_per_node":1
- },
- "output_name":false
- },
- "GATK2-realign":{
- "repository":"arvados",
- "script_version":"e820bd1c6890f93ea1a84ffd5730bbf0e3d8e153",
- "script":"GATK2-realign",
- "script_parameters":{
- "input":{
- "output_of":"picard-gatk2-prep"
- },
- "gatk_bundle":{
- "value":"d237a90bae3870b3b033aea1e99de4a9+10820+K at qr1hi",
- "required":true
- },
- "picard_zip":{
- "value":"687f74675c6a0e925dec619cc2bec25f+77",
- "required":true
- },
- "gatk_tbz":{
- "value":"7e0a277d6d2353678a11f56bab3b13f2+87",
- "required":true
- },
- "regions":{
- "value":"13b53dbe1ec032dfc495fd974aa5dd4a+87/S02972011_Covered_sort_merged.bed"
- },
- "region_padding":{
- "value":10
- }
- },
- "runtime_constraints":{
- "max_tasks_per_node":2
- },
- "output_name":false
- },
- "GATK2-bqsr":{
- "repository":"arvados",
- "script_version":"e820bd1c6890f93ea1a84ffd5730bbf0e3d8e153",
- "script":"GATK2-bqsr",
- "script_parameters":{
- "input":{
- "output_of":"GATK2-realign"
- },
- "gatk_bundle":{
- "value":"d237a90bae3870b3b033aea1e99de4a9+10820+K at qr1hi",
- "required":true
- },
- "picard_zip":{
- "value":"687f74675c6a0e925dec619cc2bec25f+77",
- "required":true
- },
- "gatk_tbz":{
- "value":"7e0a277d6d2353678a11f56bab3b13f2+87",
- "required":true
- }
- },
- "output_name":false
- },
- "GATK2-merge-call":{
- "repository":"arvados",
- "script_version":"e820bd1c6890f93ea1a84ffd5730bbf0e3d8e153",
- "script":"GATK2-merge-call",
- "script_parameters":{
- "input":{
- "output_of":"GATK2-bqsr"
- },
- "gatk_bundle":{
- "value":"d237a90bae3870b3b033aea1e99de4a9+10820+K at qr1hi",
- "required":true
- },
- "picard_zip":{
- "value":"687f74675c6a0e925dec619cc2bec25f+77",
- "required":true
- },
- "gatk_tbz":{
- "value":"7e0a277d6d2353678a11f56bab3b13f2+87",
- "required":true
- },
- "regions":{
- "value":"13b53dbe1ec032dfc495fd974aa5dd4a+87/S02972011_Covered_sort_merged.bed"
- },
- "region_padding":{
- "value":10
- },
- "GATK2_UnifiedGenotyper_args":{
- "default":[
- "-stand_call_conf",
- "30.0",
- "-stand_emit_conf",
- "30.0",
- "-dcov",
- "200"
- ]
- }
- },
- "output_name":"Variant calls from UnifiedGenotyper"
- }
- }
-}
diff --git a/doc/examples/pipeline_templates/rtg-fq-snp.json b/doc/examples/pipeline_templates/rtg-fq-snp.json
deleted file mode 100644
index c951c4c43..000000000
--- a/doc/examples/pipeline_templates/rtg-fq-snp.json
+++ /dev/null
@@ -1,76 +0,0 @@
-{
- "name":"Real Time Genomics / PE fastq to snp",
- "components":{
- "extract_reference":{
- "script":"file-select",
- "script_parameters":{
- "names":[
- "human_g1k_v37.fasta.gz"
- ],
- "input":"d237a90bae3870b3b033aea1e99de4a9+10820+K at qr1hi"
- },
- "script_version":"4c1f8cd1431ece2ef11c130d48bb2edfd2f00ec2"
- },
- "reformat_reference":{
- "script_version":"4c1f8cd1431ece2ef11c130d48bb2edfd2f00ec2",
- "script":"rtg-fasta2sdf",
- "script_parameters":{
- "input":{
- "output_of":"extract_reference"
- },
- "rtg_binary_zip":"5d33618193f763b7dc3a3fdfa11d452e+95+K at qr1hi",
- "rtg_license":{
- "optional":false
- }
- }
- },
- "reformat_reads":{
- "script_version":"4c1f8cd1431ece2ef11c130d48bb2edfd2f00ec2",
- "script":"rtg-fastq2sdf",
- "script_parameters":{
- "input":{
- "optional":false
- },
- "rtg_binary_zip":"5d33618193f763b7dc3a3fdfa11d452e+95+K at qr1hi",
- "rtg_license":{
- "optional":false
- }
- }
- },
- "map_reads":{
- "script_version":"4c1f8cd1431ece2ef11c130d48bb2edfd2f00ec2",
- "script":"rtg-map",
- "script_parameters":{
- "input":{
- "output_of":"reformat_reads"
- },
- "reference":{
- "output_of":"reformat_reference"
- },
- "rtg_binary_zip":"5d33618193f763b7dc3a3fdfa11d452e+95+K at qr1hi",
- "rtg_license":{
- "optional":false
- }
- },
- "runtime_constraints":{
- "max_tasks_per_node":1
- }
- },
- "report_snp":{
- "script_version":"4c1f8cd1431ece2ef11c130d48bb2edfd2f00ec2",
- "script":"rtg-snp",
- "script_parameters":{
- "input":{
- "output_of":"map_reads"
- },
- "reference":{
- "output_of":"reformat_reference"
- },
- "rtg_binary_zip":"5d33618193f763b7dc3a3fdfa11d452e+95+K at qr1hi",
- "rtg_license":{
- "optional":false
- }
- }
- }
- }
-}
diff --git a/doc/examples/ruby/list-active-nodes.rb b/doc/examples/ruby/list-active-nodes.rb
deleted file mode 100755
index a3eb20540..000000000
--- a/doc/examples/ruby/list-active-nodes.rb
+++ /dev/null
@@ -1,16 +0,0 @@
-#!/usr/bin/env ruby
-# Copyright (C) The Arvados Authors. All rights reserved.
-#
-# SPDX-License-Identifier: CC-BY-SA-3.0
-
-abort 'Error: Ruby >= 1.9.3 required.' if RUBY_VERSION < '1.9.3'
-
-require 'arvados'
-
-arv = Arvados.new(api_version: 'v1')
-arv.node.list[:items].each do |node|
- if node[:crunch_worker_state] != 'down'
- ping_age = (Time.now - Time.parse(node[:last_ping_at])).to_i rescue -1
- puts "#{node[:uuid]} #{node[:crunch_worker_state]} #{ping_age}"
- end
-end
diff --git a/doc/sdk/java-v2/example.html.textile.liquid b/doc/sdk/java-v2/example.html.textile.liquid
index e73f968c8..8d2fc2f4a 100644
--- a/doc/sdk/java-v2/example.html.textile.liquid
+++ b/doc/sdk/java-v2/example.html.textile.liquid
@@ -28,7 +28,7 @@ public class CollectionExample {
public static void main(String[] argv) {
ConfigProvider conf = ExternalConfigProvider.builder().
apiProtocol("https").
- apiHost("qr1hi.arvadosapi.com").
+ apiHost("zzzzz.arvadosapi.com").
apiPort(443).
apiToken("...").
build();
diff --git a/doc/sdk/python/cookbook.html.textile.liquid b/doc/sdk/python/cookbook.html.textile.liquid
index bd7f64b33..ff8b8052e 100644
--- a/doc/sdk/python/cookbook.html.textile.liquid
+++ b/doc/sdk/python/cookbook.html.textile.liquid
@@ -47,7 +47,7 @@ h2. Get input of a CWL workflow
{% codeblock as python %}
import arvados
api = arvados.api()
-container_request_uuid="qr1hi-xvhdp-zzzzzzzzzzzzzzz"
+container_request_uuid="zzzzz-xvhdp-zzzzzzzzzzzzzzz"
container_request = api.container_requests().get(uuid=container_request_uuid).execute()
print(container_request["mounts"]["/var/lib/cwl/cwl.input.json"])
{% endcodeblock %}
@@ -58,7 +58,7 @@ h2. Get output of a CWL workflow
import arvados
import arvados.collection
api = arvados.api()
-container_request_uuid="qr1hi-xvhdp-zzzzzzzzzzzzzzz"
+container_request_uuid="zzzzz-xvhdp-zzzzzzzzzzzzzzz"
container_request = api.container_requests().get(uuid=container_request_uuid).execute()
collection = arvados.collection.CollectionReader(container_request["output_uuid"])
print(collection.open("cwl.output.json").read())
@@ -89,7 +89,7 @@ def get_cr_state(cr_uuid):
elif c['runtime_status'].get('warning', None):
return 'Warning'
return c['state']
-container_request_uuid = 'qr1hi-xvhdp-zzzzzzzzzzzzzzz'
+container_request_uuid = 'zzzzz-xvhdp-zzzzzzzzzzzzzzz'
print(get_cr_state(container_request_uuid))
{% endcodeblock %}
@@ -98,7 +98,7 @@ h2. List input of child requests
{% codeblock as python %}
import arvados
api = arvados.api()
-parent_request_uuid = "qr1hi-xvhdp-zzzzzzzzzzzzzzz"
+parent_request_uuid = "zzzzz-xvhdp-zzzzzzzzzzzzzzz"
namefilter = "bwa%" # the "like" filter uses SQL pattern match syntax
container_request = api.container_requests().get(uuid=parent_request_uuid).execute()
parent_container_uuid = container_request["container_uuid"]
@@ -117,7 +117,7 @@ h2. List output of child requests
{% codeblock as python %}
import arvados
api = arvados.api()
-parent_request_uuid = "qr1hi-xvhdp-zzzzzzzzzzzzzzz"
+parent_request_uuid = "zzzzz-xvhdp-zzzzzzzzzzzzzzz"
namefilter = "bwa%" # the "like" filter uses SQL pattern match syntax
container_request = api.container_requests().get(uuid=parent_request_uuid).execute()
parent_container_uuid = container_request["container_uuid"]
@@ -136,7 +136,7 @@ h2. List failed child requests
{% codeblock as python %}
import arvados
api = arvados.api()
-parent_request_uuid = "qr1hi-xvhdp-zzzzzzzzzzzzzzz"
+parent_request_uuid = "zzzzz-xvhdp-zzzzzzzzzzzzzzz"
container_request = api.container_requests().get(uuid=parent_request_uuid).execute()
parent_container_uuid = container_request["container_uuid"]
child_requests = api.container_requests().list(filters=[
@@ -155,7 +155,7 @@ h2. Get log of a child request
import arvados
import arvados.collection
api = arvados.api()
-container_request_uuid = "qr1hi-xvhdp-zzzzzzzzzzzzzzz"
+container_request_uuid = "zzzzz-xvhdp-zzzzzzzzzzzzzzz"
container_request = api.container_requests().get(uuid=container_request_uuid).execute()
collection = arvados.collection.CollectionReader(container_request["log_uuid"])
for c in collection:
@@ -168,7 +168,7 @@ h2(#sharing_link). Create a collection sharing link
import arvados
api = arvados.api()
download="https://your.download.server"
-collection_uuid="qr1hi-4zz18-zzzzzzzzzzzzzzz"
+collection_uuid="zzzzz-4zz18-zzzzzzzzzzzzzzz"
token = api.api_client_authorizations().create(body={"api_client_authorization":{"scopes": [
"GET /arvados/v1/collections/%s" % collection_uuid,
"GET /arvados/v1/collections/%s/" % collection_uuid,
@@ -184,8 +184,8 @@ Note, if two collections have files of the same name, the contents will be conca
import arvados
import arvados.collection
api = arvados.api()
-project_uuid = "qr1hi-tpzed-zzzzzzzzzzzzzzz"
-collection_uuids = ["qr1hi-4zz18-aaaaaaaaaaaaaaa", "qr1hi-4zz18-bbbbbbbbbbbbbbb"]
+project_uuid = "zzzzz-tpzed-zzzzzzzzzzzzzzz"
+collection_uuids = ["zzzzz-4zz18-aaaaaaaaaaaaaaa", "zzzzz-4zz18-bbbbbbbbbbbbbbb"]
combined_manifest = ""
for u in collection_uuids:
c = api.collections().get(uuid=u).execute()
@@ -200,7 +200,7 @@ h2. Upload a file into a new collection
import arvados
import arvados.collection
-project_uuid = "qr1hi-j7d0g-zzzzzzzzzzzzzzz"
+project_uuid = "zzzzz-j7d0g-zzzzzzzzzzzzzzz"
collection_name = "My collection"
filename = "file1.txt"
@@ -222,7 +222,7 @@ h2. Download a file from a collection
import arvados
import arvados.collection
-collection_uuid = "qr1hi-4zz18-zzzzzzzzzzzzzzz"
+collection_uuid = "zzzzz-4zz18-zzzzzzzzzzzzzzz"
filename = "file1.txt"
api = arvados.api()
diff --git a/doc/sdk/ruby/example.html.textile.liquid b/doc/sdk/ruby/example.html.textile.liquid
index b8c0dcbb8..f2ea1c09d 100644
--- a/doc/sdk/ruby/example.html.textile.liquid
+++ b/doc/sdk/ruby/example.html.textile.liquid
@@ -55,7 +55,7 @@ first_repo = repos[:items][0]
puts "UUID of first repo returned is #{first_repo[:uuid]}"</code>
{% endcodeblock %}
-UUID of first repo returned is qr1hi-s0uqq-b1bnybpx3u5temz
+UUID of first repo returned is zzzzz-s0uqq-b1bnybpx3u5temz
h2. update
diff --git a/doc/user/cwl/cwl-runner.html.textile.liquid b/doc/user/cwl/cwl-runner.html.textile.liquid
index 2be803b52..8af1c574f 100644
--- a/doc/user/cwl/cwl-runner.html.textile.liquid
+++ b/doc/user/cwl/cwl-runner.html.textile.liquid
@@ -13,12 +13,6 @@ SPDX-License-Identifier: CC-BY-SA-3.0
{% include 'tutorial_expectations' %}
-{% include 'notebox_begin' %}
-
-By default, the @arvados-cwl-runner@ is installed on Arvados shell nodes. If you want to submit jobs from somewhere else, such as your workstation, you may install "arvados-cwl-runner.":#setup
-
-{% include 'notebox_end' %}
-
This tutorial will demonstrate how to submit a workflow at the command line using @arvados-cwl-runner at .
h2. Running arvados-cwl-runner
diff --git a/doc/user/getting_started/ssh-access-unix.html.textile.liquid b/doc/user/getting_started/ssh-access-unix.html.textile.liquid
index 284d0a1f0..c2d5da29b 100644
--- a/doc/user/getting_started/ssh-access-unix.html.textile.liquid
+++ b/doc/user/getting_started/ssh-access-unix.html.textile.liquid
@@ -49,7 +49,7 @@ ssh-rsa AAAAB3NzaC1ycEDoNotUseExampleKeyDoNotUseExampleKeyDoNotUseExampleKeyDoNo
Now you can set up @ssh-agent@ (next) or proceed with "adding your key to the Arvados Workbench.":#workbench
-h3. Set up ssh-agent (recommended)
+h3. Set up ssh-agent (optional)
If you find you are entering your passphrase frequently, you can use @ssh-agent@ to manage your credentials. Use @ssh-add -l@ to test if you already have ssh-agent running:
@@ -80,11 +80,21 @@ When everything is set up, @ssh-add -l@ should yield output that looks something
{% include 'ssh_addkey' %}
-h3. Connecting to the virtual machine
+h3. Connecting directly
-Use the following command to connect to the _shell_ VM instance as _you_. Replace *<code>you at shell</code>* at the end of the following command with your *login* and *hostname* from Workbench:
+If the VM is available on the public Internet (or you are on the same private network as the VM) you can connect directly with @ssh at . You can probably copy-and-paste the text from *Command line* column directly into a terminal.
-notextile. <pre><code>$ <span class="userinput">ssh -o "ProxyCommand ssh -p2222 turnout at switchyard.{{ site.arvados_api_host }} -x -a <b>shell</b>" -x <b>you at shell</b></span></code></pre>
+Use the following example command to connect as _you_ to the _shell.ClusterID.example.com_ VM instance. Replace *<code>you at shell.ClusterID.example.com</code>* at the end of the following command with your *login* and *hostname* from Workbench.
+
+notextile. <pre><code>$ <span class="userinput">ssh <b>you at shell.ClusterID.example.com</b></span></code></pre>
+
+h3. Connecting through switchyard
+
+Some Arvados installations use "switchyard" to isolate shell VMs from the public Internet.
+
+Use the following example command to connect to the _shell_ VM instance as _you_. Replace *<code>you at shell</code>* at the end of the following command with your *login* and *hostname* from Workbench:
+
+notextile. <pre><code>$ <span class="userinput">ssh -o "ProxyCommand ssh -p2222 turnout at switchyard.ClusterID.example.com -x -a <b>shell</b>" -x <b>you at shell</b></span></code></pre>
This command does several things at once. You usually cannot log in directly to virtual machines over the public Internet. Instead, you log into a "switchyard" server and then tell the switchyard which virtual machine you want to connect to.
@@ -99,7 +109,7 @@ This command does several things at once. You usually cannot log in directly to
You should now be able to log into the Arvados VM and "check your environment.":check-environment.html
-h3. Configuration (recommended)
+h4. Configuration (recommended)
The command line above is cumbersome, but you can configure SSH to remember many of these settings. Add this text to the file @.ssh/config@ in your home directory (create a new file if @.ssh/config@ doesn't exist):
diff --git a/doc/user/getting_started/ssh-access-windows.html.textile.liquid b/doc/user/getting_started/ssh-access-windows.html.textile.liquid
index 0406e7c03..ee4ab4e92 100644
--- a/doc/user/getting_started/ssh-access-windows.html.textile.liquid
+++ b/doc/user/getting_started/ssh-access-windows.html.textile.liquid
@@ -9,13 +9,13 @@ Copyright (C) The Arvados Authors. All rights reserved.
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-This document is for accessing an Arvados VM using SSH keys in Windows environments. If you would like to use to access VM through your browser, please visit the "Accessing an Arvados VM with Webshell":vm-login-with-webshell.html page. If you are using a Unix environment (Linux, OS X, Cygwin), please visit the "Accessing an Arvados VM with SSH - Unix Environments":ssh-access-unix.html page.
+This document is for accessing an Arvados VM using SSH keys in Windows environments using PuTTY. If you would like to use to access VM through your browser, please visit the "Accessing an Arvados VM with Webshell":vm-login-with-webshell.html page. If you are using a Unix environment (Linux, OS X, Cygwin, or Windows Subsystem for Linux), please visit the "Accessing an Arvados VM with SSH - Unix Environments":ssh-access-unix.html page.
{% include 'ssh_intro' %}
h1(#gettingkey). Getting your SSH key
-(Note: if you are using the SSH client that comes with "Cygwin":http://cygwin.com, please use instructions found in the "Accessing an Arvados VM with SSH - Unix Environments":ssh-access-unix.html page.)
+(Note: If you are using the SSH client that comes with "Cygwin":http://cygwin.com or Windows Subsystem for Linux (WSL) please use instructions found in the "Accessing an Arvados VM with SSH - Unix Environments":ssh-access-unix.html page.)
We will be using PuTTY to connect to Arvados. "PuTTY":http://www.chiark.greenend.org.uk/~sgtatham/putty/ is a free (MIT-licensed) Win32 Telnet and SSH client. PuTTY includes all the tools a Windows user needs to create private keys and make SSH connections to your virtual machines in the Arvados Cloud.
@@ -57,6 +57,16 @@ Pageant is a PuTTY utility that manages your private keys so is not necessary to
h3. Initial configuration
+h4. Connecting directly
+
+# Open PuTTY from the Start Menu.
+# On the Session screen set the Host Name (or IP address) to “shell.ClusterID.example.com”, which is the hostname listed in the _Virtual Machines_ page.
+# On the Session screen set the Port to “22”.
+# On the Connection %(rarr)→% Data screen set the Auto-login username to the username listed in the *Login name* column on the Arvados Workbench Virtual machines_ page.
+# Return to the Session screen. In the Saved Sessions box, enter a name for this configuration and click Save.
+
+h4. Connecting through switchyard
+
# Open PuTTY from the Start Menu.
# On the Session screen set the Host Name (or IP address) to “shell”, which is the hostname listed in the _Virtual Machines_ page.
# On the Session screen set the Port to “22”.
diff --git a/doc/user/getting_started/vm-login-with-webshell.html.textile.liquid b/doc/user/getting_started/vm-login-with-webshell.html.textile.liquid
index 551002e55..2aa494ae9 100644
--- a/doc/user/getting_started/vm-login-with-webshell.html.textile.liquid
+++ b/doc/user/getting_started/vm-login-with-webshell.html.textile.liquid
@@ -15,6 +15,10 @@ h2(#webshell). Access VM using webshell
Webshell gives you access to an arvados virtual machine from your browser with no additional setup.
+{% include 'notebox_begin' %}
+Some Arvados clusters may not have webshell set up. If you do not see a "Log in" button or "web shell" column, you will have to follow the "Unix":ssh-access-unix.html or "Windows":ssh-access-windows.html @ssh@ instructions.
+{% include 'notebox_end' %}
+
In the Arvados Workbench, click on the dropdown menu icon <span class="fa fa-lg fa-user"></span> <span class="caret"></span> in the upper right corner of the top navigation menu to access the user settings menu, and click on the menu item *Virtual machines* to see the list of virtual machines you can access. If you do not have access to any virtual machines, please click on <span class="btn btn-sm btn-primary">Send request for shell access</span> or send an email to "support at curoverse.com":mailto:support at curoverse.com.
Each row in the Virtual Machines panel lists the hostname of the VM, along with a <code>Log in as *you*</code> button under the column "Web shell". Clicking on this button will open up a webshell terminal for you in a new browser tab and log you in.
diff --git a/doc/user/getting_started/workbench.html.textile.liquid b/doc/user/getting_started/workbench.html.textile.liquid
index fc704227e..52f1931c9 100644
--- a/doc/user/getting_started/workbench.html.textile.liquid
+++ b/doc/user/getting_started/workbench.html.textile.liquid
@@ -9,13 +9,19 @@ Copyright (C) The Arvados Authors. All rights reserved.
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-If you are using the default Arvados instance for this guide, you can Access Arvados Workbench using this link:
+{% include 'notebox_begin' %}
+This guide covers the classic Arvados Workbench web application, sometimes referred to as "Workbench 1". There is also a new Workbench web application under development called "Workbench 2". Sites which have both Workbench applications installed will have a dropdown menu option "Switch to Workbench 2" to switch between versions.
+
+This guide will be updated to cover "Workbench 2" in the future.
+{% include 'notebox_end' %}
+
+If you are using the "playground" Arvados instance for this guide, you can Access Arvados Workbench using this link:
<a href="{{site.arvados_workbench_host}}/" target="_blank">{{site.arvados_workbench_host}}/</a>
(If you are using a different Arvados instance than the default for this guide, replace *{{ site.arvados_workbench_host }}* with your private instance in all of the examples in this guide.)
-You may be asked to log in using a Google account. Arvados uses only your name and email address from Google services for identification, and will never access any personal information. If you are accessing Arvados for the first time, the Workbench may indicate your account status is *New / inactive*. If this is the case, contact the administrator of the Arvados instance to request activation of your account.
+You will be asked to log in. Arvados uses only your name and email address for identification, and will never access any personal information. If you are accessing Arvados for the first time, the Workbench may indicate your account status is *New / inactive*. If this is the case, contact the administrator of the Arvados instance to request activation of your account.
Once your account is active, logging in to the Workbench will present you with the Dashboard. This gives a summary of your projects and recent activity in the Arvados instance. "You are now ready to run your first pipeline.":{{ site.baseurl }}/user/tutorials/tutorial-workflow-workbench.html
diff --git a/doc/user/topics/arv-copy.html.textile.liquid b/doc/user/topics/arv-copy.html.textile.liquid
index 0f0e40be9..805d71b2d 100644
--- a/doc/user/topics/arv-copy.html.textile.liquid
+++ b/doc/user/topics/arv-copy.html.textile.liquid
@@ -9,17 +9,13 @@ Copyright (C) The Arvados Authors. All rights reserved.
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-{% include 'crunch1only_begin' %}
-On those sites, the "copy a pipeline template" feature described below is not available. However, "copy a workflow" feature is not yet implemented.
-{% include 'crunch1only_end' %}
-
This tutorial describes how to copy Arvados objects from one cluster to another by using @arv-copy at .
{% include 'tutorial_expectations' %}
h2. arv-copy
- at arv-copy@ allows users to copy collections and pipeline templates from one cluster to another. By default, @arv-copy@ will recursively go through a template and copy all dependencies associated with the object.
+ at arv-copy@ allows users to copy collections and workflows from one cluster to another. By default, @arv-copy@ will recursively go through the workflow and copy all dependencies associated with the object.
For example, let's copy from the <a href="https://playground.arvados.org/">Arvados playground</a>, also known as *qr1hi*, to *dst_cluster*. The names *qr1hi* and *dst_cluster* are interchangable with any cluster name. You can find the cluster name from the prefix of the uuid of the object you want to copy. For example, in *qr1hi*-4zz18-tci4vn4fa95w0zx, the cluster name is qr1hi.
@@ -60,33 +56,6 @@ For example, this will copy the collection to project dst_cluster-j7d0g-a894213u
</code></pre>
</notextile>
-h3. How to copy a pipeline template
-
-{% include 'arv_copy_expectations' %}
-
-We will use the uuid @qr1hi-p5p6p-9pkaxt6qjnkxhhu@ as an example pipeline template.
-
-<notextile>
-<pre><code>~$ <span class="userinput">arv-copy --src qr1hi --dst dst_cluster --dst-git-repo $USER/tutorial qr1hi-p5p6p-9pkaxt6qjnkxhhu</span>
-To git at git.dst_cluster.arvadosapi.com:$USER/tutorial.git
- * [new branch] git_git_qr1hi_arvadosapi_com_arvados_git_ac21f0d45a76294aaca0c0c0fdf06eb72d03368d -> git_git_qr1hi_arvadosapi_com_arvados_git_ac21f0d45a76294aaca0c0c0fdf06eb72d03368d
-arvados.arv-copy[19694] INFO: Success: created copy with uuid dst_cluster-p5p6p-rym2h5ub9m8ofwj
-</code></pre>
-</notextile>
-
-New branches in the destination git repo will be created for each branch used in the pipeline template. For example, if your source branch was named ac21f0d45a76294aaca0c0c0fdf06eb72d03368d, your new branch will be named @git_git_qr1hi_arvadosapi_com_reponame_git_ac21f0d45a76294aaca0c0c0fdf06eb72d03368d at .
-
-By default, if you copy a pipeline template recursively, you will find that the template as well as all the dependencies are in your home project.
-
-If you would like to copy the object without dependencies, you can use the @--no-recursive@ tag.
-
-For example, we can copy the same object using this tag.
-
-<notextile>
-<pre><code>~$ <span class="userinput">arv-copy --src qr1hi --dst dst_cluster --dst-git-repo $USER/tutorial --no-recursive qr1hi-p5p6p-9pkaxt6qjnkxhhu</span>
-</code></pre>
-</notextile>
-
h3. How to copy a workflow
We will use the uuid @zzzzz-7fd4e-sampleworkflow1@ as an example workflow.
diff --git a/doc/user/tutorials/add-new-repository.html.textile.liquid b/doc/user/tutorials/add-new-repository.html.textile.liquid
index 9d8e768a7..cb69d460a 100644
--- a/doc/user/tutorials/add-new-repository.html.textile.liquid
+++ b/doc/user/tutorials/add-new-repository.html.textile.liquid
@@ -9,7 +9,7 @@ Copyright (C) The Arvados Authors. All rights reserved.
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-Arvados repositories are managed through the Git revision control system. You can use these repositories to store your crunch scripts and run them in the arvados cluster.
+Arvados supports managing git repositories. You can acess these repositories using your Arvados credentials and share them with other Arvados users.
{% include 'tutorial_expectations' %}
diff --git a/doc/user/tutorials/git-arvados-guide.html.textile.liquid b/doc/user/tutorials/git-arvados-guide.html.textile.liquid
index 2e255219d..ad719a66e 100644
--- a/doc/user/tutorials/git-arvados-guide.html.textile.liquid
+++ b/doc/user/tutorials/git-arvados-guide.html.textile.liquid
@@ -9,20 +9,13 @@ Copyright (C) The Arvados Authors. All rights reserved.
SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
-This tutorial describes how to work with a new Arvados git repository. Working with an Arvados git repository is analogous to working with other public git repositories. It will show you how to upload custom scripts to a remote Arvados repository, so you can use it in Arvados pipelines.
+This tutorial describes how to work with an Arvados-managed git repository. Working with an Arvados git repository is very similar to working with other public git repositories.
{% include 'tutorial_expectations' %}
{% include 'tutorial_git_repo_expectations' %}
-{% include 'notebox_begin' %}
-For more information about using Git, try
-<notextile>
-<pre><code>$ <span class="userinput">man gittutorial</span></code></pre>
-</notextile> or *"search Google for Git tutorials":http://google.com/#q=git+tutorial*.
-{% include 'notebox_end' %}
-
-h2. Cloning an Arvados repository
+h2. Cloning a git repository
Before you start using Git, you should do some basic configuration (you only need to do this the first time):
@@ -65,33 +58,22 @@ Create a git branch named *tutorial_branch* in the *tutorial* Arvados git reposi
h2. Adding scripts to an Arvados repository
-Arvados crunch scripts need to be added in a *crunch_scripts* subdirectory in the repository. If this subdirectory does not exist, first create it in the local repository and change to that directory:
-
-<notextile>
-<pre><code>~/tutorial$ <span class="userinput">mkdir crunch_scripts</span>
-~/tutorial$ <span class="userinput">cd crunch_scripts</span></code></pre>
-</notextile>
-
-Next, using @nano@ or your favorite Unix text editor, create a new file called @hash.py@ in the @crunch_scripts@ directory.
-
-notextile. <pre>~/tutorial/crunch_scripts$ <code class="userinput">nano hash.py</code></pre>
-
-Add the following code to compute the MD5 hash of each file in a collection
+A git repository is a good place to store the CWL workflows that you run on Arvados.
-<notextile> {% code 'tutorial_hash_script_py' as python %} </notextile>
+First, create a simple CWL CommandLineTool:
-Make the file executable:
+notextile. <pre>~/tutorials$ <code class="userinput">nano hello.cwl</code></pre>
-notextile. <pre><code>~/tutorial/crunch_scripts$ <span class="userinput">chmod +x hash.py</span></code></pre>
+<notextile> {% code 'tutorial_hello_cwl' as yaml %} </notextile>
Next, add the file to the git repository. This tells @git@ that the file should be included on the next commit.
-notextile. <pre><code>~/tutorial/crunch_scripts$ <span class="userinput">git add hash.py</span></code></pre>
+notextile. <pre><code>~/tutorial$ <span class="userinput">git add hello.cwl</span></code></pre>
Next, commit your changes. All staged changes are recorded into the local git repository:
<notextile>
-<pre><code>~/tutorial/crunch_scripts$ <span class="userinput">git commit -m "my first script"</span>
+<pre><code>~/tutorial$ <span class="userinput">git commit -m "my first script"</span>
</code></pre>
</notextile>
@@ -102,4 +84,4 @@ Finally, upload your changes to the remote repository:
</code></pre>
</notextile>
-Although this tutorial shows how to add a python script to Arvados, the same steps can be used to add any of your custom bash, R, or python scripts to an Arvados repository.
+The same steps can be used to add any of your custom bash, R, or python scripts to an Arvados repository.
commit 59af072670472944e6e377f67abca212da74bd32
Author: Peter Amstutz <peter.amstutz at curii.com>
Date: Fri Jan 3 10:10:47 2020 -0500
15964: Delete long out of date (and unused) 'getting started' section
Arvados-DCO-1.1-Signed-off-by: Peter Amstutz <peter.amstutz at curii.com>
diff --git a/doc/_includes/_tutorial_cluster_name.liquid b/doc/_includes/_tutorial_cluster_name.liquid
index 22fbc463e..3833abbf2 100644
--- a/doc/_includes/_tutorial_cluster_name.liquid
+++ b/doc/_includes/_tutorial_cluster_name.liquid
@@ -5,5 +5,5 @@ SPDX-License-Identifier: CC-BY-SA-3.0
{% endcomment %}
{% include 'notebox_begin' %}
-This tutorial assumes you are using the default Arvados instance, @qr1hi at . If you are using a different instance, replace @qr1hi@ with your instance. See "Accessing Arvados Workbench":{{site.baseurl}}/user/getting_started/workbench.html for more details.
+This tutorial assumes you are using the playground Arvados instance, @qr1hi at . If you are using a different instance, replace @qr1hi@ with your instance. See "Accessing Arvados Workbench":{{site.baseurl}}/user/getting_started/workbench.html for more details.
{% include 'notebox_end' %}
diff --git a/doc/install/copy_pipeline_from_curoverse.html.textile.liquid b/doc/install/copy_pipeline_from_curoverse.html.textile.liquid
deleted file mode 100644
index 2c2b3c466..000000000
--- a/doc/install/copy_pipeline_from_curoverse.html.textile.liquid
+++ /dev/null
@@ -1,68 +0,0 @@
----
-layout: default
-navsection: installguide
-title: Copy pipeline from the Arvados Playground
-...
-{% comment %}
-Copyright (C) The Arvados Authors. All rights reserved.
-
-SPDX-License-Identifier: CC-BY-SA-3.0
-{% endcomment %}
-
-This tutorial describes how to find and copy a publicly shared pipeline from the Arvados Playground. Please note that you can use similar steps to copy any template you can access from the Arvados Playground to your cluster.
-
-h3. Access a public pipeline in the Arvados Playground using Workbench
-
-the Arvados Playground provides access to some public data, which can be used to experience Arvados in action. Let's access a public pipeline and copy it to your cluster, so that you can run it in your environment.
-
-Start by visiting the "*Arvados Playground public projects page*":https://playground.arvados.org/projects/public. This page lists all the publicly accessible projects in this arvados installation. Click on one of these projects to open it. We will use "*lobSTR v.3 (Public)*":https://playground.arvados.org/projects/qr1hi-j7d0g-up6qgpqz5ie2vfq as the example in this tutorial.
-
-Once in the "*lobSTR v.3 (Public)*":https://playground.arvados.org/projects/qr1hi-j7d0g-up6qgpqz5ie2vfq project, click on the *Pipeline templates* tab. In the pipeline templates tab, you will see a template named *lobSTR v.3*. Click on the <span class="fa fa-lg fa-gears"></span> *Show* button to the left of this name. This will take to you to the "*lobSTR v.3*":https://playground.arvados.org/pipeline_templates/qr1hi-p5p6p-9pkaxt6qjnkxhhu template page.
-
-Once in this page, you can take the *uuid* of this template from the address bar, which is *qr1hi-p5p6p-9pkaxt6qjnkxhhu*. Next, we will copy this template to your Arvados instance.
-
-h3. Copying a pipeline template from the Arvados Playground to your cluster
-
-As described above, navigate to the publicly shared pipeline template "*lobSTR v.3*":https://playground.arvados.org/pipeline_templates/qr1hi-p5p6p-9pkaxt6qjnkxhhu on the Arvados Playground. We will now copy this template with uuid *qr1hi-p5p6p-9pkaxt6qjnkxhhu* to your cluster.
-
-{% include 'tutorial_expectations' %}
-
-We will use the Arvados *arv-copy* command to copy this template to your cluster. In order to use arv-copy, first you need to setup the source and destination cluster configuration files. Here, *qr1hi* would be the source cluster and your Arvados instance would be the *dst_cluster*.
-
-During this setup, if you have an account in the Arvados Playground, you can use "your access token":#using-your-token to create the source configuration file. If you do not have an account in the Arvados Playground, you can use the "anonymous access token":#using-anonymous-token for the source cluster configuration.
-
-h4(#using-anonymous-token). *Configuring source and destination setup files using anonymous access token*
-
-Configure the source and destination clusters as described in the "*Using arv-copy*":http://doc.arvados.org/user/topics/arv-copy.html tutorial in user guide, while using *5vqmz9mik2ou2k9objb8pnyce8t97p6vocyaouqo3qalvpmjs5* as the API token for source configuration.
-
-<notextile>
-<pre><code>~$ <span class="userinput">cd ~/.config/arvados</span>
-~$ <span class="userinput">echo "ARVADOS_API_HOST=qr1hi.arvadosapi.com" >> qr1hi.conf</span>
-~$ <span class="userinput">echo "ARVADOS_API_TOKEN=5vqmz9mik2ou2k9objb8pnyce8t97p6vocyaouqo3qalvpmjs5" >> qr1hi.conf</span>
-</code></pre>
-</notextile>
-
-You can now copy the pipeline template from *qr1hi* to *your cluster*. Replace *dst_cluster* with the *ClusterID* of your cluster.
-
-<notextile>
-<pre><code>~$ <span class="userinput"> arv-copy --no-recursive --src qr1hi --dst dst_cluster qr1hi-p5p6p-9pkaxt6qjnkxhhu</span>
-</code></pre>
-</notextile>
-
-*Note:* When you are using anonymous access token to copy the template, you will not be able to do a recursive copy since you will not be able to provide the dst-git-repo parameter. In order to perform a recursive copy of the template, you would need to use the Arvados API token from your account as explained in the "using your token":#using-your-token section below.
-
-h4(#using-your-token). *Configuring source and destination setup files using personal access token*
-
-If you already have an account in the Arvados Playground, you can follow the instructions in the "*Using arv-copy*":http://doc.arvados.org/user/topics/arv-copy.html user guide to get your *Current token* for source and destination clusters, and use them to create the source *qr1hi.conf* and dst_cluster.conf configuration files.
-
-You can now copy the pipeline template from *qr1hi* to *your cluster* with or without recursion. Replace *dst_cluster* with the *ClusterID* of your cluster.
-
-*Non-recursive copy:*
-<notextile>
-<pre><code>~$ <span class="userinput"> arv-copy --no-recursive --src qr1hi --dst dst_cluster qr1hi-p5p6p-9pkaxt6qjnkxhhu</span></code></pre>
-</notextile>
-
-*Recursive copy:*
-<notextile>
-<pre><code>~$ <span class="userinput">arv-copy --src qr1hi --dst dst_cluster --dst-git-repo $USER/tutorial qr1hi-p5p6p-9pkaxt6qjnkxhhu</span></code></pre>
-</notextile>
diff --git a/doc/start/getting_started/firstpipeline.html.textile.liquid b/doc/start/getting_started/firstpipeline.html.textile.liquid
deleted file mode 100644
index 43369a3bb..000000000
--- a/doc/start/getting_started/firstpipeline.html.textile.liquid
+++ /dev/null
@@ -1,94 +0,0 @@
----
-layout: default
-navsection: start
-title: Run your first pipeline in minutes
-...
-{% comment %}
-Copyright (C) The Arvados Authors. All rights reserved.
-
-SPDX-License-Identifier: CC-BY-SA-3.0
-{% endcomment %}
-
-h2. LobSTR v3
-
-In this quickstart guide, we'll run an existing pipeline with pre-existing data. Step-by-step instructions are shown below. You can follow along using your own local install or by using the <a href="https://playground.arvados.org/">Arvados Playground</a> (any Google account can be used to log in).
-
-(For more information about this pipeline, see our <a href="https://dev.arvados.org/projects/arvados/wiki/LobSTR_tutorial">detailed lobSTR guide</a>).
-
-<div id="carousel-firstpipe" class="carousel slide" data-interval="false">
- <!-- Indicators -->
- <ol class="carousel-indicators">
- <li data-target="#carousel-firstpipe" data-slide-to="0" class="active"></li>
- <li data-target="#carousel-firstpipe" data-slide-to="1"></li>
- <li data-target="#carousel-firstpipe" data-slide-to="2"></li>
- <li data-target="#carousel-firstpipe" data-slide-to="3"></li>
- <li data-target="#carousel-firstpipe" data-slide-to="4"></li>
- <li data-target="#carousel-firstpipe" data-slide-to="5"></li>
- <li data-target="#carousel-firstpipe" data-slide-to="6"></li>
- </ol>
-
- <!-- Wrapper for slides -->
- <div class="carousel-inner" role="listbox">
- <div class="item active">
- <img src="{{ site.baseurl }}/images/quickstart/1.png" alt="Step 1. At the dashboard, click 'Run a pipeline...'.">
- <div class="carousel-caption">
- Step 1. At the dashboard, click 'Run a pipeline...'.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/quickstart/2.png" alt="Choose 'lobstr v.3' and hit 'Next'.">
- <div class="carousel-caption">
- Choose 'lobstr v.3' and hit 'Next'.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/quickstart/3.png" alt="Rename the pipeline instance, then click 'Run'. Click 'Choose' to change the default inputs.">
- <div class="carousel-caption">
- Rename the pipeline instance, then click 'Run'. Click 'Choose' to change the default inputs.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/quickstart/4.png" alt="Here we search for and choose new inputs.">
- <div class="carousel-caption">
- Here we search for and choose new inputs.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/quickstart/5.png" alt="After the job completes, you can re-run it with one click.">
- <div class="carousel-caption">
- After the job completes, you can re-run it with one click.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/quickstart/6.png" alt="You can inspect details about the pipeline which are automatically logged.">
- <div class="carousel-caption">
- You can inspect automatically-logged details about the pipeline.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/quickstart/7.png" alt="Click 'Create sharing link' to share the output files with people outside Arvados. [END]">
- <div class="carousel-caption">
- Click 'Create sharing link' to share the output files with people outside Arvados. [END]
- </div>
- </div>
-
- </div>
-
- <!-- Controls -->
- <a class="left carousel-control" href="#carousel-firstpipe" role="button" data-slide="prev">
- <span class="glyphicon glyphicon-chevron-left" aria-hidden="true"></span>
- <span class="sr-only">Previous</span>
- </a>
- <a class="right carousel-control" href="#carousel-firstpipe" role="button" data-slide="next">
- <span class="glyphicon glyphicon-chevron-right" aria-hidden="true"></span>
- <span class="sr-only">Next</span>
- </a>
-</div>
-
-Tip: You may need to make your browser window bigger to see full-size images in the gallery above.
diff --git a/doc/start/getting_started/nextsteps.html.textile.liquid b/doc/start/getting_started/nextsteps.html.textile.liquid
deleted file mode 100644
index dd059ea8d..000000000
--- a/doc/start/getting_started/nextsteps.html.textile.liquid
+++ /dev/null
@@ -1,12 +0,0 @@
----
-layout: default
-navsection: start
-title: Check out the User Guide
-...
-{% comment %}
-Copyright (C) The Arvados Authors. All rights reserved.
-
-SPDX-License-Identifier: CC-BY-SA-3.0
-{% endcomment %}
-
-Now that you've finished the Getting Started guide, check out the "User Guide":{{site.baseurl}}/user/index.html. The User Guide goes into more depth than the Getting Started guide, covers how to develop your own pipelines in addition to using pre-existing pipelines, covers the Arvados command line tools in addition to the Workbench graphical interface to Arvados, and can be referenced in any order.
diff --git a/doc/start/getting_started/publicproject.html.textile.liquid b/doc/start/getting_started/publicproject.html.textile.liquid
deleted file mode 100644
index 0fabad7aa..000000000
--- a/doc/start/getting_started/publicproject.html.textile.liquid
+++ /dev/null
@@ -1,133 +0,0 @@
----
-layout: default
-navsection: start
-title: Visit an Arvados Public Project
-...
-{% comment %}
-Copyright (C) The Arvados Authors. All rights reserved.
-
-SPDX-License-Identifier: CC-BY-SA-3.0
-{% endcomment %}
-
-h2. <a href="https://workbench.qr1hi.arvadosapi.com/projects/qr1hi-j7d0g-662ij1pcw6bj8uj">Mason Lab - Pathomap / Ancestry Mapper (Public)</a>
-
-You can see Arvados in action by accessing the <a href="https://workbench.qr1hi.arvadosapi.com/projects/qr1hi-j7d0g-662ij1pcw6bj8uj">Mason Lab - Pathomap / Ancestry Mapper (Public) project</a>. By visiting this project, you can see what an Arvados project is, access data collections in this project, and click through a pipeline instance's contents.
-
-You will be accessing this project in read-only mode and will not be able to make any modifications such as running a new pipeline instance.
-
-<div id="carousel-publicproject" class="carousel slide" data-interval="false">
- <!-- Indicators -->
- <ol class="carousel-indicators">
- <li data-target="#carousel-publicproject" data-slide-to="0" class="active"></li>
- <li data-target="#carousel-publicproject" data-slide-to="1"></li>
- <li data-target="#carousel-publicproject" data-slide-to="2"></li>
- <li data-target="#carousel-publicproject" data-slide-to="3"></li>
- <li data-target="#carousel-publicproject" data-slide-to="4"></li>
- <li data-target="#carousel-publicproject" data-slide-to="5"></li>
- <li data-target="#carousel-publicproject" data-slide-to="6"></li>
- <li data-target="#carousel-publicproject" data-slide-to="7"></li>
- <li data-target="#carousel-publicproject" data-slide-to="8"></li>
- <li data-target="#carousel-publicproject" data-slide-to="9"></li>
- <li data-target="#carousel-publicproject" data-slide-to="10"></li>
- <li data-target="#carousel-publicproject" data-slide-to="11"></li>
- </ol>
-
- <!-- Wrapper for slides -->
- <div class="carousel-inner" role="listbox">
- <div class="item active">
- <img src="{{ site.baseurl }}/images/publicproject/description.png" alt="Step 1. The project's first tab, *Description*, describes what this project is all about.">
- <div class="carousel-caption">
- Step 1. The project's first tab, *Description*, describes what this project is all about.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/publicproject/collections.png" alt="The *Data collections* tab contains the various pipeline inputs, logs, and outputs.">
- <div class="carousel-caption">
- The *Data collections* tab contains the various pipeline inputs, logs, and outputs.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/publicproject/instances.png" alt="You can see the jobs and pipelines in this project by accessing the *Jobs and pipelines* tab.">
- <div class="carousel-caption">
- You can see the jobs and pipelines in this project by accessing the *Jobs and pipelines* tab.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/publicproject/collection-show.png" alt="In the *Data collections* tab, click on the *Show* icon to the left of a collection to see the collection contents.">
- <div class="carousel-caption">
- In the *Data collections* tab, click on the *Show* icon to the left of a collection to see the collection contents.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/publicproject/collection-files.png" alt="The collection page lists the details about it. The *Files* tab can be used to view and download individual files in it.">
- <div class="carousel-caption">
- The collection page lists the details about it. The *Files* tab can be used to view and download individual files in it.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/publicproject/collection-graph.png" alt="The collection *Provenance graph* tab gives a visual representation of this collection's provenance.">
- <div class="carousel-caption">
- The collection *Provenance graph* tab gives a visual representation of this collection's provenance.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/publicproject/instance-show.png" alt="In the project *Jobs and pipelines* tab, click on the *Show* icon to the left of a pipeline to access the pipeline contents.">
- <div class="carousel-caption">
- In the project *Jobs and pipelines* tab, click on the *Show* icon to the left of a pipeline to access the pipeline contents.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/publicproject/instance-components.png" alt="The pipeline *Components* tab details the various jobs in it and how long it took to run it.">
- <div class="carousel-caption">
- The pipeline *Components* tab details the various jobs in it and how long it took to run it.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/publicproject/instance-job.png" alt="Click on the down arrow in one of the job rows to see the job details. You can also click on the job's output.">
- <div class="carousel-caption">
- Click on the down arrow <i class="fa fa-lg fa-fw fa-caret-down"></i> in one of the job rows to see the job details. You can also click on the job's output.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/publicproject/instance-log.png" alt="The *Log* tab can be used to see the log for the pipeline instance.">
- <div class="carousel-caption">
- The *Log* tab can be used to see the log for the pipeline instance.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/publicproject/instance-graph.png" alt="The *Graph* tab provides a visual representation of the pipeline run.">
- <div class="carousel-caption">
- The *Graph* tab provides a visual representation of the pipeline run.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/publicproject/instance-advanced.png" alt="The *Advanced* tab can be used to access metadata about the pipeline. [END]">
- <div class="carousel-caption">
- The *Advanced* tab can be used to access metadata about the pipeline. [END]
- </div>
- </div>
- </div>
-
- <!-- Controls -->
- <a class="left carousel-control" href="#carousel-publicproject" role="button" data-slide="prev">
- <span class="glyphicon glyphicon-chevron-left" aria-hidden="true"></span>
- <span class="sr-only">Previous</span>
- </a>
- <a class="right carousel-control" href="#carousel-publicproject" role="button" data-slide="next">
- <span class="glyphicon glyphicon-chevron-right" aria-hidden="true"></span>
- <span class="sr-only">Next</span>
- </a>
-</div>
-
-Tip: You may need to make your browser window bigger to see full-size images in the gallery above.
diff --git a/doc/start/getting_started/sharedata.html.textile.liquid b/doc/start/getting_started/sharedata.html.textile.liquid
deleted file mode 100644
index 02e0b7032..000000000
--- a/doc/start/getting_started/sharedata.html.textile.liquid
+++ /dev/null
@@ -1,102 +0,0 @@
----
-layout: default
-navsection: start
-title: Sharing Data
-...
-{% comment %}
-Copyright (C) The Arvados Authors. All rights reserved.
-
-SPDX-License-Identifier: CC-BY-SA-3.0
-{% endcomment %}
-
-You can easily share data entirely through Workbench, the web interface to Arvados.
-
-h2. Upload and share your existing data
-
-Step-by-step instructions are shown below.
-
-<div id="carousel-sharedata" class="carousel slide" data-interval="false">
- <!-- Indicators -->
- <ol class="carousel-indicators">
- <li data-target="#carousel-sharedata" data-slide-to="0" class="active"></li>
- <li data-target="#carousel-sharedata" data-slide-to="1"></li>
- <li data-target="#carousel-sharedata" data-slide-to="2"></li>
- <li data-target="#carousel-sharedata" data-slide-to="3"></li>
- <li data-target="#carousel-sharedata" data-slide-to="4"></li>
- <li data-target="#carousel-sharedata" data-slide-to="5"></li>
- <li data-target="#carousel-sharedata" data-slide-to="6"></li>
- <li data-target="#carousel-sharedata" data-slide-to="7"></li>
- </ol>
-
- <!-- Wrapper for slides -->
- <div class="carousel-inner" role="listbox">
- <div class="item active">
- <img src="{{ site.baseurl }}/images/uses/gotohome.png" alt="Step 1. From the dashboard, go to your Home project.">
- <div class="carousel-caption">
- Step 1. From the dashboard, go to your Home project.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/uses/uploaddata.png" alt="Click 'Add data' → 'Upload files'.">
- <div class="carousel-caption">
- Click 'Add data' → 'Upload files'.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/uses/choosefiles.png" alt="A new collection is created automatically. Choose files to upload and hit Start.">
- <div class="carousel-caption">
- A new collection is created automatically. Choose files to upload and hit Start.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/uses/uploading.png" alt="Files will upload and stay uploaded even if the browser is closed.">
- <div class="carousel-caption">
- Files will upload and stay uploaded even if the browser is closed.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/uses/rename.png" alt="Rename the collection appropriately.">
- <div class="carousel-caption">
- Rename the collection appropriately.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/uses/sharing.png" alt="Click 'Create sharing link'. You can click 'unshare' at any later point.">
- <div class="carousel-caption">
- Click 'Create sharing link'. You can click 'Unshare' at any later point.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/uses/shared.png" alt="Now just share this link with anyone you want.">
- <div class="carousel-caption">
- Now just share this link with anyone you want.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/uses/sharedsubdirs.png" alt="Here's a more complex collection. [END]">
- <div class="carousel-caption">
- Here's a more complex collection. [END]
- </div>
- </div>
-
- </div>
-
- <!-- Controls -->
- <a class="left carousel-control" href="#carousel-sharedata" role="button" data-slide="prev">
- <span class="glyphicon glyphicon-chevron-left" aria-hidden="true"></span>
- <span class="sr-only">Previous</span>
- </a>
- <a class="right carousel-control" href="#carousel-sharedata" role="button" data-slide="next">
- <span class="glyphicon glyphicon-chevron-right" aria-hidden="true"></span>
- <span class="sr-only">Next</span>
- </a>
-</div>
-
-Tip: You may need to make your browser window bigger to see full-size images in the gallery above.
diff --git a/doc/start/index.html.textile.liquid b/doc/start/index.html.textile.liquid
deleted file mode 100644
index cddfb8e44..000000000
--- a/doc/start/index.html.textile.liquid
+++ /dev/null
@@ -1,133 +0,0 @@
----
-layout: default
-navsection: start
-title: Welcome to Arvados!
-...
-{% comment %}
-Copyright (C) The Arvados Authors. All rights reserved.
-
-SPDX-License-Identifier: CC-BY-SA-3.0
-{% endcomment %}
-
-This guide provides an introduction to using Arvados to solve big data bioinformatics problems.
-
-h2. What is Arvados?
-
-Arvados is a free and open source bioinformatics platform for genomic and biomedical data.
-
-We address the needs of IT directors, lab principals, and bioinformaticians.
-
-h2. Why use Arvados?
-
-Arvados enables you to quickly begin using cloud computing resources in your bioinformatics work. It allows you to track your methods and datasets, share them securely, and easily re-run analyses.
-
-h3. Take a look (Screenshots gallery)
-
-<div id="carousel-keyfeatures" class="carousel slide" data-interval="false">
- <!-- Indicators -->
- <ol class="carousel-indicators">
- <li data-target="#carousel-keyfeatures" data-slide-to="0" class="active"></li>
- <li data-target="#carousel-keyfeatures" data-slide-to="1"></li>
- <li data-target="#carousel-keyfeatures" data-slide-to="2"></li>
- <li data-target="#carousel-keyfeatures" data-slide-to="3"></li>
- <li data-target="#carousel-keyfeatures" data-slide-to="4"></li>
- <li data-target="#carousel-keyfeatures" data-slide-to="5"></li>
- <li data-target="#carousel-keyfeatures" data-slide-to="6"></li>
- <li data-target="#carousel-keyfeatures" data-slide-to="7"></li>
- <li data-target="#carousel-keyfeatures" data-slide-to="8"></li>
- </ol>
-
- <!-- Wrapper for slides -->
- <div class="carousel-inner" role="listbox">
- <div class="item active">
- <img src="{{ site.baseurl }}/images/keyfeatures/dashboard2.png" alt="[START] After logging in, you will see Workbench's dashboard.">
- <div class="carousel-caption">
- [START] After logging in, you will see Workbench's dashboard.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/keyfeatures/running2.png" alt="Pipelines describe a set of computational tasks (jobs).">
- <div class="carousel-caption">
- Pipelines describe a set of computational tasks (jobs).
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/keyfeatures/log.png" alt="The output of all jobs is logged and stored automatically.">
- <div class="carousel-caption">
- The output of all jobs is logged and stored automatically.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/keyfeatures/graph.png" alt="Pipelines can also be viewed in auto-generated graph form.">
- <div class="carousel-caption">
- Pipelines can also be viewed in auto-generated graph form.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/keyfeatures/rerun.png" alt="Pipelines can easily be re-run exactly as before, or...">
- <div class="carousel-caption">
- Pipelines can easily be re-run exactly as before, or...
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/keyfeatures/chooseinputs.png" alt="...you can change parameters or pick new datasets.">
- <div class="carousel-caption">
- ...you can change parameters or pick new datasets.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/keyfeatures/webupload.png" alt="With web upload, data can be uploaded right in Workbench.">
- <div class="carousel-caption">
- With web upload, data can be uploaded right in Workbench.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/keyfeatures/collectionpage.png" alt="Collections allow sharing datasets and job outputs easily. 'Create sharing link' with one click.">
- <div class="carousel-caption">
- Collections allow sharing datasets and job outputs easily. 'Create sharing link' with one click.
- </div>
- </div>
-
- <div class="item">
- <img src="{{ site.baseurl }}/images/keyfeatures/provenance.png" alt="Data provenance is tracked automatically. [END]">
- <div class="carousel-caption">
- Data provenance is tracked automatically. [END]
- </div>
- </div>
-
-
- </div>
-
- <!-- Controls -->
- <a class="left carousel-control" href="#carousel-keyfeatures" role="button" data-slide="prev">
- <span class="glyphicon glyphicon-chevron-left" aria-hidden="true"></span>
- <span class="sr-only">Previous</span>
- </a>
- <a class="right carousel-control" href="#carousel-keyfeatures" role="button" data-slide="next">
- <span class="glyphicon glyphicon-chevron-right" aria-hidden="true"></span>
- <span class="sr-only">Next</span>
- </a>
-</div>
-
-Note: Workbench is the web interface to Arvados.
-Tip: You may need to make your browser window bigger to see full-size images in the gallery above.
-
-h3. Key Features
-
-<ul>
-<li><strong>Track your methods</strong><br/>
-We log every compute job: software versions, machine images, input and output data hashes. Rely on a computer, not your memory and your note-taking skills.<br/><br/></li>
-<li><strong>Share your methods</strong><br/>
-Show other people what you did. Let them use your workflow on their own data. Publish a permalink to your methods and data, so others can reproduce and build on them easily.<br/><br/></li>
-<li><strong>Track data origin</strong><br/>
-Did you really only use fully consented public data in this analysis?<br/><br/></li>
-<li><strong>Get results sooner</strong><br/>
-Run your compute jobs faster by using multi-nodes and multi-cores, even if your programs are single-threaded.<br/><br/></li>
-</ul>
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list