[ARVADOS] updated: 2211249b1a8b6f0a9ff88664a2227ebe5800e08b
git at public.curoverse.com
git at public.curoverse.com
Wed Feb 12 14:36:10 EST 2014
Summary of changes:
.../app/assets/stylesheets/application.css.scss | 14 +
.../app/views/layouts/application.html.erb | 12 -
doc/_config.yml | 4 +
doc/_includes/_navbar_left.liquid | 26 +-
doc/_includes/_navbar_top.liquid | 44 +-
doc/_layouts/default.html.liquid | 27 +-
.../badges.css.scss => doc/css/badges.css | 0
doc/css/bootstrap-responsive.min.css | 20 -
doc/css/bootstrap-theme.css | 347 +
doc/css/bootstrap-theme.css.map | 1 +
doc/css/bootstrap-theme.min.css | 7 +
doc/css/bootstrap.css | 9502 ++++++++++----------
doc/css/bootstrap.css.map | 1 +
doc/css/bootstrap.min.css | 7 +
doc/css/nav-list.css | 24 +
doc/fonts/glyphicons-halflings-regular.eot | Bin 0 -> 20290 bytes
doc/fonts/glyphicons-halflings-regular.svg | 229 +
doc/fonts/glyphicons-halflings-regular.ttf | Bin 0 -> 41236 bytes
doc/fonts/glyphicons-halflings-regular.woff | Bin 0 -> 23292 bytes
{apps/workbench/app/assets => doc}/images/dax.png | Bin 66943 -> 66943 bytes
doc/index.html.liquid | 38 +-
doc/js/bootstrap.js | 1951 ++++
doc/js/bootstrap.min.js | 13 +-
.../check-environment.html.textile.liquid | 26 +-
.../getting_started/ssh-access.html.textile.liquid | 16 +-
doc/user/index.html.textile.liquid | 11 +-
doc/user/reference/api-tokens.html.textile.liquid | 63 +-
doc/user/reference/sdk-cli.html.textile.liquid | 64 +-
.../running-external-program.html.textile.liquid | 24 +-
.../tutorial-firstscript.html.textile.liquid | 40 +-
...rial-gatk-variantfiltration.html.textile.liquid | 20 +-
.../tutorial-job-debug.html.textile.liquid | 22 +-
.../tutorials/tutorial-job1.html.textile.liquid | 34 +-
.../tutorials/tutorial-keep.html.textile.liquid | 55 +-
.../tutorial-parallel.html.textile.liquid | 26 +-
.../tutorial-trait-search.html.textile.liquid | 2 +-
36 files changed, 7367 insertions(+), 5303 deletions(-)
copy apps/workbench/app/assets/stylesheets/badges.css.scss => doc/css/badges.css (100%)
delete mode 100644 doc/css/bootstrap-responsive.min.css
create mode 100644 doc/css/bootstrap-theme.css
create mode 100644 doc/css/bootstrap-theme.css.map
create mode 100644 doc/css/bootstrap-theme.min.css
create mode 100644 doc/css/bootstrap.css.map
create mode 100644 doc/css/bootstrap.min.css
create mode 100644 doc/css/nav-list.css
create mode 100644 doc/fonts/glyphicons-halflings-regular.eot
create mode 100644 doc/fonts/glyphicons-halflings-regular.svg
create mode 100644 doc/fonts/glyphicons-halflings-regular.ttf
create mode 100644 doc/fonts/glyphicons-halflings-regular.woff
copy {apps/workbench/app/assets => doc}/images/dax.png (100%)
create mode 100644 doc/js/bootstrap.js
via 2211249b1a8b6f0a9ff88664a2227ebe5800e08b (commit)
via 926625e0650d6e055ee098798694af3006ae34a9 (commit)
via 95ced779d1f6a78539e21a646a1f8a78482443ac (commit)
via ba64505e002d2d3a00d14ea567e80889534f14bf (commit)
via fc38c67d9b3b8ae6a1d891e8005c43d6af6edd06 (commit)
via 8c285503b45e89a891236221aba26488d2d74832 (commit)
via 720d44bbcf5299dd243a3b29e8b18c207290660b (commit)
via 22bf98c71f0a3e2c58bc591214d3e9ef13c4327f (commit)
from 77e12f54c6b5e97abb1f045163658f7deb31d5c4 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
commit 2211249b1a8b6f0a9ff88664a2227ebe5800e08b
Merge: 926625e 77e12f5
Author: Peter Amstutz <peter.amstutz at curoverse.com>
Date: Wed Feb 12 14:36:58 2014 -0500
Merge branch 'master' of git.clinicalfuture.com:arvados
commit 926625e0650d6e055ee098798694af3006ae34a9
Merge: e28dffc 95ced77
Author: Peter Amstutz <peter.amstutz at curoverse.com>
Date: Wed Feb 12 14:36:04 2014 -0500
Merge branch '2061-update-docs'
diff --cc apps/workbench/app/assets/stylesheets/application.css.scss
index 5ebb572,580be64..55263c7
--- a/apps/workbench/app/assets/stylesheets/application.css.scss
+++ b/apps/workbench/app/assets/stylesheets/application.css.scss
@@@ -126,6 -126,17 +126,20 @@@ ul.arvados-nav li ul li
display:inline-block;
}
+li.notification {
+ padding: 10px;
++
+ .arvados-nav-container {
+ top: 70px;
+ height: calc(100% - 70px);
+ overflow: auto;
+ z-index: 2;
+ }
+
+ .arvados-nav-active {
+ background: rgb(66, 139, 202);
+ }
+
+ .arvados-nav-active a {
+ color: white;
}
commit 95ced779d1f6a78539e21a646a1f8a78482443ac
Author: Peter Amstutz <peter.amstutz at curoverse.com>
Date: Wed Feb 12 14:33:07 2014 -0500
Finished updating tutorials for latest sprint
diff --git a/doc/user/index.html.textile.liquid b/doc/user/index.html.textile.liquid
index c8aee3c..982b1c3 100644
--- a/doc/user/index.html.textile.liquid
+++ b/doc/user/index.html.textile.liquid
@@ -36,10 +36,12 @@ This manual uses the following typographic conventions:
<notextile>
<ul>
-<li>Code blocks which are set aside from the text indicate user input to the system. Commands that should be entered into a Unix shell are indicated with a '$' at the beginning of the line, and the entire user input <span class="userinput">will be highlighted</span>, possibly followed by example command output in black:
-<pre><code>$ <span class="userinput">ls foo</span>
+<li>Code blocks which are set aside from the text indicate user input to the system. Commands that should be entered into a Unix shell are indicated by the directory where you should enter the command ('~' indicates your home directory) followed by '$', followed by the highlighted <span class="userinput">command to enter</span> (do not enter the '$'), and possibly followed by example command output in black. For example, the following block indicates that you should type "ls foo" while in your home directory and the expected output will be "foo".
+
+<pre><code>~$ <span class="userinput">ls foo</span>
foo
</code></pre></li>
+
<li>Code blocks inline with text emphasize specific <code>programs</code>, <code>files</code>, or <code>options</code> that are being discussed.</li>
<li>Bold text emphasizes <b>specific items</b> to look when discussing Arvados Workbench pages.</li>
<li>A sequence of steps separated by right arrows (<span class="rarr">→</span>) indicate a path the user should follow through the Arvados Workbench to access some piece of information under discussion. The steps indicate a menu, hyperlink, column name, field name, or other label on the page that guide the user where to look or click.
diff --git a/doc/user/reference/api-tokens.html.textile.liquid b/doc/user/reference/api-tokens.html.textile.liquid
index 1694c4e..4c49d4a 100644
--- a/doc/user/reference/api-tokens.html.textile.liquid
+++ b/doc/user/reference/api-tokens.html.textile.liquid
@@ -46,6 +46,13 @@ $ <span class="userinput">export ARVADOS_API_TOKEN=2jv9346o3966345u7ueuim7a1zaao
* @ARVADOS_API_HOST@ tells @arv@ which host to connect to
* @ARVADOS_API_TOKEN@ is the secret key used by the Arvados API server to authenticate access.
+If you are connecting to a development instance with a unverified/self-signed SSL certificate, set this variable to skip SSL validation:
+
+<notextile>
+<pre><code>$ <span class="userinput">export ARVADOS_API_HOST_INSECURE=1
+</code></pre>
+</notextile>
+
h2. settings.conf
Arvados tools will also look for the authentication information in @~/.config/arvados/settings.conf at . If you have already put the variables into the environment with instructions above, you can use these commands to create an Arvados configuration file:
diff --git a/doc/user/reference/sdk-cli.html.textile.liquid b/doc/user/reference/sdk-cli.html.textile.liquid
index 55e9c3c..5effb1d 100644
--- a/doc/user/reference/sdk-cli.html.textile.liquid
+++ b/doc/user/reference/sdk-cli.html.textile.liquid
@@ -3,79 +3,45 @@ layout: default
navsection: userguide
navmenu: Reference
title: "Command line interface"
-
...
h1. Reference: Command Line Interface
-If you are logged in to an Arvados VM, the command line SDK should be installed. Try:
-
- at arv --help@
-
-<!-- _Help is not helpful. See bug #1667_ -->
-
-h3. First...
+*First, you should be "logged into an Arvados VM instance":{{site.basedoc}}/user/getting_started/ssh-access.html#login, and have a "working environment.":{{site.basedoc}}/user/getting_started/check-environment.html*
-Set the ARVADOS_API_HOST environment variable.
-
- at export ARVADOS_API_HOST={{ site.arvados_api_host }}@
+h3. Usage
-Log in to Workbench and get an API token for your account. Set the ARVADOS_API_TOKEN environment variable.
+ at arv [global_options] resource_type resource_method [method_parameters]@
- at export
-ARVADOS_API_TOKEN=c0vdbi8wp7f703lbthyadlvmaivgldxssy3l32isslmax93k9@
+h4. Global options
-If you are using a development instance with an unverifiable SSL certificate, set the ARVADOS_API_HOST_INSECURE environment variable.
+- @--format=json@ := Output response as JSON
+- @--format=yaml@ := Output response as YAML
+- @--format=uuid@ := Output only the UUIDs of object(s) in the API response, one per line.
- at export ARVADOS_API_HOST_INSECURE=1@
-<!-- _This should link back to "api-tokens":api-tokens.html instead of
-re-explaining it__ -->
+h3. Resource types and methods
-h3. Usage
+Get list of resource types
+ at arv --resources@
- at arv [global_options] resource_type resource_method [method_parameters]@
+Get list of resource methods for the "user" resource type
+ at arv user --help@
-<!-- _This is what arv --help really ought to print out_ -->
h3. Basic examples
-Get UUID of the current user
- at arv user current@
-
-Get entire record (json) for current user
+Get record for current user
@arv -h user current@
-Get entire record for identified user
+Get entire record for some specific user
@arv -h user get --uuid 6dnxa-tpzed-iimd25zhzh84gbk@
Update user record
@arv user update --uuid 6dnxa-tpzed-iimd25zhzh84gbk --first-name "Bob"@
-Get list of groups (showing just UUIDs)
+Get list of groups
@arv group list@
-Get list of groups (showing entire records)
- at arv -h group list@
-
Delete a group
@arv group delete --uuid 6dnxa-j7d0g-iw7i6n43d37jtog@
-
-h3. Global options
-
-<!-- _Move these up to before "basic examples", and give examples of what
-these options do and how they might be useful._ -->
-
-- @--json@, @-j@ := Output entire response as compact JSON.
-
-- @--pretty@, @--human@, @-h@ := Output entire response as JSON with whitespace for better human-readability.
-
-- @--uuid@ := Output only the UUIDs of object(s) in the API response, one per line.
-
-h3. Resource types and methods
-
-Get list of resource types
- at arv --help@
-
-Get list of resource methods for the "user" resource type
- at arv user --help@
diff --git a/doc/user/tutorials/running-external-program.html.textile.liquid b/doc/user/tutorials/running-external-program.html.textile.liquid
index e286013..875e3cf 100644
--- a/doc/user/tutorials/running-external-program.html.textile.liquid
+++ b/doc/user/tutorials/running-external-program.html.textile.liquid
@@ -17,31 +17,35 @@ In this tutorial, you will use the external program @md5sum@ to compute hashes i
Start by entering the @crunch_scripts@ directory of your git repository:
<notextile>
-<pre><code>$ <span class="userinput">cd you/crunch_scripts</span>
+<pre><code>~$ <span class="userinput">cd <b>you</b>/crunch_scripts</span>
</code></pre>
</notextile>
-
-Next, using your favorite text editor, create a new file called @run-md5sum.py@ in the @crunch_scripts@ directory. Add the following code to use the @md5sum@ program to compute the hash of each file in a collection:
+
+Next, using @nano@ or your favorite Unix text editor, create a new file called @run-md5sum.py@ in the @crunch_scripts@ directory.
+
+notextile. <pre>~/<b>you</b>/crunch_scripts$ <code class="userinput">nano run-md5sum.py</code></pre>
+
+Add the following code to use the @md5sum@ program to compute the hash of each file in a collection:
<pre><code class="userinput">{% include 'run_md5sum_py' %}</code></pre>
Make the file executable:
-notextile. <pre><code>$ <span class="userinput">chmod +x run-md5sum.py</span></code></pre>
+notextile. <pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">chmod +x run-md5sum.py</span></code></pre>
Next, add the file to @git@ staging, commit and push:
<notextile>
-<pre><code>$ <span class="userinput">git add run-md5sum.py</span>
-$ <span class="userinput">git commit -m"run external md5sum program"</span>
-$ <span class="userinput">git push origin master</span>
+<pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">git add run-md5sum.py</span>
+~/<b>you</b>/crunch_scripts$ <span class="userinput">git commit -m"run external md5sum program"</span>
+~/<b>you</b>/crunch_scripts$ <span class="userinput">git push origin master</span>
</code></pre>
</notextile>
You should now be able to run your new script using Crunch, with "script" referring to our new "run-md5sum.py" script.
<notextile>
-<pre><code>$ <span class="userinput">cat >the_job <<EOF
+<pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">cat >~/the_job <<EOF
{
"script": "run-md5sum.py",
"script_version": "you:master",
@@ -51,13 +55,13 @@ You should now be able to run your new script using Crunch, with "script" referr
}
}
EOF</span>
-$ <span class="userinput">arv -h job create --job "$(cat the_job)"</span>
+~/<b>you</b>/crunch_scripts$ <span class="userinput">arv -h job create --job "$(cat the_job)"</span>
{
...
"uuid":"qr1hi-xxxxx-xxxxxxxxxxxxxxx"
...
}
-$ <span class="userinput">arv -h job get --uuid qr1hi-xxxxx-xxxxxxxxxxxxxxx</span>
+~/<b>you</b>/crunch_scripts$ <span class="userinput">arv -h job get --uuid qr1hi-xxxxx-xxxxxxxxxxxxxxx</span>
{
...
"output":"4d164b1658c261b9afc6b479130016a3+54",
diff --git a/doc/user/tutorials/tutorial-firstscript.html.textile.liquid b/doc/user/tutorials/tutorial-firstscript.html.textile.liquid
index 2422bb2..41f8a84 100644
--- a/doc/user/tutorials/tutorial-firstscript.html.textile.liquid
+++ b/doc/user/tutorials/tutorial-firstscript.html.textile.liquid
@@ -12,7 +12,7 @@ In this tutorial, we will write the "hash" script demonstrated in the first tuto
*This tutorial assumes that you are "logged into an Arvados VM instance":{{site.basedoc}}/user/getting_started/ssh-access.html#login, and have a "working environment.":{{site.basedoc}}/user/getting_started/check-environment.html*
-This tutorial uses _you_ to denote your username. Replace _you_ with your user name in all the following examples.
+This tutorial uses *@you@* to denote your username. Replace *@you@* with your user name in all the following examples.
h2. Setting up Git
@@ -21,20 +21,20 @@ As discussed in the previous tutorial, all Crunch scripts are managed through th
First, you should do some basic configuration for git (you only need to do this the first time):
<notextile>
-<pre><code>$ <span class="userinput">git config --global user.name "Your Name"</span>
-$ <span class="userinput">git config --global user.email you at example.com</span></code></pre>
+<pre><code>~$ <span class="userinput">git config --global user.name "Your Name"</span>
+~$ <span class="userinput">git config --global user.email <b>you</b>@example.com</span></code></pre>
</notextile>
-On the Arvados Workbench, navigate to _Access %(rarr)→% Repositories._ You should see two repositories, one named "arvados" (under the *name* column) and a second with your user name. Next to *name* is the column *push_url*. Copy the *push_url* cell associated with your repository. This should look like <code>git at git.{{ site.arvados_api_host }}:you.git</code>.
+On the Arvados Workbench, navigate to _Compute %(rarr)→% Code repositories._ You should see two repositories, one named "arvados" (under the *name* column) and a second with your user name. Next to *name* is the column *push_url*. Copy the *push_url* cell associated with your repository. This should look like <notextile><code>git at git.{{ site.arvados_api_host }}:<b>you</b>.git</code></notextile>.
Next, on the Arvados virtual machine, clone your git repository:
<notextile>
-<pre><code>$ <span class="userinput">git clone git at git.{{ site.arvados_api_host }}:you.git</span>
-Cloning into 'you'...</code></pre>
+<pre><code>~$ <span class="userinput">git clone git at git.{{ site.arvados_api_host }}:<b>you</b>.git</span>
+Cloning into '<b>you</b>'...</code></pre>
</notextile>
-This will create an git checkout in the directory called @you at .
+This will create an git checkout in the directory called *@you@*.
{% include 'notebox_begin' %}
For more information about using @git@, try
@@ -46,17 +46,17 @@ or <b>"click here to search Google for git tutorials":http://google.com/#q=git+t
h2. Creating a Crunch script
-Start by entering the @you@ directory created by @git clone at . Next create a subdirectory called @crunch_scripts@ and change to that directory:
+Start by entering the *@you@* directory created by @git clone at . Next create a subdirectory called @crunch_scripts@ and change to that directory:
<notextile>
-<pre><code>$ <span class="userinput">cd you</span>
-$ <span class="userinput">mkdir crunch_scripts</span>
-$ <span class="userinput">cd crunch_scripts</span></code></pre>
+<pre><code>~$ <span class="userinput">cd <b>you</b></span>
+~/<b>you</b>$ <span class="userinput">mkdir crunch_scripts</span>
+~/<b>you</b>$ <span class="userinput">cd crunch_scripts</span></code></pre>
</notextile>
Next, using @nano@ or your favorite Unix text editor, create a new file called @hash.py@ in the @crunch_scripts@ directory.
-notextile. <pre>$ <code class="userinput">nano hash.py</code></pre>
+notextile. <pre>~/<b>you</b>/crunch_scripts$ <code class="userinput">nano hash.py</code></pre>
Add the following code to compute the md5 hash of each file in a collection:
@@ -64,16 +64,16 @@ Add the following code to compute the md5 hash of each file in a collection:
Make the file executable:
-notextile. <pre><code>$ <span class="userinput">chmod +x hash.py</span></code></pre>
+notextile. <pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">chmod +x hash.py</span></code></pre>
Next, add the file to @git@ staging. This tells @git@ that the file should be included on the next commit.
-notextile. <pre><code>$ <span class="userinput">git add hash.py</span></code></pre>
+notextile. <pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">git add hash.py</span></code></pre>
Next, commit your changes to git. All staged changes are recorded into the local @git@ repository:
<notextile>
-<pre><code>$ <span class="userinput">git commit -m"my first script"</span>
+<pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">git commit -m"my first script"</span>
[master (root-commit) 27fd88b] my first script
1 file changed, 33 insertions(+)
create mode 100755 crunch_scripts/hash.py</code></pre>
@@ -82,7 +82,7 @@ Next, commit your changes to git. All staged changes are recorded into the loca
Finally, upload your changes to the Arvados server:
<notextile>
-<pre><code>$ <span class="userinput">git push origin master</span>
+<pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">git push origin master</span>
Counting objects: 4, done.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (4/4), 682 bytes, done.
@@ -94,7 +94,7 @@ To git at git.qr1hi.arvadosapi.com:you.git
You should now be able to run your script using Crunch, similar to how we did it in the "first tutorial.":tutorial-job1.html The field @"script_version"@ should be @you:master@ to tell Crunch to run the script at the head of the "master" git branch, which you just uploaded.
<notextile>
-<pre><code>$ <span class="userinput">cat >the_job <<EOF
+<pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">cat >~/the_job <<EOF
{
"script": "hash.py",
"script_version": "you:master",
@@ -104,19 +104,19 @@ You should now be able to run your script using Crunch, similar to how we did it
}
}
EOF</span>
-$ <span class="userinput">arv -h job create --job "$(cat the_job)"</span>
+~/<b>you</b>/crunch_scripts$ <span class="userinput">arv job create --job "$(cat ~/the_job)"</span>
{
...
"uuid":"qr1hi-xxxxx-xxxxxxxxxxxxxxx"
...
}
-$ <span class="userinput">arv -h job get --uuid qr1hi-xxxxx-xxxxxxxxxxxxxxx</span>
+~/<b>you</b>/crunch_scripts$ <span class="userinput">arv job get --uuid qr1hi-xxxxx-xxxxxxxxxxxxxxx</span>
{
...
"output":"880b55fb4470b148a447ff38cacdd952+54",
...
}
-$ <span class="userinput">arv keep get 880b55fb4470b148a447ff38cacdd952+54/md5sum.txt</span>
+~/<b>you</b>/crunch_scripts$ <span class="userinput">arv keep get 880b55fb4470b148a447ff38cacdd952+54/md5sum.txt</span>
44b8ae3fde7a8a88d2f7ebd237625b4f var-GS000016015-ASM.tsv.bz2
</code></pre>
</notextile>
diff --git a/doc/user/tutorials/tutorial-gatk-variantfiltration.html.textile.liquid b/doc/user/tutorials/tutorial-gatk-variantfiltration.html.textile.liquid
index 04894fd..9372dad 100644
--- a/doc/user/tutorials/tutorial-gatk-variantfiltration.html.textile.liquid
+++ b/doc/user/tutorials/tutorial-gatk-variantfiltration.html.textile.liquid
@@ -17,7 +17,7 @@ h2. Installing GATK
Download the GATK binary tarball[1] -- e.g., @GenomeAnalysisTK-2.6-4.tar.bz2@ -- and "copy it to your Arvados VM":tutorial-keep.html.
<notextile>
-<pre><code>$ <span class="userinput">arv keep put GenomeAnalysisTK-2.6-4.tar.bz2</span>
+<pre><code>~$ <span class="userinput">arv keep put GenomeAnalysisTK-2.6-4.tar.bz2</span>
c905c8d8443a9c44274d98b7c6cfaa32+94
</code></pre>
</notextile>
@@ -25,7 +25,7 @@ c905c8d8443a9c44274d98b7c6cfaa32+94
Next, you need the GATK Resource Bundle[2]. This may already be available in Arvados. If not, you will need to download the files listed below and put them into Keep.
<notextile>
-<pre><code>$ <span class="userinput">arv keep ls -s d237a90bae3870b3b033aea1e99de4a9+10820</span>
+<pre><code>~$ <span class="userinput">arv keep ls -s d237a90bae3870b3b033aea1e99de4a9+10820</span>
50342 1000G_omni2.5.b37.vcf.gz
1 1000G_omni2.5.b37.vcf.gz.md5
464 1000G_omni2.5.b37.vcf.idx.gz
@@ -94,11 +94,11 @@ h2. Submit a GATK job
The Arvados distribution includes an example crunch script ("crunch_scripts/GATK2-VariantFiltration":https://arvados.org/projects/arvados/repository/revisions/master/entry/crunch_scripts/GATK2-VariantFiltration) that runs the GATK VariantFiltration tool with some default settings.
<notextile>
-<pre><code>$ <span class="userinput">src_version=76588bfc57f33ea1b36b82ca7187f465b73b4ca4</span>
-$ <span class="userinput">vcf_input=5ee633fe2569d2a42dd81b07490d5d13+82</span>
-$ <span class="userinput">gatk_binary=c905c8d8443a9c44274d98b7c6cfaa32+94</span>
-$ <span class="userinput">gatk_bundle=d237a90bae3870b3b033aea1e99de4a9+10820</span>
-$ <span class="userinput">cat >the_job <<EOF
+<pre><code>~$ <span class="userinput">src_version=76588bfc57f33ea1b36b82ca7187f465b73b4ca4</span>
+~$ <span class="userinput">vcf_input=5ee633fe2569d2a42dd81b07490d5d13+82</span>
+~$ <span class="userinput">gatk_binary=c905c8d8443a9c44274d98b7c6cfaa32+94</span>
+~$ <span class="userinput">gatk_bundle=d237a90bae3870b3b033aea1e99de4a9+10820</span>
+~$ <span class="userinput">cat >the_job <<EOF
{
"script":"GATK2-VariantFiltration",
"script_version":"$src_version",
@@ -120,7 +120,7 @@ EOF</span>
Now start a job:
<notextile>
-<pre><code>$ <span class="userinput">arv -h job create --job "$(cat the_job)"</span>
+<pre><code>~$ <span class="userinput">arv -h job create --job "$(cat the_job)"</span>
{
"href":"https://qr1hi.arvadosapi.com/arvados/v1/jobs/qr1hi-8i9sb-n9k7qyp7bs5b9d4",
"kind":"arvados#job",
@@ -160,7 +160,7 @@ Now start a job:
],
"log_stream_href":"https://qr1hi.arvadosapi.com/arvados/v1/jobs/qr1hi-8i9sb-n9k7qyp7bs5b9d4/log_tail_follow"
}
-$ <span class="userinput">arv job log_tail_follow --uuid qr1hi-8i9sb-n9k7qyp7bs5b9d4</span>
+~$ <span class="userinput">arv job log_tail_follow --uuid qr1hi-8i9sb-n9k7qyp7bs5b9d4</span>
Tue Dec 17 19:02:16 2013 salloc: Granted job allocation 1251
Tue Dec 17 19:02:17 2013 qr1hi-8i9sb-n9k7qyp7bs5b9d4 4867 check slurm allocation
Tue Dec 17 19:02:17 2013 qr1hi-8i9sb-n9k7qyp7bs5b9d4 4867 node compute13 - 8 slots
@@ -204,7 +204,7 @@ Tue Dec 17 19:04:12 2013 qr1hi-8i9sb-n9k7qyp7bs5b9d4 4867 log manifest is 1e77a
Once the job completes, the output can be found in hu34D5B9-exome-filtered.vcf:
-<notextile><pre><code>$ <span class="userinput">arv keep ls bedd6ff56b3ae9f90d873b1fcb72f9a3+91</span>
+<notextile><pre><code>~$ <span class="userinput">arv keep ls bedd6ff56b3ae9f90d873b1fcb72f9a3+91</span>
hu34D5B9-exome-filtered.vcf
</code></pre>
</notextile>
diff --git a/doc/user/tutorials/tutorial-job-debug.html.textile.liquid b/doc/user/tutorials/tutorial-job-debug.html.textile.liquid
index 063eb12..38ef3be 100644
--- a/doc/user/tutorials/tutorial-job-debug.html.textile.liquid
+++ b/doc/user/tutorials/tutorial-job-debug.html.textile.liquid
@@ -19,13 +19,13 @@ h2. Create a new script
Change to your git directory and create a new script in "crunch_scripts/".
<notextile>
-<pre><code>$ <span class="userinput">cd you/crunch_scripts</span>
-$ <span class="userinput">cat >hello-world.py <<EOF
+<pre><code>~$ <span class="userinput">cd <b>you</b>/crunch_scripts</span>
+~/<b>you</b>/crunch_scripts$ <span class="userinput">cat >hello-world.py <<EOF
#!/usr/bin/env python
print "hello world"
EOF</span>
-$ <span class="userinput">chmod +x hello-world.py</span>
+~/<b>you</b>/crunch_scripts$ <span class="userinput">chmod +x hello-world.py</span>
</code></pre>
</notextile>
@@ -34,14 +34,14 @@ h2. Using arv-crunch-job to run the job in your VM
Instead of a git commit hash, we provide the path to the directory in the "script_version" parameter. The script specified in "script" will actually be searched for in the "crunch_scripts/" subdirectory of the directory specified "script_version". Although we are running the script locally, the script still requires access to the Arvados API server and Keep storage service. The job will be recorded in the Arvados job history, and visible in Workbench.
<notextile>
-<pre><code>$ <span class="userinput">cat >the_job <<EOF
+<pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">cat >~/the_job <<EOF
{
"script":"hello-world.py",
"script_version":"/home/you/you",
"script_parameters":{}
}
EOF</span>
-$ <span class="userinput">arv-crunch-job --job "$(cat the_job)"</span>
+~/<b>you</b>/crunch_scripts</span>$ <span class="userinput">arv-crunch-job --job "$(cat ~/the_job)"</span>
2013-12-12_21:36:42 qr1hi-8i9sb-okzukfzkpbrnhst 29827 check slurm allocation
2013-12-12_21:36:42 qr1hi-8i9sb-okzukfzkpbrnhst 29827 node localhost - 1 slots
2013-12-12_21:36:42 qr1hi-8i9sb-okzukfzkpbrnhst 29827 start
@@ -75,7 +75,7 @@ bc. 2013-12-12_21:36:42 qr1hi-8i9sb-okzukfzkpbrnhst 29827 0 stderr hello world
The script's output is captured in the log, which is useful for print statement debugging. However, although this script returned a status code of 0 (success), the job failed. Why? For a job to complete successfully scripts must explicitly add their output to Keep, and then tell Arvados about it. Here is a second try:
<notextile>
-<pre><code>$ <span class="userinput">cat >hello-world.py <<EOF
+<pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">cat >hello-world.py <<EOF
#!/usr/bin/env python
import arvados
@@ -97,15 +97,15 @@ arvados.current_task().set_output(out_collection)
# Done!
EOF</span>
-$ <span class="userinput">chmod +x hello-world-fixed.py</span>
-$ <span class="userinput">cat >the_job <<EOF
+~/<b>you</b>/crunch_scripts$ <span class="userinput">chmod +x hello-world-fixed.py</span>
+~/<b>you</b>/crunch_scripts$ <span class="userinput">cat >~/the_job <<EOF
{
"script":"hello-world-fixed.py",
"script_version":"/home/you/you",
"script_parameters":{}
}
EOF</span>
-$ <span class="userinput">arv-crunch-job --job "$(cat the_job)"</span>
+~/<b>you</b>/crunch_scripts$ <span class="userinput">arv-crunch-job --job "$(cat ~/the_job)"</span>
2013-12-12_21:56:59 qr1hi-8i9sb-79260ykfew5trzl 31578 check slurm allocation
2013-12-12_21:56:59 qr1hi-8i9sb-79260ykfew5trzl 31578 node localhost - 1 slots
2013-12-12_21:57:00 qr1hi-8i9sb-79260ykfew5trzl 31578 start
@@ -134,7 +134,7 @@ $ <span class="userinput">arv-crunch-job --job "$(cat the_job)"</span>
The job succeeded, with output in Keep object @576c44d762ba241b0a674aa43152b52a+53 at . Let's look at our output:
<notextile>
-<pre><code>$ <span class="userinput">arv keep get 576c44d762ba241b0a674aa43152b52a+53/hello.txt</span>
+<pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">arv keep get 576c44d762ba241b0a674aa43152b52a+53/hello.txt</span>
hello world
</code></pre>
</notextile>
@@ -151,6 +151,6 @@ h3. Testing job scripts without SDKs and Keep access
Read and write data to @/tmp/@ instead of Keep. This only works with the Python SDK.
-notextile. <pre><code>$ <span class="userinput">export KEEP_LOCAL_STORE=/tmp</span></code></pre>
+notextile. <pre><code>~$ <span class="userinput">export KEEP_LOCAL_STORE=/tmp</span></code></pre>
Next, "parallel tasks.":tutorial-parallel.html
diff --git a/doc/user/tutorials/tutorial-job1.html.textile.liquid b/doc/user/tutorials/tutorial-job1.html.textile.liquid
index b88ba60..88478bb 100644
--- a/doc/user/tutorials/tutorial-job1.html.textile.liquid
+++ b/doc/user/tutorials/tutorial-job1.html.textile.liquid
@@ -29,7 +29,7 @@ For your first job, you will run the "hash" crunch script using the Arvados syst
Crunch jobs are described using JSON objects. For example:
<notextile>
-<pre><code>$ <span class="userinput">cat >the_job <<EOF
+<pre><code>~$ <span class="userinput">cat >the_job <<EOF
{
"script": "hash",
"script_version": "arvados:master",
@@ -46,13 +46,13 @@ EOF
* @<<EOF@ tells the shell to direct the following lines into the standard input for @cat@ up until it sees the line @EOF@
* @>the_job@ redirects standard output to a file called @the_job@
* @"script"@ specifies the name of the script to run. The script is searched for in the "crunch_scripts/" subdirectory of the @git@ checkout specified by @"script_version"@.
-* @"script_version"@ specifies the version of the script that you wish to run. This can be in the form of an explicit @git@ revision hash, or in the form "repository:branch" (in which case it will take the HEAD of the specified branch). Arvados logs the script version that was used in the run, enabling you to go back and re-run any past job with the guarantee that the exact same code will be used as was used in the previous run. You can access a list of available @git@ repositories on the Arvados workbench through _Access %(rarr)→% Repositories_.
+* @"script_version"@ specifies the version of the script that you wish to run. This can be in the form of an explicit @git@ revision hash, or in the form "repository:branch" (in which case it will take the HEAD of the specified branch). Arvados logs the script version that was used in the run, enabling you to go back and re-run any past job with the guarantee that the exact same code will be used as was used in the previous run. You can access a list of available @git@ repositories on the Arvados workbench under _Compute %(rarr)→% Code repositories_.
* @"script_parameters"@ are provided to the script. In this case, the input is the locator for the collection that we inspected in the previous section.
Use @arv job create@ to actually submit the job. It should print out a JSON object which describes the newly created job:
<notextile>
-<pre><code>$ <span class="userinput">arv -h job create --job "$(cat the_job)"</span>
+<pre><code>~$ <span class="userinput">arv job create --job "$(cat the_job)"</span>
{
"href":"https://qr1hi.arvadosapi.com/arvados/v1/jobs/qr1hi-8i9sb-1pm1t02dezhupss",
"kind":"arvados#job",
@@ -98,22 +98,22 @@ The job is new queued and will start running as soon as it reaches the front of
h2. Monitor job progress
-Go to Workbench, and use the menu to navigate to _Compute %(rarr)→% Jobs_. The job you submitted can be identified by the *uuid* row, which will match the "uuid" field of the JSON object returned when the job was created.
+Go to the Workbench dashboard. Your job should be at the top of the "Recent jobs" table. This table refreshes automatically. When the job has completed successfully, it will show <span class="label label-success">finished</span> in the *Status* column.
-Hit "Refresh" until it finishes. Successful completion is indicated by a green check mark in the *status* column.
+On the command line, you can access log messages while the job runs using @arv job log_tail_follow@:
-You can access log messages while the job runs using @arv job log_tail_follow@:
-
-notextile. <pre><code>$ <span class="userinput">arv job log_tail_follow --uuid qr1hi-8i9sb-xxxxxxxxxxxxxxx</span></code></pre>
+notextile. <pre><code>~$ <span class="userinput">arv job log_tail_follow --uuid qr1hi-8i9sb-xxxxxxxxxxxxxxx</span></code></pre>
This will print out the last several lines of the log for that job.
h2. Inspect the job output
-You can access the job output under the *output* column of the _Compute %(rarr)→% Jobs_ page. Alternately, you can use @arv job get@ to access a JSON object describing the output:
+On the workbench dashboard, look for the *Output* column of the *Recent jobs* table. Click on the link under *Output* for your job to go to the files page with the job output. The files page lists all the files that were output by the job. Click on the link under the *files* column to view a file, or click on the download icon <span class="glyphicon glyphicon-download-alt"></span> to download the output file.
+
+On the command line, you can use @arv job get@ to access a JSON object describing the output:
<notextile>
-<pre><code>$ <span class="userinput">arv -h job get --uuid qr1hi-8i9sb-xxxxxxxxxxxxxxx</span>
+<pre><code>~$ <span class="userinput">arv job get --uuid qr1hi-8i9sb-xxxxxxxxxxxxxxx</span>
{
"href":"https://qr1hi.arvadosapi.com/arvados/v1/jobs/qr1hi-8i9sb-1pm1t02dezhupss",
"kind":"arvados#job",
@@ -162,15 +162,15 @@ You can access the job output under the *output* column of the _Compute %(rarr)&
Now you can list the files in the collection:
<notextile>
-<pre><code>$ <span class="userinput">arv keep get 880b55fb4470b148a447ff38cacdd952+54</span>
-. 78b268d1e03d87f8270bdee9d5d427c5+61 0:61:md5sum.txt
+<pre><code>~$ <span class="userinput">arv keep ls 880b55fb4470b148a447ff38cacdd952+54</span>
+md5sum.txt
</code></pre>
</notextile>
This collection consists of the @md5sum.txt@ file. Use @arv keep get@ to show the contents of the @md5sum.txt@ file:
<notextile>
-<pre><code>$ <span class="userinput">arv keep get 880b55fb4470b148a447ff38cacdd952+54/md5sum.txt</span>
+<pre><code>~$ <span class="userinput">arv keep get 880b55fb4470b148a447ff38cacdd952+54/md5sum.txt</span>
44b8ae3fde7a8a88d2f7ebd237625b4f var-GS000016015-ASM.tsv.bz2
</code></pre>
</notextile>
@@ -179,10 +179,12 @@ This md5 hash matches the md5 hash which we computed earlier.
h2. The job log
-When the job completes, you can access the job log. The keep identifier listed in the @"log"@ field from @arv job get@ specifies a collection. You can list the files in the collection:
+When the job completes, you can access the job log. On the workbench dashboard, this is the link under the *Log* column of the *Recent jobs* table.
+
+On the command line, the keep identifier listed in the @"log"@ field from @arv job get@ specifies a collection. You can list the files in the collection:
<notextile>
-<pre><code>$ <span class="userinput">arv keep ls xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx+91</span>
+<pre><code>~$ <span class="userinput">arv keep ls xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx+91</span>
qr1hi-8i9sb-xxxxxxxxxxxxxxx.log.txt
</code></pre>
</notextile>
@@ -190,7 +192,7 @@ qr1hi-8i9sb-xxxxxxxxxxxxxxx.log.txt
The log collection consists of one log file named with the job id. You can access it using @arv keep get@:
<notextile>
-<pre><code>$ <span class="userinput">arv keep get xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx+91/qr1hi-8i9sb-xxxxxxxxxxxxxxx.log.txt</span>
+<pre><code>~$ <span class="userinput">arv keep get xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx+91/qr1hi-8i9sb-xxxxxxxxxxxxxxx.log.txt</span>
2013-12-16_20:44:35 qr1hi-8i9sb-1pm1t02dezhupss 7575 check slurm allocation
2013-12-16_20:44:35 qr1hi-8i9sb-1pm1t02dezhupss 7575 node compute13 - 8 slots
2013-12-16_20:44:36 qr1hi-8i9sb-1pm1t02dezhupss 7575 start
diff --git a/doc/user/tutorials/tutorial-keep.html.textile.liquid b/doc/user/tutorials/tutorial-keep.html.textile.liquid
index 16c113d..e020823 100644
--- a/doc/user/tutorials/tutorial-keep.html.textile.liquid
+++ b/doc/user/tutorials/tutorial-keep.html.textile.liquid
@@ -27,14 +27,13 @@ In the following tutorials, replace <b><code>you</code></b> with your user id.
First, log into the Arvados VM instance and set up the staging area:
-notextile. <pre><code>$ <span class="userinput">mkdir /scratch/<b>you</b></span></code></pre>
+notextile. <pre><code>~$ <span class="userinput">mkdir /scratch/<b>you</b></span></code></pre>
Next, download the file:
<notextile>
-<pre><code>$ <span class="userinput">mkdir /scratch/<b>you</b></span>
-$ <span class="userinput">cd /scratch/<b>you</b></span>
-$ <span class="userinput">curl -o var-GS000016015-ASM.tsv.bz2 'https://warehouse.personalgenomes.org/warehouse/f815ec01d5d2f11cb12874ab2ed50daa+234+K@ant/var-GS000016015-ASM.tsv.bz2'</span>
+<pre><code>~$ <span class="userinput">cd /scratch/<b>you</b></span>
+/scratch/<b>you</b>$ <span class="userinput">curl -o var-GS000016015-ASM.tsv.bz2 'https://warehouse.personalgenomes.org/warehouse/f815ec01d5d2f11cb12874ab2ed50daa+234+K@ant/var-GS000016015-ASM.tsv.bz2'</span>
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 216M 100 216M 0 0 10.0M 0 0:00:21 0:00:21 --:--:-- 9361k
@@ -45,31 +44,31 @@ $ <span class="userinput">curl -o var-GS000016015-ASM.tsv.bz2 'https://warehouse
If you have your own data, for example @MyData.vcf@, you can use @scp@ or @rsync@ to copy from your local workstation to the shell VM (run this on your local workstation):
-notextile. <pre><code>$ <span class="userinput">scp MyData.vcf <b>you at shell</b>.arvados:/scratch/<b>you</b>/MyData.vcf</span></code></pre>
+notextile. <pre><code>~$ <span class="userinput">scp MyData.vcf <b>you at shell</b>.arvados:/scratch/<b>you</b>/MyData.vcf</span></code></pre>
{% include 'notebox_end' %}
-Now use @arv keep put@ to add your VCF data to Keep:
+Now use @arv keep put@ to add your VCF data to Keep, then delete the local copy of the file:
<notextile>
-<pre><code>$ <span class="userinput">cd /scratch/<b>you</b></span>
-$ <span class="userinput">arv keep put var-GS000016015-ASM.tsv.bz2</span>
+<pre><code>/scratch/<b>you</b>$ <span class="userinput">arv keep put var-GS000016015-ASM.tsv.bz2</span>
c1bad4b39ca5a924e481008009d94e32+210
+/scratch/<b>you</b>$ <span class="userinput">rm var-GS000016015-ASM.tsv.bz2</span>
</code></pre>
</notextile>
-The output value @c1bad4b39ca5a924e481008009d94e32+210@ is the Keep locator. This enables you to access the file you just uploaded, and is explained in the next section.
+The output value @c1bad4b39ca5a924e481008009d94e32+210@ from @arv keep put@ is the Keep locator. This enables you to access the file you just uploaded, and is explained in the next section.
h2. Putting a directory
You can also use @arv keep put@ to add an entire directory:
<notextile>
-<pre><code>$ <span class="userinput">mkdir tmp</span>
-$ <span class="userinput">echo "hello alice" > tmp/alice.txt</span>
-$ <span class="userinput">echo "hello bob" > tmp/bob.txt</span>
-$ <span class="userinput">echo "hello carol" > tmp/carol.txt</span>
-$ <span class="userinput">arv keep put tmp</span>
+<pre><code>/scratch/<b>you</b>$ <span class="userinput">mkdir tmp</span>
+/scratch/<b>you</b>$ <span class="userinput">echo "hello alice" > tmp/alice.txt</span>
+/scratch/<b>you</b>$ <span class="userinput">echo "hello bob" > tmp/bob.txt</span>
+/scratch/<b>you</b>$ <span class="userinput">echo "hello carol" > tmp/carol.txt</span>
+/scratch/<b>you</b>$ <span class="userinput">arv keep put tmp</span>
0M / 0M 100.0%
887cd41e9c613463eab2f0d885c6dd96+83
</code></pre>
@@ -84,23 +83,23 @@ In order to reassemble the file, Keep stores a *collection* data block which lis
In this example we will use @c1bad4b39ca5a924e481008009d94e32+210@ which we added to keep in the previous section. First let us examine the contents of this collection using @arv keep get@:
<notextile>
-<pre><code>$ <span class="userinput">arv keep get c1bad4b39ca5a924e481008009d94e32+210</span>
+<pre><code>/scratch/<b>you</b>$ <span class="userinput">arv keep get c1bad4b39ca5a924e481008009d94e32+210</span>
. 204e43b8a1185621ca55a94839582e6f+67108864 b9677abbac956bd3e86b1deb28dfac03+67108864 fc15aff2a762b13f521baf042140acec+67108864 323d2a3ce20370c4ca1d3462a344f8fd+25885655 0:227212247:var-GS000016015-ASM.tsv.bz2
</code></pre>
</notextile>
- at arv keep get@ fetches the contents of the locator @c1bad4b39ca5a924e481008009d94e32+210 at . This is a locator for a collection data block, so it fetches the contents of the collection. In this example, this collection consists of a single file @var-GS000016015-ASM.tsv.bz2@ which is 227212247 bytes long, and is stored using four sequential data blocks, <code>204e43b8a1185621ca55a94839582e6f+67108864</code>, <code>b9677abbac956bd3e86b1deb28dfac03+67108864</code>, <code>fc15aff2a762b13f521baf042140acec+67108864</code>, <code>323d2a3ce20370c4ca1d3462a344f8fd+25885655</code>.
+The command @arv keep get@ fetches the contents of the locator @c1bad4b39ca5a924e481008009d94e32+210 at . This is a locator for a collection data block, so it fetches the contents of the collection. In this example, this collection consists of a single file @var-GS000016015-ASM.tsv.bz2@ which is 227212247 bytes long, and is stored using four sequential data blocks, <code>204e43b8a1185621ca55a94839582e6f+67108864</code>, <code>b9677abbac956bd3e86b1deb28dfac03+67108864</code>, <code>fc15aff2a762b13f521baf042140acec+67108864</code>, <code>323d2a3ce20370c4ca1d3462a344f8fd+25885655</code>.
Let's use @arv keep get@ to download the first datablock:
-notextile. <pre><code>$ <span class="userinput">arv keep get 204e43b8a1185621ca55a94839582e6f+67108864 > block1</span></code></pre>
+notextile. <pre><code>/scratch/<b>you</b>$ <span class="userinput">arv keep get 204e43b8a1185621ca55a94839582e6f+67108864 > block1</span></code></pre>
Let's look at the size and compute the md5 hash of @block1@:
<notextile>
-<pre><code>$ <span class="userinput">ls -l block1</span>
+<pre><code>/scratch/<b>you</b>$ <span class="userinput">ls -l block1</span>
-rw-r--r-- 1 you group 67108864 Dec 9 20:14 block1
-$ <span class="userinput">md5sum block1</span>
+/scratch/<b>you</b>$ <span class="userinput">md5sum block1</span>
204e43b8a1185621ca55a94839582e6f block1
</code></pre>
</notextile>
@@ -111,12 +110,14 @@ Notice that the block identifer <code>204e43b8a1185621ca55a94839582e6f+67108864<
Next, let's use @arv keep get@ to download and reassemble @var-GS000016015-ASM.tsv.bz2@ using the following command:
-notextile. <pre><code>$ <span class="userinput">arv keep get c1bad4b39ca5a924e481008009d94e32+210/var-GS000016015-ASM.tsv.bz2 .</span></code></pre>
+<notextile>
+<pre><code>/scratch/<b>you</b>$ <span class="userinput">arv keep get c1bad4b39ca5a924e481008009d94e32+210/var-GS000016015-ASM.tsv.bz2 .</span>
+</code></pre>
This downloads the file @var-GS000016015-ASM.tsv.bz2@ described by collection @c1bad4b39ca5a924e481008009d94e32+210@ from Keep and places it into the local directory. Now that we have the file, we can compute the md5 hash of the complete file:
<notextile>
-<pre><code>$ <span class="userinput">md5sum var-GS000016015-ASM.tsv.bz2</span>
+<pre><code>/scratch/<b>you</b>$ <span class="userinput">md5sum var-GS000016015-ASM.tsv.bz2</span>
44b8ae3fde7a8a88d2f7ebd237625b4f var-GS000016015-ASM.tsv.bz2
</code></pre>
</notextile>
@@ -126,9 +127,9 @@ h2. Accessing Collections
There are a couple of other ways to access a collection. You may view the contents of a collection using @arv keep ls@:
<notextile>
-<pre><code>$ <span class="userinput">arv keep ls c1bad4b39ca5a924e481008009d94e32+210</span>
+<pre><code>/scratch/<b>you</b>$ <span class="userinput">arv keep ls c1bad4b39ca5a924e481008009d94e32+210</span>
var-GS000016015-ASM.tsv.bz2
-$ <span class="userinput">arv keep ls -s c1bad4b39ca5a924e481008009d94e32+210</span>
+/scratch/<b>you</b>$ <span class="userinput">arv keep ls -s c1bad4b39ca5a924e481008009d94e32+210</span>
221887 var-GS000016015-ASM.tsv.bz2
</code></pre>
</notextile>
diff --git a/doc/user/tutorials/tutorial-parallel.html.textile.liquid b/doc/user/tutorials/tutorial-parallel.html.textile.liquid
index d8254a6..7985cf3 100644
--- a/doc/user/tutorials/tutorial-parallel.html.textile.liquid
+++ b/doc/user/tutorials/tutorial-parallel.html.textile.liquid
@@ -13,31 +13,35 @@ In the tutorial "writing a crunch script,":tutorial-firstscript.html our script
Start by entering the @crunch_scripts@ directory of your git repository:
<notextile>
-<pre><code>$ <span class="userinput">cd you/crunch_scripts</span>
+<pre><code>~$ <span class="userinput">cd <b>you</b>/crunch_scripts</span>
</code></pre>
</notextile>
-Next, using your favorite text editor, create a new file called @parallel-hash.py@ in the @crunch_scripts@ directory. Add the following code to compute the md5 hash of each file in a collection:
+Next, using @nano@ or your favorite Unix text editor, create a new file called @parallel-hash.py@ in the @crunch_scripts@ directory.
+
+notextile. <pre>~/<b>you</b>/crunch_scripts$ <code class="userinput">nano parallel-hash.py</code></pre>
+
+Add the following code to compute the md5 hash of each file in a
<pre><code class="userinput">{% include 'parallel_hash_script_py' %}</code></pre>
Make the file executable:
-notextile. <pre><code>$ <span class="userinput">chmod +x parallel-hash.py</span></code></pre>
+notextile. <pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">chmod +x parallel-hash.py</span></code></pre>
Next, add the file to @git@ staging, commit and push:
<notextile>
-<pre><code>$ <span class="userinput">git add parallel-hash.py</span>
-$ <span class="userinput">git commit -m"parallel hash"</span>
-$ <span class="userinput">git push origin master</span>
+<pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">git add parallel-hash.py</span>
+~/<b>you</b>/crunch_scripts$ <span class="userinput">git commit -m"parallel hash"</span>
+~/<b>you</b>/crunch_scripts$ <span class="userinput">git push origin master</span>
</code></pre>
</notextile>
You should now be able to run your new script using Crunch, with "script" referring to our new "parallel-hash.py" script. We will use a different input from our previous examples. We will use @887cd41e9c613463eab2f0d885c6dd96+83@ which consists of three files, "alice.txt", "bob.txt" and "carol.txt" (the example collection used previously in "fetching data from Arvados using Keep":tutorial-keep.html).
<notextile>
-<pre><code>$ <span class="userinput">cat >the_job <<EOF
+<pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">cat >~/the_job <<EOF
{
"script": "parallel-hash.py",
"script_version": "you:master",
@@ -47,13 +51,13 @@ You should now be able to run your new script using Crunch, with "script" referr
}
}
EOF</span>
-$ <span class="userinput">arv -h job create --job "$(cat the_job)"</span>
+~/<b>you</b>/crunch_scripts$ <span class="userinput">arv -h job create --job "$(cat ~/the_job)"</span>
{
...
"uuid":"qr1hi-xxxxx-xxxxxxxxxxxxxxx"
...
}
-$ <span class="userinput">arv -h job get --uuid qr1hi-xxxxx-xxxxxxxxxxxxxxx</span>
+~/<b>you</b>/crunch_scripts$ <span class="userinput">arv -h job get --uuid qr1hi-xxxxx-xxxxxxxxxxxxxxx</span>
{
...
"output":"e2ccd204bca37c77c0ba59fc470cd0f7+162",
@@ -65,11 +69,11 @@ $ <span class="userinput">arv -h job get --uuid qr1hi-xxxxx-xxxxxxxxxxxxxxx</spa
Because the job ran in parallel, each instance of parallel-hash creates a separate @md5sum.txt@ as output. Arvados automatically collates theses files into a single collection, which is the output of the job:
<notextile>
-<pre><code>$ <span class="userinput">arv keep get e2ccd204bca37c77c0ba59fc470cd0f7+162</span>
+<pre><code>~/<b>you</b>/crunch_scripts$ <span class="userinput">arv keep get e2ccd204bca37c77c0ba59fc470cd0f7+162</span>
md5sum.txt
md5sum.txt
md5sum.txt
-$ <span class="userinput">arv keep get e2ccd204bca37c77c0ba59fc470cd0f7+162/md5sum.txt</span>
+~/<b>you</b>/crunch_scripts$ <span class="userinput">arv keep get e2ccd204bca37c77c0ba59fc470cd0f7+162/md5sum.txt</span>
0f1d6bcf55c34bed7f92a805d2d89bbf alice.txt
504938460ef369cd275e4ef58994cffe bob.txt
8f3b36aff310e06f3c5b9e95678ff77a carol.txt
diff --git a/doc/user/tutorials/tutorial-trait-search.html.textile.liquid b/doc/user/tutorials/tutorial-trait-search.html.textile.liquid
index b602411..13945ff 100644
--- a/doc/user/tutorials/tutorial-trait-search.html.textile.liquid
+++ b/doc/user/tutorials/tutorial-trait-search.html.textile.liquid
@@ -17,7 +17,7 @@ In the tutorial examples, three angle brackets (>>>) will be used to de
Start by running Python.
<notextile>
-<pre><code>$ <span class="userinput">python</span>
+<pre><code>~$ <span class="userinput">python</span>
Python 2.7.3 (default, Jan 2 2013, 13:56:14)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list