[ARVADOS] updated: beb640f493b0828898e83608e6994ff95811dada
Git user
git at public.curoverse.com
Tue Jan 17 20:46:22 EST 2017
Summary of changes:
doc/install/install-docker.html.textile.liquid | 241 -------------------------
doc/user/topics/arv-docker.html.textile.liquid | 114 ++++++------
2 files changed, 61 insertions(+), 294 deletions(-)
delete mode 100644 doc/install/install-docker.html.textile.liquid
via beb640f493b0828898e83608e6994ff95811dada (commit)
from 5add8076df3bb034a9e0029a52cbc61e4bb9275e (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
commit beb640f493b0828898e83608e6994ff95811dada
Author: Ward Vandewege <ward at curoverse.com>
Date: Tue Jan 17 20:43:07 2017 -0500
More documentation updates to reflect the end of support for Debian
Wheezy and CentOS 6. Also remove obsolete arv-docker installation page,
which was no longer linked from anywhere.
refs #10800
diff --git a/doc/install/install-docker.html.textile.liquid b/doc/install/install-docker.html.textile.liquid
deleted file mode 100644
index 922c329..0000000
--- a/doc/install/install-docker.html.textile.liquid
+++ /dev/null
@@ -1,241 +0,0 @@
----
-layout: default
-navsection: installguide
-title: Build and install Docker images
-...
-
-This method is intended for evaluation and development on a local workstation. It is not suitable for production use in a cluster deployment.
-
-h2. Prerequisites
-
-# A GNU/Linux (virtual) machine
-# A working Docker installation (see "Installing Docker":https://docs.docker.com/installation/)
-# A working Go installation (see "Install the Go tools":https://golang.org/doc/install)
-# A working Ruby installation, with the Bundler gem installed
-
-h3. Install Ruby and Bundler
-
-{% include 'install_ruby_and_bundler' %}
-
-h2. Download the source tree
-
-<notextile>
-<pre><code>~$ <span class="userinput">cd $HOME</span> # (or wherever you want to install)
-~$ <span class="userinput">git clone https://github.com/curoverse/arvados.git</span>
-</code></pre></notextile>
-
-See also: "Downloading the source code":https://dev.arvados.org/projects/arvados/wiki/Download on the Arvados wiki.
-
-h2. Building the Arvados Docker containers
-
-First of all, a suitable @config.yml@ file is needed. The easiest way to generate one based on the provided @config.yml.example@ file is to run the @build.sh@ script. If no @config.yml@ file exists, it will will prompt for a few inputs, write the @config.yml@ file, and then proceed to build all the Docker containers. If @config.yml@ exists, invoking @build.sh@ will simply build all Docker containers or update those that need to be updated.
-
-Run @build.sh@ without arguments to generate @config.yml@ and build all Docker containers (this will take a while!):
-
-<notextile>
-<pre><code>
-~$ <span class="userinput">./build.sh</span>
-Generating config.yml.
-Arvados needs to know the email address of the administrative user,
-so that when that user logs in they are automatically made an admin.
-This should be an email address associated with a Google account.
-
-Enter your Google ID email address here:
-
-...
-
-Step 7 : ADD generated/setup.sh /usr/local/bin/setup.sh
- ---> d7c0e7fdf7ab
-Removing intermediate container f3d81180795d
-Step 8 : CMD ["/usr/bin/supervisord", "-n"]
- ---> Running in 84c64cb9f0d5
- ---> d6cbb5002604
-Removing intermediate container 84c64cb9f0d5
-Successfully built d6cbb5002604
-date >shell-image
-</code></pre></notextile>
-
-If all goes well, you should now have a number of Docker images built (the image id will be different):
-
-<notextile>
-<pre><code>~$ <span class="userinput">docker.io images</span>
-REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
-arvados/shell latest d6cbb5002604 10 minutes ago 1.613 GB
-arvados/sso latest 377f1fa0108e 11 minutes ago 1.807 GB
-arvados/keep latest ade0e0d2dd00 12 minutes ago 210.8 MB
-arvados/keepproxy latest b39ef0baba02 12 minutes ago 241.6 MB
-arvados/workbench latest b0e4fb6da385 12 minutes ago 2.218 GB
-arvados/doc latest 4b64daec9454 12 minutes ago 1.524 GB
-arvados/compute latest 7f1f5f7faf54 13 minutes ago 1.862 GB
-arvados/slurm latest f5bfd1008e6b 17 minutes ago 1.573 GB
-arvados/api latest 6b93c5f5fc42 17 minutes ago 2.274 GB
-arvados/passenger latest add2d11fdf24 18 minutes ago 1.738 GB
-arvados/base latest 81eaadd0c6f5 22 minutes ago 1.463 GB
-arvados/debian 7.6 f339ce275c01 6 days ago 116.8 MB
-arvados/debian latest f339ce275c01 6 days ago 116.8 MB
-arvados/debian wheezy f339ce275c01 6 days ago 116.8 MB
-crosbymichael/skydock latest e985023521f6 3 months ago 510.7 MB
-crosbymichael/skydns next 79c99a4608ed 3 months ago 525 MB
-crosbymichael/skydns latest 1923ce648d4c 5 months ago 137.5 MB
-</code></pre></notextile>
-
-h2. Updating the Arvados Docker containers
-
- at build.sh@ takes a few optional arguments:
-
-<notextile>
-<pre><code>
-~$ <span class="userinput"> ./build.sh --help</span>
-
-usage: ./build.sh [options]
-
-Calling ./build.sh without arguments will build all Arvados docker images
-
-./build.sh options:
- -h, --help Print this help text
- clean Clear all build information
- realclean clean and remove all Arvados Docker images except arvados/debian
- deepclean realclean and remove arvados/debian, crosbymichael/skydns and
- crosbymichael/skydns Docker images
-</code></pre></notextile>
-
-If there has been an update to the Arvados Docker building code, it is safest to rebuild the Arvados Docker images from scratch. All build information can be cleared with the 'clean' option to build.sh:
-
-<notextile>
-<pre><code>~$ <span class="userinput">./build.sh clean</span></code></pre>
-</notextile>
-
-You can also use 'realclean', which does what 'clean' does and in addition removes all Arvados Docker containers and images from your system, with the exception of the arvados/debian image.
-
-<notextile>
-<pre><code>~$ <span class="userinput">./build.sh realclean</span></code></pre>
-</notextile>
-
-Finally, the 'deepclean' option does what 'realclean' does, and also removes the arvados/debian, crosbymichael/skydns and crosbymichael/skydock images.
-
-<notextile>
-<pre><code>~$ <span class="userinput">./build.sh deepclean</span></code></pre>
-</notextile>
-
-h2. Running the Arvados Docker containers
-
-The @arvdock@ command can be used to start and stop the docker containers. It has a number of options:
-
-<notextile>
-<pre><code>
-~$ <span class="userinput">./arvdock -h</span>
-
-usage: ./arvdock (start|stop|restart|reset|test) [options]
-
-start run new or restart stopped arvados containers
-stop stop arvados containers
-restart stop and then start arvados containers
-reset stop and delete containers WARNING: this will delete the data inside Arvados!
-test run tests
-
-./arvdock start/stop/restart options:
- -d[port], --doc[=port] Documentation server (default port 9898)
- -w[port], --workbench[=port] Workbench server (default port 9899)
- -s[port], --sso[=port] SSO server (default port 9901)
- -a[port], --api[=port] API server (default port 9900)
- -c, --compute Compute nodes (starts 2)
- -v, --vm Shell server
- -n, --nameserver Nameserver
- -k, --keep Keep servers
- -p, --keepproxy Keepproxy server
- -h, --help Display this help and exit
-
- If no options are given, the action is applied to all servers.
-
-./arvdock test [testname] [testname] ...
- By default, all tests are run.
-</code>
-</pre>
-</notextile>
-
-To debug issues with the Docker containers use @docker exec@, for example:
-
-<notextile>
-<pre><code>
-~$ <span class="userinput">docker exec -t -i api_server /bin/bash</span>
-</code></pre></notextile>
-
-To start the docker containers:
-
-<notextile>
-<pre><code>
-~$ <span class="userinput">./arvdock start</span>
-start_api=false
-start_compute=false
-start_doc=false
-start_keep=false
-start_keepproxy=false
-start_nameserver=false
-start_sso=false
-start_vm=false
-start_workbench=false
-Starting crosbymichael/skydns container...
-skydns
-/usr/bin/docker.io run -d -p 172.17.42.1:53:53/udp --name skydns crosbymichael/skydns -nameserver 8.8.8.8:53 -domain arvados
-89922ec786cbd2098ed6bae205468a675657cdb2db0e0bfdfe76d1e43cb2fe35
-Starting crosbymichael/skydock container...
-skydock
-/usr/bin/docker.io run -d -v /var/run/docker.sock:/docker.sock --name skydock crosbymichael/skydock -ttl 30 -environment dev -s /docker.sock -domain arvados -name skydns
-de6a35bb20fb9063fb97218ba2554974546eed969d43b1f5aa31a1ac9576e802
-Starting container: api_server
- /usr/bin/docker.io start api_server
-Started container: api_server
-Starting container: compute0
- /usr/bin/docker.io start compute0
-Started container: compute0
-Starting container: compute1
- /usr/bin/docker.io start compute1
-Started container: compute1
-Starting container: keep_server_0
- /usr/bin/docker.io start keep_server_0
-Started container: keep_server_0
-Starting container: keep_server_1
- /usr/bin/docker.io start keep_server_1
-Started container: keep_server_1
-Starting container: keepproxy_server
- /usr/bin/docker.io start keepproxy_server
-Started container: keepproxy_server
-Starting container: doc_server
- /usr/bin/docker.io start doc_server
-Started container: doc_server
-
-******************************************************************
-You can access the Arvados documentation at http://doc.dev.arvados
-******************************************************************
-
-Starting container: shell
- /usr/bin/docker.io start shell
-Started container: shell
-Starting container: workbench_server
- /usr/bin/docker.io start workbench_server
-Started container: workbench_server
-
-********************************************************************
-You can access the Arvados workbench at http://workbench.dev.arvados
-********************************************************************
-
-*****************************************************************
-To access Arvados you must add the Arvados nameserver to the top
-of your DNS configuration in /etc/resolv.conf:
-nameserver 172.17.42.1
-******************************************************************
-
-</code></pre></notextile>
-
-h2. Accessing workbench
-
-Point your browser to the Dockerized workbench:
-
-<notextile>
-<pre><code><span class="userinput">http://workbench.dev.arvados</span>
-</code></pre>
-</notextile>
-
-Now use the google account you specified as @API_AUTO_ADMIN_USER@ in @config.yml@ to log in.
-
-You will be prompted by your browser that you are accessing a site with an untrusted SSL certificate. This is normal; by default the Arvados Docker installation uses self-signed SSL certificates for the SSO and API servers, respectively. If you use a local SSO server in a Docker container, you will be prompted *twice*. The default is to use the Curoverse SSO server.
diff --git a/doc/user/topics/arv-docker.html.textile.liquid b/doc/user/topics/arv-docker.html.textile.liquid
index 5ccd5ac..bbae7ec 100644
--- a/doc/user/topics/arv-docker.html.textile.liquid
+++ b/doc/user/topics/arv-docker.html.textile.liquid
@@ -51,31 +51,27 @@ Next, enter the container using @docker run@, providing the arvados/jobs image a
<notextile>
<pre><code>$ <span class="userinput">docker run --interactive --tty --user root arvados/jobs /bin/bash</span>
-root at a0e8299b59aa:/#
+root at fbf1d0f529d5:/#
</code></pre>
</notextile>
Next, update the package list using @apt-get update at .
<notextile>
-<pre><code>root at a0e8299b59aa:/# <span class="userinput">apt-get update</span>
-Get:1 http://apt.arvados.org wheezy Release.gpg [490 B]
-Get:2 http://apt.arvados.org wheezy Release [1568 B]
-Get:3 http://apt.arvados.org wheezy/main amd64 Packages [34.6 kB]
-Get:4 http://ftp.us.debian.org wheezy Release.gpg [1655 B]
-Get:5 http://ftp.us.debian.org wheezy-updates Release.gpg [836 B]
-Get:6 http://ftp.us.debian.org wheezy Release [168 kB]
-Ign http://apt.arvados.org wheezy/main Translation-en
-Get:7 http://security.debian.org wheezy/updates Release.gpg [836 B]
-Get:8 http://security.debian.org wheezy/updates Release [102 kB]
-Get:9 http://ftp.us.debian.org wheezy-updates Release [124 kB]
-Get:10 http://ftp.us.debian.org wheezy/main amd64 Packages [5841 kB]
-Get:11 http://security.debian.org wheezy/updates/main amd64 Packages [218 kB]
-Get:12 http://security.debian.org wheezy/updates/main Translation-en [123 kB]
-Hit http://ftp.us.debian.org wheezy/main Translation-en
-Hit http://ftp.us.debian.org wheezy-updates/main amd64 Packages/DiffIndex
-Hit http://ftp.us.debian.org wheezy-updates/main Translation-en/DiffIndex
-Fetched 6617 kB in 5s (1209 kB/s)
+<pre><code>root at fbf1d0f529d5:/# apt-get update
+Hit http://security.debian.org jessie/updates InRelease
+Ign http://httpredir.debian.org jessie InRelease
+Ign http://apt.arvados.org jessie InRelease
+Hit http://apt.arvados.org jessie Release.gpg
+Get:1 http://security.debian.org jessie/updates/main amd64 Packages [431 kB]
+Hit http://apt.arvados.org jessie Release
+Hit http://httpredir.debian.org jessie-updates InRelease
+Get:2 http://apt.arvados.org jessie/main amd64 Packages [257 kB]
+Get:3 http://httpredir.debian.org jessie-updates/main amd64 Packages [17.6 kB]
+Hit http://httpredir.debian.org jessie Release.gpg
+Hit http://httpredir.debian.org jessie Release
+Get:4 http://httpredir.debian.org jessie/main amd64 Packages [9049 kB]
+Fetched 9755 kB in 2s (3894 kB/s)
Reading package lists... Done
</code></pre>
</notextile>
@@ -83,45 +79,57 @@ Reading package lists... Done
In this example, we will install the "R" statistical language Debian package "r-base-core". Use @apt-get install@:
<notextile>
-<pre><code>root at a0e8299b59aa:/# <span class="userinput">apt-get install r-base-core</span>
+<pre><code>root at fbf1d0f529d5:/# <span class="userinput">apt-get install r-base-core</span>
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
[...]
-libxv1 libxxf86dga1 libxxf86vm1 r-base-core r-base-dev r-base-html r-cran-boot r-cran-class r-cran-cluster r-cran-codetools
+ libxxf86vm1 make patch r-base-core r-base-dev r-cran-boot r-cran-class
+ r-cran-cluster r-cran-codetools r-cran-foreign r-cran-kernsmooth
+ r-cran-lattice r-cran-mass r-cran-matrix r-cran-mgcv r-cran-nlme r-cran-nnet
+ r-cran-rpart r-cran-spatial r-cran-survival r-doc-html r-recommended
[...]
Suggested packages:
[...]
The following NEW packages will be installed:
[...]
- libxv1 libxxf86dga1 libxxf86vm1 r-base r-base-core r-base-dev r-base-html r-cran-boot r-cran-class r-cran-cluster
+ libxxf86vm1 make patch r-base-core r-base-dev r-cran-boot r-cran-class
+ r-cran-cluster r-cran-codetools r-cran-foreign r-cran-kernsmooth
+ r-cran-lattice r-cran-mass r-cran-matrix r-cran-mgcv r-cran-nlme r-cran-nnet
+ r-cran-rpart r-cran-spatial r-cran-survival r-doc-html r-recommended
[...]
-0 upgraded, 107 newly installed, 0 to remove and 9 not upgraded.
-Need to get 88.2 MB of archives.
-After this operation, 219 MB of additional disk space will be used.
+0 upgraded, 203 newly installed, 0 to remove and 39 not upgraded.
+Need to get 124 MB of archives.
+After this operation, 334 MB of additional disk space will be used.
Do you want to continue [Y/n]? y
[...]
-Get:85 http://ftp.us.debian.org/debian/ wheezy/main r-base-core amd64 2.15.1-4 [20.6 MB]
-Get:86 http://ftp.us.debian.org/debian/ wheezy/main r-base-dev all 2.15.1-4 [3882 B]
-Get:87 http://ftp.us.debian.org/debian/ wheezy/main r-cran-boot all 1.3-5-1 [472 kB]
+Get:130 http://httpredir.debian.org/debian/ jessie/main r-cran-cluster amd64 1.15.3-1 [475 kB]
+Get:131 http://httpredir.debian.org/debian/ jessie/main r-base-dev all 3.1.1-1 [4018 B]
+Get:132 http://httpredir.debian.org/debian/ jessie/main r-cran-boot all 1.3-13-1 [571 kB]
+Get:133 http://httpredir.debian.org/debian/ jessie/main r-cran-codetools all 0.2-9-1 [45.7 kB]
+Get:134 http://httpredir.debian.org/debian/ jessie/main r-cran-rpart amd64 4.1-8-1 [862 kB]
+Get:135 http://httpredir.debian.org/debian/ jessie/main r-cran-foreign amd64 0.8.61-1 [213 kB]
[...]
-Fetched 88.2 MB in 2min 17s (642 kB/s)
-Extracting templates from packages: 100%
-Preconfiguring packages ...
+Fetched 124 MB in 52s (2380 kB/s)
+debconf: delaying package configuration, since apt-utils is not installed
[...]
-Unpacking r-base-core (from .../r-base-core_2.15.1-4_amd64.deb) ...
+Unpacking r-base-core (3.1.1-1+b2) ...
Selecting previously unselected package r-base-dev.
-Unpacking r-base-dev (from .../r-base-dev_2.15.1-4_all.deb) ...
+Preparing to unpack .../r-base-dev_3.1.1-1_all.deb ...
+Unpacking r-base-dev (3.1.1-1) ...
Selecting previously unselected package r-cran-boot.
-Unpacking r-cran-boot (from .../r-cran-boot_1.3-5-1_all.deb) ...
+Preparing to unpack .../r-cran-boot_1.3-13-1_all.deb ...
+Unpacking r-cran-boot (1.3-13-1) ...
+Selecting previously unselected package r-cran-mass.
[...]
-Setting up r-base-core (2.15.1-4) ...
-Setting R_PAPERSIZE_USER default to 'a4'
+Setting up r-base-core (3.1.1-1+b2) ...
Creating config file /etc/R/Renviron with new version
-Setting up r-base-dev (2.15.1-4) ...
-Setting up r-cran-boot (1.3-5-1) ...
+Setting up r-base-dev (3.1.1-1) ...
+Setting up r-cran-boot (1.3-13-1) ...
+Setting up r-cran-mass (7.3-34-1) ...
+Setting up r-cran-class (7.3-11-1) ...
[...]
</code></pre>
</notextile>
@@ -129,11 +137,10 @@ Setting up r-cran-boot (1.3-5-1) ...
Now we can verify that "R" is installed:
<notextile>
-<pre><code>root at a0e8299b59aa:/# <span class="userinput">R</span>
+<pre><code>root at fbf1d0f529d5:/# <span class="userinput">R</span>
-R version 2.15.1 (2012-06-22) -- "Roasted Marshmallows"
-Copyright (C) 2012 The R Foundation for Statistical Computing
-ISBN 3-900051-07-0
+R version 3.1.1 (2014-07-10) -- "Sock it to Me"
+Copyright (C) 2014 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
@@ -156,16 +163,17 @@ Note that you are not limited to installing Debian packages. You may compile pr
h2. Create a new image
-We're now ready to create a new Docker image. First, quit the container, then use @docker commit@ to create a new image from the stopped container. The container id can be found in the default hostname of the container displayed in the prompt, in this case @a0e8299b59aa@:
+We're now ready to create a new Docker image. First, quit the container, then use @docker commit@ to create a new image from the stopped container. The container id can be found in the default hostname of the container displayed in the prompt, in this case @fbf1d0f529d5@:
<notextile>
-<pre><code>root at a0e8299b59aa:/# <span class="userinput">exit</span>
-$ <span class="userinput">docker commit a0e8299b59aa arvados/jobs-with-r</span>
-33ea6b87792364cb9989a149c36a31e5a9c8cf96694ba05f66545ad7b842522e
+<pre><code>root at fbf1d0f529d5:/# <span class="userinput">exit</span>
+$ <span class="userinput">docker commit fbf1d0f529d5 arvados/jobs-with-r</span>
+sha256:2818853ff9f9af5d7f77979803baac9c4710790ad2b84c1a754b02728fdff205
$ <span class="userinput">docker images</span>
-REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
-arvados/jobs-with-r latest 33ea6b877923 43 seconds ago 1.607 GB
-arvados/jobs latest 3132168f2acb 22 hours ago 1.314 GB
+$ docker images |head
+REPOSITORY TAG IMAGE ID CREATED SIZE
+arvados/jobs-with-r latest 2818853ff9f9 9 seconds ago 703.1 MB
+arvados/jobs latest 12b9f859d48c 4 days ago 362 MB
</code></pre>
</notextile>
@@ -175,12 +183,12 @@ Finally, we are ready to upload the new Docker image to Arvados. Use @arv keep
<notextile>
<pre><code>$ <span class="userinput">arv keep docker arvados/jobs-with-r</span>
-1591M / 1591M 100.0%
-Collection saved as 'Docker image arvados/jobs-with-r:latest 33ea6b877923'
-qr1hi-4zz18-3fk2px2ji25nst2
+703M / 703M 100.0%
+Collection saved as 'Docker image arvados/jobs-with-r:latest 2818853ff9f9'
+qr1hi-4zz18-abcdefghijklmno
$ <span class="userinput">arv keep docker</span>
REPOSITORY TAG IMAGE ID COLLECTION CREATED
-arvados/jobs-with-r latest 33ea6b877923 qr1hi-4zz18-3fk2px2ji25nst2 Thu Oct 16 13:58:53 2014
+arvados/jobs-with-r latest 2818853ff9f9 qr1hi-4zz18-abcdefghijklmno Tue Jan 17 20:35:53 2017
</code></pre>
</notextile>
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list