[ARVADOS] updated: 1.3.0-1613-gbaa2bf80c
Git user
git at public.curoverse.com
Mon Sep 16 15:36:37 UTC 2019
Summary of changes:
doc/api/permission-model.html.textile.liquid | 2 +-
...onfigure-azure-blob-storage.html.textile.liquid | 11 ++++----
.../configure-fs-storage.html.textile.liquid | 17 +++++++++---
...configure-s3-object-storage.html.textile.liquid | 9 +++---
doc/install/install-keepstore.html.textile.liquid | 2 +-
lib/config/config.default.yml | 32 +++++++++++++++++++---
lib/config/generated_config.go | 32 +++++++++++++++++++---
sdk/go/arvados/config.go | 3 --
8 files changed, 82 insertions(+), 26 deletions(-)
via baa2bf80cc078868191494ccb1631c426f4e0496 (commit)
via cf5d136d81bd22ce5340243643a4734f3cf20856 (commit)
via 07724cb2413ea22e583b9d579b514d81a97911be (commit)
via e201aabcc4d6d457a662ad0b4e51f3524b5bc865 (commit)
via ef2d67b9251839b44146f1df16f8cdd2d8faa38f (commit)
via 3dec49afe102f61d6db045aa8dadec442d055b71 (commit)
from 76ee36c49088274ab4e8465c0f4b20878d66c706 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
commit baa2bf80cc078868191494ccb1631c426f4e0496
Author: Tom Clegg <tclegg at veritasgenetics.com>
Date: Mon Sep 16 11:31:09 2019 -0400
13647: Add volume config doc links.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg at veritasgenetics.com>
diff --git a/lib/config/config.default.yml b/lib/config/config.default.yml
index b8c7fccca..572a2558e 100644
--- a/lib/config/config.default.yml
+++ b/lib/config/config.default.yml
@@ -813,7 +813,8 @@ Clusters:
Driver: s3
DriverParameters:
- # for s3 driver
+ # for s3 driver -- see
+ # https://doc.arvados.org/install/configure-s3-object-storage.html
AccessKey: aaaaa
SecretKey: aaaaa
Endpoint: ""
@@ -826,7 +827,8 @@ Clusters:
RaceWindow: 24h
UnsafeDelete: false
- # for azure driver
+ # for azure driver -- see
+ # https://doc.arvados.org/install/configure-azure-blob-storage.html
StorageAccountName: aaaaa
StorageAccountKey: aaaaa
StorageBaseURL: core.windows.net
@@ -838,7 +840,8 @@ Clusters:
WriteRaceInterval: 15s
WriteRacePollTime: 1s
- # for local directory driver
+ # for local directory driver -- see
+ # https://doc.arvados.org/install/configure-fs-storage.html
Root: /var/lib/arvados/keep-data
Serialize: false
diff --git a/lib/config/generated_config.go b/lib/config/generated_config.go
index 62c0a5791..32c101a5a 100644
--- a/lib/config/generated_config.go
+++ b/lib/config/generated_config.go
@@ -819,7 +819,8 @@ Clusters:
Driver: s3
DriverParameters:
- # for s3 driver
+ # for s3 driver -- see
+ # https://doc.arvados.org/install/configure-s3-object-storage.html
AccessKey: aaaaa
SecretKey: aaaaa
Endpoint: ""
@@ -832,7 +833,8 @@ Clusters:
RaceWindow: 24h
UnsafeDelete: false
- # for azure driver
+ # for azure driver -- see
+ # https://doc.arvados.org/install/configure-azure-blob-storage.html
StorageAccountName: aaaaa
StorageAccountKey: aaaaa
StorageBaseURL: core.windows.net
@@ -844,7 +846,8 @@ Clusters:
WriteRaceInterval: 15s
WriteRacePollTime: 1s
- # for local directory driver
+ # for local directory driver -- see
+ # https://doc.arvados.org/install/configure-fs-storage.html
Root: /var/lib/arvados/keep-data
Serialize: false
commit cf5d136d81bd22ce5340243643a4734f3cf20856
Author: Tom Clegg <tclegg at veritasgenetics.com>
Date: Mon Sep 16 11:24:20 2019 -0400
13647: Make keepstore fs config more consistent with s3/azure.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg at veritasgenetics.com>
diff --git a/doc/install/configure-azure-blob-storage.html.textile.liquid b/doc/install/configure-azure-blob-storage.html.textile.liquid
index 1bed748ad..2f68c8a4b 100644
--- a/doc/install/configure-azure-blob-storage.html.textile.liquid
+++ b/doc/install/configure-azure-blob-storage.html.textile.liquid
@@ -59,8 +59,9 @@ Volumes are configured in the @Volumes@ section of the cluster configuration fil
# volume. In this example, keep0 has read/write access, and
# keep1 has read-only access.
#
- # If the AccessViaHosts section is empty, all keepstore
- # servers will have read/write access to the volume.
+ # If the AccessViaHosts section is empty or omitted, all
+ # keepstore servers will have read/write access to the
+ # volume.
"http://<span class="userinput">keep0.uuid_prefix.example.com</span>:25107/": {}
"http://<span class="userinput">keep1.uuid_prefix.example.com</span>:25107/": {ReadOnly: true}
diff --git a/doc/install/configure-fs-storage.html.textile.liquid b/doc/install/configure-fs-storage.html.textile.liquid
index 2e9781326..be0a48cb8 100644
--- a/doc/install/configure-fs-storage.html.textile.liquid
+++ b/doc/install/configure-fs-storage.html.textile.liquid
@@ -65,14 +65,23 @@ Note that each volume has an AccessViaHosts section indicating that (for example
Root: /mnt/local-disk
</code></pre></notextile>
-In the case of a network-attached filesystem accessible by all keepstore servers, the AccessViaHosts section is omitted.
+In the case of a network-attached filesystem, the AccessViaHosts section can have multiple entries. If the filesystem is accessible by all keepstore servers, the AccessViaHosts section can be empty, or omitted entirely.
<notextile>
<pre><code>Clusters:
<span class="userinput">uuid_prefix</span>:
Volumes:
- # This volume is used by all keepstore servers.
<span class="userinput">uuid_prefix</span>-nyw5e-<span class="userinput">000000000000002</span>:
+ AccessViaHosts:
+ # This section determines which keepstore servers access the
+ # volume. In this example, keep0 has read/write access, and
+ # keep1 has read-only access.
+ #
+ # If the AccessViaHosts section is empty or omitted, all
+ # keepstore servers will have read/write access to the
+ # volume.
+ "http://<span class="userinput">keep0.uuid_prefix.example.com</span>:25107/": {}
+ "http://<span class="userinput">keep1.uuid_prefix.example.com</span>:25107/": {ReadOnly: true}
Driver: Directory
DriverParameters:
Root: /mnt/network-attached-filesystem
diff --git a/doc/install/configure-s3-object-storage.html.textile.liquid b/doc/install/configure-s3-object-storage.html.textile.liquid
index 56d3a7628..6e725b344 100644
--- a/doc/install/configure-s3-object-storage.html.textile.liquid
+++ b/doc/install/configure-s3-object-storage.html.textile.liquid
@@ -24,8 +24,9 @@ Volumes are configured in the @Volumes@ section of the cluster configuration fil
# volume. In this example, keep0 has read/write access, and
# keep1 has read-only access.
#
- # If the AccessViaHosts section is empty, all keepstore
- # servers will have read/write access to the volume.
+ # If the AccessViaHosts section is empty or omitted, all
+ # keepstore servers will have read/write access to the
+ # volume.
"http://<span class="userinput">keep0.uuid_prefix.example.com</span>:25107/": {}
"http://<span class="userinput">keep1.uuid_prefix.example.com</span>:25107/": {ReadOnly: true}
commit 07724cb2413ea22e583b9d579b514d81a97911be
Author: Tom Clegg <tclegg at veritasgenetics.com>
Date: Mon Sep 16 11:03:40 2019 -0400
13647: Fix typos.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg at veritasgenetics.com>
diff --git a/doc/api/permission-model.html.textile.liquid b/doc/api/permission-model.html.textile.liquid
index 7ee179071..1f08ea419 100644
--- a/doc/api/permission-model.html.textile.liquid
+++ b/doc/api/permission-model.html.textile.liquid
@@ -70,7 +70,7 @@ A privileged user account exists for the use by internal Arvados components. Th
h2. Anoymous user and group
-An Arvado site may be configued to allow users to browse resources without requiring a log in. In this case, permissions for non-logged-in users are associated with the "anonymous" user. To make objects visible to the public, they can be shared with the "anonymous" group. The anonymous user uuid is @{siteprefix}-tpzed-anonymouspublic at . The anonymous group uuid is @{siteprefix}-j7d0g-anonymouspublic at .
+An Arvados site may be configured to allow users to browse resources without requiring a login. In this case, permissions for non-logged-in users are associated with the "anonymous" user. To make objects visible to the public, they can be shared with the "anonymous" group. The anonymous user uuid is @{siteprefix}-tpzed-anonymouspublic at . The anonymous group uuid is @{siteprefix}-j7d0g-anonymouspublic at .
h2. Example
diff --git a/doc/install/configure-azure-blob-storage.html.textile.liquid b/doc/install/configure-azure-blob-storage.html.textile.liquid
index 3699c5048..1bed748ad 100644
--- a/doc/install/configure-azure-blob-storage.html.textile.liquid
+++ b/doc/install/configure-azure-blob-storage.html.textile.liquid
@@ -42,7 +42,7 @@ azure storage container create exampleContainerName</span>
</code></pre>
</notextile>
-Note that Keepstore services may be configued to use multiple Azure Storage accounts and multiple containers within a storage account.
+Note that Keepstore services may be configured to use multiple Azure Storage accounts and multiple containers within a storage account.
h2. Configure keepstore
commit e201aabcc4d6d457a662ad0b4e51f3524b5bc865
Author: Tom Clegg <tclegg at veritasgenetics.com>
Date: Mon Sep 16 11:01:52 2019 -0400
13647: Add AccessViaHosts doc comment.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg at veritasgenetics.com>
diff --git a/doc/install/configure-fs-storage.html.textile.liquid b/doc/install/configure-fs-storage.html.textile.liquid
index aef6f13dc..2e9781326 100644
--- a/doc/install/configure-fs-storage.html.textile.liquid
+++ b/doc/install/configure-fs-storage.html.textile.liquid
@@ -25,7 +25,7 @@ Note that each volume has an AccessViaHosts section indicating that (for example
Volumes:
<span class="userinput">uuid_prefix</span>-nyw5e-<span class="userinput">000000000000000</span>:
AccessViaHosts:
- "http://<span class="userinput">keep0.uuid_prefix.example.com</span>:25107/": {}
+ "http://<span class="userinput">keep0.uuid_prefix.example.com</span>:25107": {}
Driver: Directory
DriverParameters:
# The directory that will be used as the backing store.
@@ -59,7 +59,7 @@ Note that each volume has an AccessViaHosts section indicating that (for example
<span class="userinput">uuid_prefix</span>-nyw5e-<span class="userinput">000000000000001</span>:
AccessViaHosts:
- "http://keep1.<span class="userinput">uuid_prefix</span>.example.com:25107/": {}
+ "http://keep1.<span class="userinput">uuid_prefix</span>.example.com:25107": {}
Driver: Directory
DriverParameters:
Root: /mnt/local-disk
diff --git a/lib/config/config.default.yml b/lib/config/config.default.yml
index 94cd1c602..b8c7fccca 100644
--- a/lib/config/config.default.yml
+++ b/lib/config/config.default.yml
@@ -787,9 +787,24 @@ Clusters:
Volumes:
SAMPLE:
+ # AccessViaHosts specifies which keepstore processes can read
+ # and write data on the volume.
+ #
+ # For a local filesystem, AccessViaHosts has one entry,
+ # indicating which server the filesystem is located on.
+ #
+ # For a network-attached backend accessible by all keepstore
+ # servers, like a cloud storage bucket or an NFS mount,
+ # AccessViaHosts can be empty/omitted.
+ #
+ # Further info/examples:
+ # https://doc.arvados.org/install/configure-fs-storage.html
+ # https://doc.arvados.org/install/configure-s3-object-storage.html
+ # https://doc.arvados.org/install/configure-azure-blob-storage.html
AccessViaHosts:
SAMPLE:
ReadOnly: false
+ "http://host1.example:25107": {}
ReadOnly: false
Replication: 1
StorageClasses:
diff --git a/lib/config/generated_config.go b/lib/config/generated_config.go
index 00cefd70d..62c0a5791 100644
--- a/lib/config/generated_config.go
+++ b/lib/config/generated_config.go
@@ -793,9 +793,24 @@ Clusters:
Volumes:
SAMPLE:
+ # AccessViaHosts specifies which keepstore processes can read
+ # and write data on the volume.
+ #
+ # For a local filesystem, AccessViaHosts has one entry,
+ # indicating which server the filesystem is located on.
+ #
+ # For a network-attached backend accessible by all keepstore
+ # servers, like a cloud storage bucket or an NFS mount,
+ # AccessViaHosts can be empty/omitted.
+ #
+ # Further info/examples:
+ # https://doc.arvados.org/install/configure-fs-storage.html
+ # https://doc.arvados.org/install/configure-s3-object-storage.html
+ # https://doc.arvados.org/install/configure-azure-blob-storage.html
AccessViaHosts:
SAMPLE:
ReadOnly: false
+ "http://host1.example:25107": {}
ReadOnly: false
Replication: 1
StorageClasses:
commit ef2d67b9251839b44146f1df16f8cdd2d8faa38f
Author: Tom Clegg <tclegg at veritasgenetics.com>
Date: Mon Sep 16 11:01:28 2019 -0400
13647: Move Rendezvous doc comment to default config file.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg at veritasgenetics.com>
diff --git a/lib/config/config.default.yml b/lib/config/config.default.yml
index b26987185..94cd1c602 100644
--- a/lib/config/config.default.yml
+++ b/lib/config/config.default.yml
@@ -25,7 +25,13 @@ Clusters:
# listening, and reachable from other hosts in the cluster.
SAMPLE:
InternalURLs:
- "http://example.host:12345": {}
+ "http://host1.example:12345": {}
+ "http://host2.example:12345":
+ # Rendezvous is normally empty/omitted. When changing the
+ # URL of a Keepstore service, Rendezvous should be set to
+ # the old URL (with trailing slash omitted) to preserve
+ # rendezvous ordering.
+ Rendezvous: ""
SAMPLE:
Rendezvous: ""
ExternalURL: "-"
diff --git a/lib/config/generated_config.go b/lib/config/generated_config.go
index bb64fcb8e..00cefd70d 100644
--- a/lib/config/generated_config.go
+++ b/lib/config/generated_config.go
@@ -31,7 +31,13 @@ Clusters:
# listening, and reachable from other hosts in the cluster.
SAMPLE:
InternalURLs:
- "http://example.host:12345": {}
+ "http://host1.example:12345": {}
+ "http://host2.example:12345":
+ # Rendezvous is normally empty/omitted. When changing the
+ # URL of a Keepstore service, Rendezvous should be set to
+ # the old URL (with trailing slash omitted) to preserve
+ # rendezvous ordering.
+ Rendezvous: ""
SAMPLE:
Rendezvous: ""
ExternalURL: "-"
diff --git a/sdk/go/arvados/config.go b/sdk/go/arvados/config.go
index 4a7221f05..076a3c44d 100644
--- a/sdk/go/arvados/config.go
+++ b/sdk/go/arvados/config.go
@@ -297,9 +297,6 @@ func (su URL) String() string {
}
type ServiceInstance struct {
- // Rendezvous is normally empty; when changing the URL of a
- // Keepstore service, Rendezvous can be set to the old URL to
- // preserve rendezvous ordering.
Rendezvous string `json:",omitempty"`
}
commit 3dec49afe102f61d6db045aa8dadec442d055b71
Author: Tom Clegg <tclegg at veritasgenetics.com>
Date: Mon Sep 16 09:51:38 2019 -0400
13647: Use *.uuid_prefix.example.com more consistently.
Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tclegg at veritasgenetics.com>
diff --git a/doc/install/configure-azure-blob-storage.html.textile.liquid b/doc/install/configure-azure-blob-storage.html.textile.liquid
index 174cb6a93..3699c5048 100644
--- a/doc/install/configure-azure-blob-storage.html.textile.liquid
+++ b/doc/install/configure-azure-blob-storage.html.textile.liquid
@@ -61,8 +61,8 @@ Volumes are configured in the @Volumes@ section of the cluster configuration fil
#
# If the AccessViaHosts section is empty, all keepstore
# servers will have read/write access to the volume.
- "http://keep0.zzzzz.example.com:25107/": {}
- "http://keep1.zzzzz.example.com:25107/": {ReadOnly: true}
+ "http://<span class="userinput">keep0.uuid_prefix.example.com</span>:25107/": {}
+ "http://<span class="userinput">keep1.uuid_prefix.example.com</span>:25107/": {ReadOnly: true}
Driver: Azure
DriverParameters:
diff --git a/doc/install/configure-fs-storage.html.textile.liquid b/doc/install/configure-fs-storage.html.textile.liquid
index 2a3117ecb..aef6f13dc 100644
--- a/doc/install/configure-fs-storage.html.textile.liquid
+++ b/doc/install/configure-fs-storage.html.textile.liquid
@@ -25,7 +25,7 @@ Note that each volume has an AccessViaHosts section indicating that (for example
Volumes:
<span class="userinput">uuid_prefix</span>-nyw5e-<span class="userinput">000000000000000</span>:
AccessViaHosts:
- "http://keep0.zzzzz.example.com:25107/": {}
+ "http://<span class="userinput">keep0.uuid_prefix.example.com</span>:25107/": {}
Driver: Directory
DriverParameters:
# The directory that will be used as the backing store.
@@ -59,7 +59,7 @@ Note that each volume has an AccessViaHosts section indicating that (for example
<span class="userinput">uuid_prefix</span>-nyw5e-<span class="userinput">000000000000001</span>:
AccessViaHosts:
- "http://keep1.zzzzz.example.com:25107/": {}
+ "http://keep1.<span class="userinput">uuid_prefix</span>.example.com:25107/": {}
Driver: Directory
DriverParameters:
Root: /mnt/local-disk
diff --git a/doc/install/configure-s3-object-storage.html.textile.liquid b/doc/install/configure-s3-object-storage.html.textile.liquid
index 9becc9a58..56d3a7628 100644
--- a/doc/install/configure-s3-object-storage.html.textile.liquid
+++ b/doc/install/configure-s3-object-storage.html.textile.liquid
@@ -26,8 +26,8 @@ Volumes are configured in the @Volumes@ section of the cluster configuration fil
#
# If the AccessViaHosts section is empty, all keepstore
# servers will have read/write access to the volume.
- "http://keep0.zzzzz.example.com:25107/": {}
- "http://keep1.zzzzz.example.com:25107/": {ReadOnly: true}
+ "http://<span class="userinput">keep0.uuid_prefix.example.com</span>:25107/": {}
+ "http://<span class="userinput">keep1.uuid_prefix.example.com</span>:25107/": {ReadOnly: true}
Driver: S3
DriverParameters:
diff --git a/doc/install/install-keepstore.html.textile.liquid b/doc/install/install-keepstore.html.textile.liquid
index 5d0e2df80..528d52f91 100644
--- a/doc/install/install-keepstore.html.textile.liquid
+++ b/doc/install/install-keepstore.html.textile.liquid
@@ -64,7 +64,7 @@ Add or update the following sections of @/etc/arvados/config.yml@ as needed. Ref
Services:
Keepstore:
InternalURLs:
- "http://<span class="userinput">keep0.uuid_prefix.your.domain</span>:25107/": {}
+ "http://<span class="userinput">keep0.uuid_prefix.example.com</span>:25107/": {}
API:
MaxKeepBlobBuffers: 128
</code></pre>
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list