[ARVADOS] updated: 1.3.0-1917-g8e1ca7677

Git user git at public.curoverse.com
Tue Nov 26 14:36:16 UTC 2019


Summary of changes:
 .../_includes/_vocabulary_migrate_py.liquid              | 15 +++++++++------
 doc/admin/workbench2-vocabulary.html.textile.liquid      | 16 +++++++++++++++-
 .../lib/arvbox/docker/service/workbench2/run-service     |  4 ++--
 3 files changed, 26 insertions(+), 9 deletions(-)
 rename tools/vocabulary-migrate/vocabulary-migrate.py => doc/_includes/_vocabulary_migrate_py.liquid (91%)

       via  8e1ca7677e20b4b2e3eb3c473bd062de19a26e36 (commit)
       via  fab8c884b8aec06d2eefa4d2fd935deacd7290ec (commit)
       via  28fd46332160103e5de4a6feb326eda4c6cb0022 (commit)
      from  7c0b5fe28d1a05132791dcd3c9c9fdd26fabdf71 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.


commit 8e1ca7677e20b4b2e3eb3c473bd062de19a26e36
Author: Lucas Di Pentima <ldipentima at veritasgenetics.com>
Date:   Tue Nov 26 11:35:10 2019 -0300

    15849: Moves script to the documentation, includes it on the admin section.
    
    Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima at veritasgenetics.com>

diff --git a/tools/vocabulary-migrate/vocabulary-migrate.py b/doc/_includes/_vocabulary_migrate_py.liquid
similarity index 96%
rename from tools/vocabulary-migrate/vocabulary-migrate.py
rename to doc/_includes/_vocabulary_migrate_py.liquid
index e72983885..3bccf6108 100644
--- a/tools/vocabulary-migrate/vocabulary-migrate.py
+++ b/doc/_includes/_vocabulary_migrate_py.liquid
@@ -1,9 +1,9 @@
 #!/usr/bin/env python
-#
-# Copyright (C) The Arvados Authors. All rights reserved.
-#
-# SPDX-License-Identifier: AGPL-3.0
-#
+{% comment %}
+Copyright (C) The Arvados Authors. All rights reserved.
+
+SPDX-License-Identifier: CC-BY-SA-3.0
+{% endcomment %}
 
 import argparse
 import copy
diff --git a/doc/admin/workbench2-vocabulary.html.textile.liquid b/doc/admin/workbench2-vocabulary.html.textile.liquid
index 82c384c28..4ff35c363 100644
--- a/doc/admin/workbench2-vocabulary.html.textile.liquid
+++ b/doc/admin/workbench2-vocabulary.html.textile.liquid
@@ -48,4 +48,18 @@ The @values@ member is optional and is used to define valid key/label pairs when
 
 When any key or value has more than one label option, Workbench2's user interface will allow the user to select any of the options. But because only the IDs are saved in the system, when the property is displayed in the user interface, the label shown will be the first of each group defined in the vocabulary file. For example, the user could select the property key @Species@ and @Homo sapiens@ as its value, but the user interface will display it as @Animal: Human@ because those labels are the first in the vocabulary definition.
 
-Internally, Workbench2 uses the IDs to do property based searches, so if the user searches by @Animal: Human@ or @Species: Homo sapiens@, both will return the same results.
\ No newline at end of file
+Internally, Workbench2 uses the IDs to do property based searches, so if the user searches by @Animal: Human@ or @Species: Homo sapiens@, both will return the same results.
+
+h2. Properties migration
+
+After installing the new vocabulary definition, it may be necessary to migrate preexisting properties that were set up using literal strings. This can be a big task depending on the number of properties on the vocabulary and the amount of collections and projects on the cluster.
+
+To help with this task we provide below a migration example script that accepts the new vocabulary definition file as an input, and uses the @ARVADOS_API_TOKEN@ and @ARVADOS_API_HOST@ environment variables to connect to the cluster, search for every collection and group that has properties with labels defined on the vocabulary file, and migrates them to the corresponding identifiers.
+
+This script will not run if the vocabulary file has duplicated labels for different keys or for different values inside a key, this is a failsafe mechanism to avoid migration errors.
+
+Please take into account that this script requires admin credentials. It also offers a @--dry-run@ flag that will report what changes are required without applying them, so it can be reviewed by an administrator.
+
+{% codeblock as python %}
+{% include 'vocabulary_migrate_py' %}
+{% endcodeblock %}
\ No newline at end of file

commit fab8c884b8aec06d2eefa4d2fd935deacd7290ec
Author: Lucas Di Pentima <ldipentima at veritasgenetics.com>
Date:   Tue Nov 26 11:16:18 2019 -0300

    15849: Errors out when a duplicated label is detected on the vocabulary.
    
    Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima at veritasgenetics.com>

diff --git a/tools/vocabulary-migrate/vocabulary-migrate.py b/tools/vocabulary-migrate/vocabulary-migrate.py
index 72bbd0ecb..e72983885 100644
--- a/tools/vocabulary-migrate/vocabulary-migrate.py
+++ b/tools/vocabulary-migrate/vocabulary-migrate.py
@@ -18,6 +18,9 @@ import arvados.util
 logger = logging.getLogger('arvados.vocabulary_migrate')
 logger.setLevel(logging.INFO)
 
+class VocabularyError(Exception):
+    pass
+
 opts = argparse.ArgumentParser(add_help=False)
 opts.add_argument('--vocabulary-file', type=str, metavar='PATH', default=None,
                   help="""
@@ -54,7 +57,7 @@ def _label_to_id_mappings(data, obj_name):
             if obj_lbl not in result:
                 result[obj_lbl] = obj_id
             else:
-                logger.warning('{} label "{}" for {} ID "{}" already seen at {} ID "{}". Skipping.'.format(obj_name, obj_lbl, obj_name, obj_id, obj_name, result[obj_lbl]))
+                raise VocabularyError('{} label "{}" for {} ID "{}" already seen at {} ID "{}".'.format(obj_name, obj_lbl, obj_name, obj_id, obj_name, result[obj_lbl]))
     return result
 
 def key_labels_to_ids(vocab):

commit 28fd46332160103e5de4a6feb326eda4c6cb0022
Author: Lucas Di Pentima <ldipentima at veritasgenetics.com>
Date:   Tue Nov 26 11:14:56 2019 -0300

    15849: Fixes file paths on wb2's arvbox config.
    
    Arvados-DCO-1.1-Signed-off-by: Lucas Di Pentima <ldipentima at veritasgenetics.com>

diff --git a/tools/arvbox/lib/arvbox/docker/service/workbench2/run-service b/tools/arvbox/lib/arvbox/docker/service/workbench2/run-service
index 85c03399f..e14704d71 100755
--- a/tools/arvbox/lib/arvbox/docker/service/workbench2/run-service
+++ b/tools/arvbox/lib/arvbox/docker/service/workbench2/run-service
@@ -21,8 +21,8 @@ fi
 cat <<EOF > /usr/src/workbench2/public/config.json
 {
   "API_HOST": "${localip}:${services[controller-ssl]}",
-  "VOCABULARY_URL": "vocabulary-example.json",
-  "FILE_VIEWERS_CONFIG_URL": "file-viewers-example.json"
+  "VOCABULARY_URL": "/vocabulary-example.json",
+  "FILE_VIEWERS_CONFIG_URL": "/file-viewers-example.json"
 }
 EOF
 

-----------------------------------------------------------------------


hooks/post-receive
-- 




More information about the arvados-commits mailing list