[ARVADOS] updated: 2.1.0-1915-gd4e10c048
Git user
git at public.arvados.org
Wed Feb 16 20:07:53 UTC 2022
Summary of changes:
doc/_config.yml | 3 +-
doc/admin/config-migration.html.textile.liquid | 106 ---------------------
doc/admin/config.html.textile.liquid | 2 -
doc/admin/upgrading.html.textile.liquid | 12 +--
.../maintenance-and-upgrading.html.textile.liquid | 10 +-
5 files changed, 12 insertions(+), 121 deletions(-)
delete mode 100644 doc/admin/config-migration.html.textile.liquid
discards 0a26fb70b43f102d98220fef914a021413025d9d (commit)
via d4e10c0482b28a1c2fb7ef48e69a673d2c6ea15a (commit)
via 1fc3e6081d1d016184f328af84c1aaf330920be0 (commit)
This update added new revisions after undoing existing revisions. That is
to say, the old revision is not a strict subset of the new revision. This
situation occurs when you --force push a change and generate a repository
containing something like this:
* -- * -- B -- O -- O -- O (0a26fb70b43f102d98220fef914a021413025d9d)
\
N -- N -- N (d4e10c0482b28a1c2fb7ef48e69a673d2c6ea15a)
When this happens we assume that you've already had alert emails for all
of the O revisions, and so we here report only the revisions in the N
branch from the common base, B.
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
commit d4e10c0482b28a1c2fb7ef48e69a673d2c6ea15a
Author: Ward Vandewege <ward at curii.com>
Date: Wed Feb 16 13:47:19 2022 -0500
18702: remove the old document about upgrading the configuration from
Arvados versions prior to 2.0, to improve in the install section.
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward at curii.com>
diff --git a/doc/_config.yml b/doc/_config.yml
index 844866c40..a2662200d 100644
--- a/doc/_config.yml
+++ b/doc/_config.yml
@@ -220,9 +220,8 @@ navbar:
- install/packages.html.textile.liquid
- Configuration:
- install/config.html.textile.liquid
- - admin/config-migration.html.textile.liquid
- - admin/config.html.textile.liquid
- admin/config-urls.html.textile.liquid
+ - admin/config.html.textile.liquid
- Maintenance and upgrading:
- admin/upgrading.html.textile.liquid
- install/maintenance-and-upgrading.html.textile.liquid
diff --git a/doc/admin/config-migration.html.textile.liquid b/doc/admin/config-migration.html.textile.liquid
deleted file mode 100644
index e36655a0f..000000000
--- a/doc/admin/config-migration.html.textile.liquid
+++ /dev/null
@@ -1,106 +0,0 @@
----
-layout: default
-navsection: installguide
-title: Migrating Configuration from v1.4 to v2.0
-...
-
-{% comment %}
-Copyright (C) The Arvados Authors. All rights reserved.
-
-SPDX-License-Identifier: CC-BY-SA-3.0
-{% endcomment %}
-
-{% include 'notebox_begin_warning' %}
-_New installations of Arvados 2.0+ can skip this section_
-{% include 'notebox_end' %}
-
-Arvados 2.0 migrates to a centralized configuration file for all components. The centralized Arvados configuration is @/etc/arvados/config.yml at . Components that support the new centralized configuration are listed below. During the migration period, legacy configuration files are still loaded and take precedence over the centralized configuration file.
-
-h2. API server
-
-The legacy API server configuration is stored in @config/application.yml@ and @config/database.yml at . After migration to @/etc/arvados/config.yml@, both of these files should be moved out of the way and/or deleted.
-
-Change to the API server directory and use the following commands:
-
-<pre>
-$ RAILS_ENV=production bin/rake config:migrate > config.yml
-$ cp config.yml /etc/arvados/config.yml
-</pre>
-
-This will print the contents of @config.yml@ after merging the legacy @application.yml@ and @database.yml@ into the existing systemwide @config.yml at . It may be redirected to a file and copied to @/etc/arvados/config.yml@ (it is safe to copy over, all configuration items from the existing @/etc/arvados/config.yml@ will be included in the migrated output).
-
-If you wish to update @config.yml@ configuration by hand, or check that everything has been migrated, use @config:diff@ to print configuration items that differ between @application.yml@ and the system @config.yml at .
-
-<pre>
-$ RAILS_ENV=production bin/rake config:diff
-</pre>
-
-This command will also report if no migrations are required.
-
-h2. Workbench
-
-The legacy workbench configuration is stored in @config/application.yml at . After migration to @/etc/arvados/config.yml@, this file should be moved out of the way and/or deleted.
-
-Change to the workbench server directory and use the following commands:
-
-<pre>
-$ RAILS_ENV=production bin/rake config:migrate > config.yml
-$ cp config.yml /etc/arvados/config.yml
-</pre>
-
-This will print the contents of @config.yml@ after merging the legacy @application.yml@ into the existing systemwide @config.yml at . It may be redirected to a file and copied to @/etc/arvados/config.yml@ (it is safe to copy over, all configuration items from the existing @/etc/arvados/config.yml@ will be included in the migrated output).
-
-If you wish to update @config.yml@ configuration by hand, or check that everything has been migrated, use @config:diff@ to print configuration items that differ between @application.yml@ and the system @config.yml at .
-
-<pre>
-$ RAILS_ENV=production bin/rake config:diff
-</pre>
-
-This command will also report if no migrations are required.
-
-h2. keepstore, keep-web, crunch-dispatch-slurm, arvados-ws, keepproxy, arv-git-httpd, keep-balance
-
-The legacy config for each component (loaded from @/etc/arvados/component/component.yml@ or a different location specified via the -legacy-component-config command line argument) takes precedence over the centralized config. After you migrate everything from the legacy config to the centralized config, you should delete @/etc/arvados/component/component.yml@ and/or stop using the corresponding -legacy-component-config argument.
-
-To migrate a component configuration, do this on each node that runs an Arvados service:
-
-# Ensure that the latest @config.yml@ is installed on the current node
-# Install @arvados-server@ using @apt-get@ or @yum at .
-# Run @arvados-server config-check@, review and apply the recommended changes to @/etc/arvados/config.yml@
-# After applying changes, re-run @arvados-server config-check@ again to check for additional warnings and recommendations.
-# When you are satisfied, delete the legacy config file, restart the service, and check its startup logs.
-# Copy the updated @config.yml@ file to your next node, and repeat the process there.
-# When you have a @config.yml@ file that includes all volumes on all keepstores, it is important to add a 'Rendezvous' parameter to the InternalURLs entries to make sure the old volume identifiers line up with the new config. If you don't do this, @keep-balance@ will want to shuffle all the existing data around to match the new volume order. The 'Rendezvous' value should be the last 15 characters of the keepstore's UUID in the old configuration. Here's an example:
-
-<notextile>
-<pre><code>Clusters:
- xxxxx:
- Services:
- Keepstore:
- InternalURLs:
- "http://keep1.xxxxx.arvadosapi.com:25107": {Rendezvous: "eim6eefaibesh3i"}
- "http://keep2.xxxxx.arvadosapi.com:25107": {Rendezvous: "yequoodalai7ahg"}
- "http://keep3.xxxxx.arvadosapi.com:25107": {Rendezvous: "eipheho6re1shou"}
- "http://keep4.xxxxx.arvadosapi.com:25107": {Rendezvous: "ahk7chahthae3oo"}
-</code></pre>
-</notextile>
-
-In this example, the keepstore with the name `keep1` had the uuid `xxxxx-bi6l4-eim6eefaibesh3i` in the old configuration.
-
-After migrating and removing all legacy config files, make sure the @/etc/arvados/config.yml@ file is identical across all system nodes -- API server, keepstore, etc. -- and restart all services to make sure they are using the latest configuration.
-
-h2. Cloud installations only: node manager
-
-Node manager is deprecated and replaced by @arvados-dispatch-cloud at . No automated config migration is available. Follow the instructions to "install the cloud dispatcher":../install/crunch2-cloud/install-dispatch-cloud.html
-
-*Only one dispatch process should be running at a time.* If you are migrating a system that currently runs Node manager and @crunch-dispatch-slurm@, it is safest to remove the @crunch-dispatch-slurm@ service entirely before installing @arvados-dispatch-cloud at .
-
-<notextile>
-<pre><code>~$ <span class="userinput">sudo systemctl --now disable crunch-dispatch-slurm</span>
-~$ <span class="userinput">sudo apt-get remove crunch-dispatch-slurm</span>
-</code></pre>
-</notextile>
-
-h2. arvados-controller, arvados-dispatch-cloud
-
-Already uses centralized config exclusively. No migration needed.
diff --git a/doc/admin/config.html.textile.liquid b/doc/admin/config.html.textile.liquid
index 745cd2853..a043d6d1c 100644
--- a/doc/admin/config.html.textile.liquid
+++ b/doc/admin/config.html.textile.liquid
@@ -12,8 +12,6 @@ SPDX-License-Identifier: CC-BY-SA-3.0
The Arvados configuration is stored at @/etc/arvados/config.yml@
-See "Migrating Configuration":config-migration.html for information about migrating from legacy component-specific configuration files.
-
{% codeblock as yaml %}
{% include 'config_default_yml' %}
{% endcodeblock %}
diff --git a/doc/admin/upgrading.html.textile.liquid b/doc/admin/upgrading.html.textile.liquid
index 7d7722326..00d8a69ab 100644
--- a/doc/admin/upgrading.html.textile.liquid
+++ b/doc/admin/upgrading.html.textile.liquid
@@ -234,7 +234,7 @@ Arvados 2.0 is a major upgrade, with many changes. Please read these upgrade no
h3. Migrating to centralized config.yml
-See "Migrating Configuration":config-migration.html for notes on migrating legacy per-component configuration files to the new centralized @/etc/arvados/config.yml at .
+See "Migrating Configuration":https://doc.arvados.org/v2.1/admin/config-migration.html for notes on migrating legacy per-component configuration files to the new centralized @/etc/arvados/config.yml at .
To ensure a smooth transition, the per-component config files continue to be read, and take precedence over the centralized configuration. Your cluster should continue to function after upgrade but before doing the full configuration migration. However, several services (keepstore, keep-web, keepproxy) require a minimal `/etc/arvados/config.yml` to start:
@@ -254,11 +254,11 @@ You can no longer specify types of keep services to balance via the @KeepService
You can no longer specify individual keep services to balance via the @config.KeepServiceList@ command line option or @KeepServiceList@ legacy config option. Instead, keep-balance will operate on all keepstore servers with @service_type:disk@ as reported by the @arv keep_service list@ command. If you are still using the legacy config, @KeepServiceList@ should be removed or keep-balance will produce an error.
-Please see the "config migration guide":{{site.baseurl}}/admin/config-migration.html and "keep-balance install guide":{{site.baseurl}}/install/install-keep-balance.html for more details.
+Please see the "config migration guide":https://doc.arvados.org/v2.1/admin/config-migration.html and "keep-balance install guide":{{site.baseurl}}/install/install-keep-balance.html for more details.
h3. Arv-git-httpd configuration migration
-(feature "#14712":https://dev.arvados.org/issues/14712 ) The arv-git-httpd package can now be configured using the centralized configuration file at @/etc/arvados/config.yml at . Configuration via individual command line arguments is no longer available. Please see "arv-git-httpd's config migration guide":{{site.baseurl}}/admin/config-migration.html#arv-git-httpd for more details.
+(feature "#14712":https://dev.arvados.org/issues/14712 ) The arv-git-httpd package can now be configured using the centralized configuration file at @/etc/arvados/config.yml at . Configuration via individual command line arguments is no longer available. Please see "arv-git-httpd's config migration guide":https://doc.arvados.org/v2.1/admin/config-migration.html#arv-git-httpd for more details.
h3. Keepstore and keep-web configuration migration
@@ -266,11 +266,11 @@ keepstore and keep-web no longer support configuration via (previously deprecate
keep-web now supports the legacy @keep-web.yml@ config format (used by Arvados 1.4) and the new cluster config file format. Please check "keep-web's install guide":{{site.baseurl}}/install/install-keep-web.html for more details.
-keepstore now supports the legacy @keepstore.yml@ config format (used by Arvados 1.4) and the new cluster config file format. Please check the "keepstore config migration notes":{{site.baseurl}}/admin/config-migration.html#keepstore and "keepstore install guide":{{site.baseurl}}/install/install-keepstore.html for more details.
+keepstore now supports the legacy @keepstore.yml@ config format (used by Arvados 1.4) and the new cluster config file format. Please check the "keepstore config migration notes":https://doc.arvados.org/v2.1/admin/config-migration.html#keepstore and "keepstore install guide":{{site.baseurl}}/install/install-keepstore.html for more details.
h3. Keepproxy configuration migration
-(feature "#14715":https://dev.arvados.org/issues/14715 ) Keepproxy can now be configured using the centralized config at @/etc/arvados/config.yml at . Configuration via individual command line arguments is no longer available and the @DisableGet@, @DisablePut@, and @PIDFile@ configuration options are no longer supported. If you are still using the legacy config and @DisableGet@ or @DisablePut@ are set to true or @PIDFile@ has a value, keepproxy will produce an error and fail to start. Please see "keepproxy's config migration guide":{{site.baseurl}}/admin/config-migration.html#keepproxy for more details.
+(feature "#14715":https://dev.arvados.org/issues/14715 ) Keepproxy can now be configured using the centralized config at @/etc/arvados/config.yml at . Configuration via individual command line arguments is no longer available and the @DisableGet@, @DisablePut@, and @PIDFile@ configuration options are no longer supported. If you are still using the legacy config and @DisableGet@ or @DisablePut@ are set to true or @PIDFile@ has a value, keepproxy will produce an error and fail to start. Please see "keepproxy's config migration guide":https://doc.arvados.org/v2.1/admin/config-migration.html#keepproxy for more details.
h3. Delete "keep_services" records
@@ -467,7 +467,7 @@ As part of story "#9945":https://dev.arvados.org/issues/9945, it was discovered
h3. New configuration
-Arvados is migrating to a centralized configuration file for all components. During the migration, legacy configuration files will continue to be loaded. See "Migrating Configuration":config-migration.html for details.
+Arvados is migrating to a centralized configuration file for all components. During the migration, legacy configuration files will continue to be loaded. See "Migrating Configuration":https://doc.arvados.org/v2.0/config-migration.html for details.
h2(#v1_3_3). v1.3.3 (2019-05-14)
commit 1fc3e6081d1d016184f328af84c1aaf330920be0
Author: Ward Vandewege <ward at curii.com>
Date: Wed Feb 16 10:39:41 2022 -0500
18702: add documentation for maintaining and upgrading Arvados. Make it
clear that the Salt-based installer is only intended for
installation.
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward at curii.com>
diff --git a/doc/_config.yml b/doc/_config.yml
index eb0f173df..844866c40 100644
--- a/doc/_config.yml
+++ b/doc/_config.yml
@@ -218,12 +218,14 @@ navbar:
- Manual installation:
- install/install-manual-prerequisites.html.textile.liquid
- install/packages.html.textile.liquid
- - admin/upgrading.html.textile.liquid
- Configuration:
- install/config.html.textile.liquid
- admin/config-migration.html.textile.liquid
- admin/config.html.textile.liquid
- admin/config-urls.html.textile.liquid
+ - Maintenance and upgrading:
+ - admin/upgrading.html.textile.liquid
+ - install/maintenance-and-upgrading.html.textile.liquid
- Core:
- install/install-api-server.html.textile.liquid
- Keep:
diff --git a/doc/admin/upgrading.html.textile.liquid b/doc/admin/upgrading.html.textile.liquid
index 2bd41829b..7d7722326 100644
--- a/doc/admin/upgrading.html.textile.liquid
+++ b/doc/admin/upgrading.html.textile.liquid
@@ -1,7 +1,7 @@
---
layout: default
navsection: installguide
-title: "Upgrading Arvados and Release notes"
+title: "Arvados upgrade notes"
...
{% comment %}
diff --git a/doc/install/maintenance-and-upgrading.html.textile.liquid b/doc/install/maintenance-and-upgrading.html.textile.liquid
new file mode 100644
index 000000000..8cfbbe8dc
--- /dev/null
+++ b/doc/install/maintenance-and-upgrading.html.textile.liquid
@@ -0,0 +1,60 @@
+---
+layout: default
+navsection: installguide
+title: Maintenance and upgrading
+...
+{% comment %}
+Copyright (C) The Arvados Authors. All rights reserved.
+
+SPDX-License-Identifier: CC-BY-SA-3.0
+{% endcomment %}
+
+# "Commercial support":#commercial_support
+# "Maintaining Arvados":#maintaining
+## "Modification of the config.yml file":#configuration
+## "Distributing the configuration file":#distribution
+## "Restart the services affected by the change":#restart
+# "Upgrading Arvados":#upgrading
+
+h2(#commercial_support). Commercial support
+
+Arvados is "100% open source software":{{site.baseurl}}/user/copying/copying.html. Anyone can download, install, maintain and upgrade it. However, if this is not something you want to spend your time and energy doing, "Curii Corporation":https://curii.com provides managed Arvados installations as well as commercial support for Arvados. Please contact "info at curii.com":mailto:info at curii.com for more information.
+
+If you'd prefer to do things yourself, a few starting points for a maintaining and upgrading Arvados can be found below.
+
+h2(#maintaining). Maintaining Arvados
+
+After Arvados is installed, periodic configuration changes may be required to adapt the software to your needs. Arvados uses a unified configuration file, which is normally found at @/etc/arvados/config.yml at .
+
+Making a configuration change to Arvados typically involves three steps:
+
+* modification of the @config.yml@ file
+* distribution of the modified file to the machines in the cluster
+* restarting of the services affected by the change
+
+h3(#configchange). Modification of the @config.yml@ file
+
+Consult the "configuration reference":{{site.baseurl}}/admin/config.html or another part of the documentation to identify the change to be made.
+
+Preserve a copy of your existing configuration file as a backup, and make the desired modification.
+
+Run @arvados-server config-check@ to make sure the configuration file has no errors and no warnings.
+
+h3(#distribution). Distribute the configuration file
+
+We recommend to keep the @config.yml@ file in sync between all the Arvados system nodes, to avoid issues with services running on different versions of the configuration.
+
+Distribution of the configuration file can be done in many ways, e.g. scp, configuration management software, etc.
+
+h3(#restart). Restart the services affected by the change
+
+If you know which Arvados service uses the specific configuration that was modified, restart those services. When in doubt, restart all Arvados system services.
+
+h2(#upgrading). Upgrading Arvados
+
+Upgrading Arvados typically involves the following steps:
+
+* consult the "upgrade notes":{{site.baseurl}}/admin/upgrading.html and the "release notes":https://arvados.org/releases/ for the release you want to upgrade to
+* rebuild and deploy the "compute node image":crunch2-cloud/install-compute-node.html (cloud only)
+* update the configuration file for the new release (see "Maintaining Arvados":#maintaining above)
+* upgrade the Arvados components on the system nodes
diff --git a/doc/install/salt-multi-host.html.textile.liquid b/doc/install/salt-multi-host.html.textile.liquid
index c3d6a92b5..d06a190ad 100644
--- a/doc/install/salt-multi-host.html.textile.liquid
+++ b/doc/install/salt-multi-host.html.textile.liquid
@@ -20,8 +20,7 @@ SPDX-License-Identifier: CC-BY-SA-3.0
# "Run the provision.sh script":#run_provision_script
# "Initial user and login":#initial_user
# "Test the installed cluster running a simple workflow":#test_install
-
-
+# "After the installation":#post_install
h2(#introduction). Introduction
@@ -290,3 +289,9 @@ INFO Final output collection d6c69a88147dde9d52a418d50ef788df+123
INFO Final process status is success
</code></pre>
</notextile>
+
+h2(#post_install). After the installation
+
+Once the installation is complete, it is recommended to keep a copy of your local configuration files. Committing them to version control is a good idea.
+
+Re-running the Salt-based installer is not recommended for maintaining and upgrading Arvados, please see "Maintenance and upgrading":maintenance-and-upgrading.html for more information.
diff --git a/doc/install/salt-single-host.html.textile.liquid b/doc/install/salt-single-host.html.textile.liquid
index ce70a30d4..a3fa63c25 100644
--- a/doc/install/salt-single-host.html.textile.liquid
+++ b/doc/install/salt-single-host.html.textile.liquid
@@ -20,6 +20,7 @@ SPDX-License-Identifier: CC-BY-SA-3.0
## "DNS configuration (single host / multiple hostnames)":#single_host_multiple_hostnames_dns_configuration
# "Initial user and login":#initial_user
# "Test the installed cluster running a simple workflow":#test_install
+# "After the installation":#post_install
h2(#single_host). Single host install using the provision.sh script
@@ -266,3 +267,9 @@ INFO Final output collection d6c69a88147dde9d52a418d50ef788df+123
INFO Final process status is success
</code></pre>
</notextile>
+
+h2(#post_install). After the installation
+
+Once the installation is complete, it is recommended to keep a copy of your local configuration files. Committing them to version control is a good idea.
+
+Re-running the Salt-based installer is not recommended for maintaining and upgrading Arvados, please see "Maintenance and upgrading":maintenance-and-upgrading.html for more information.
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list