[arvados] created: 2.7.0-6016-g2f3e76d669

[git repository hosting]

        at  2f3e76d66966a5d565af67b5ed2eb8c122a0fd97 (commit)


commit 2f3e76d66966a5d565af67b5ed2eb8c122a0fd97
Author: Brett Smith <brett.smith at curii.com>
Date:   Tue Feb 13 10:36:54 2024 -0500

    21453: Install Python package virtualenvs under /usr/lib
    
    Installing under /usr/share/python3 is a remnant from when we built
    library packages that integrated with the distribution. Now that we
    install an entire virtualenv that's tied to a specific minor version of
    Python, installing under /usr/lib makes more sense.
    
    Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>

diff --git a/build/run-library.sh b/build/run-library.sh
index 650fdde031..a395db8b77 100755
--- a/build/run-library.sh
+++ b/build/run-library.sh
@@ -702,7 +702,7 @@ fpm_build_virtualenv_worker () {
   cd $WORKSPACE/$PKG_DIR
 
   rm -rf dist/*
-  local venv_dir="dist/build/usr/share/python$PYTHON3_VERSION/dist/$PYTHON_PKG"
+  local venv_dir="dist/build/usr/lib/$PYTHON_PKG"
   echo "Creating virtualenv..."
   if ! "$PYTHON3_EXECUTABLE" -m venv "$venv_dir"; then
     printf "Error, unable to run\n  %s -m venv %s\n" "$PYTHON3_EXECUTABLE" "$venv_dir"
@@ -865,17 +865,18 @@ fpm_build_virtualenv_worker () {
   # make sure the systemd service file ends up in the right place
   # used by arvados-docker-cleaner
   if [[ -e "${systemd_unit}" ]]; then
-    COMMAND_ARR+=("usr/share/python$PYTHON3_VERSION/dist/$PKG/share/doc/$PKG/$PKG.service=/lib/systemd/system/$PKG.service")
+    COMMAND_ARR+=("$sys_venv_dir/share/doc/$PKG/$PKG.service=/lib/systemd/system/$PKG.service")
   fi
 
   COMMAND_ARR+=("${fpm_args[@]}")
 
-  # Make sure to install all our package binaries in /usr/bin.
-  # We have to walk $WORKSPACE/$PKG_DIR/bin rather than
-  # $WORKSPACE/build/usr/share/$python/dist/$PYTHON_PKG/bin/ to get the list
-  # because the latter also includes all the python binaries for the virtualenv.
-  # We have to take the copies of our binaries from the latter directory, though,
-  # because those are the ones we rewrote the shebang line of, above.
+  # Make sure to install all our package binaries in /usr/bin. We have to
+  # walk $WORKSPACE/$PKG_DIR/bin rather than $venv_dir/bin to get the list
+  # because the latter also includes scripts installed by all the
+  # dependencies in the virtualenv, which may conflict with other
+  # packages. We have to take the copies of our binaries from the latter
+  # directory, though, because those are the ones we rewrote the shebang
+  # line of, above.
   if [[ -e "$WORKSPACE/$PKG_DIR/bin" ]]; then
     for binary in `ls $WORKSPACE/$PKG_DIR/bin`; do
       COMMAND_ARR+=("$sys_venv_dir/bin/$binary=/usr/bin/")
diff --git a/doc/admin/upgrading.html.textile.liquid b/doc/admin/upgrading.html.textile.liquid
index c802098225..64a113b6f8 100644
--- a/doc/admin/upgrading.html.textile.liquid
+++ b/doc/admin/upgrading.html.textile.liquid
@@ -32,6 +32,30 @@ h2(#main). development main
 
 "previous: Upgrading to 2.7.1":#v2_7_1
 
+h3. Virtual environments inside distribution Python packages have moved
+
+The distribution packages that we publish for Python packages include an entire virtualenv with all required libraries. In Arvados 3.0 these virtualenvs have moved from @/usr/share/python3/dist/PACKAGE_NAME@ to @/usr/lib/PACKAGE_NAME@ to prevent conflicts with distribution packages and better conform to filesystem standards.
+
+If you only run the executables installed by these packages, you don't need to change anything. Those are still installed under @/usr/bin@ and will use the new location when you upgrade. If you have written your own scripts or tools that rely on these virtualenvs, you may need to update those with the new location. For example, if you have a shell script that activates the virtualenv by running:
+
+<pre><code class="shell">source /usr/share/python3/dist/python3-arvados-python-client/bin/activate</code></pre>
+
+You must update it to:
+
+<notextile>
+<pre><code class="shell">source <span class="userinput">/usr/lib/python3-arvados-python-client</span>/bin/activate</code></pre>
+</notextile>
+
+If you have a Python script with this shebang line:
+
+<pre><code class="shell">#!/usr/share/python3/dist/python3-arvados-python-client/bin/python</code></pre>
+
+You must update it to:
+
+<notextile>
+<pre><code class="shell">#!<span class="userinput">/usr/lib/python3-arvados-python-client</span>/bin/python</code></pre>
+</notextile>
+
 h3. WebDAV service uses @/var/cache@ for file content
 
 @keep-web@ now stores copies of recently accessed data blocks in @/var/cache/arvados/keep@ instead of in memory. That directory will be created automatically. The default cache size is 10% of the filesystem size. Use the new @Collections.WebDAVCache.DiskCacheSize@ config to specify a different percentage or an absolute size.

-----------------------------------------------------------------------


hooks/post-receive
--