[arvados] created: 2.7.0-6018-g8c33c582e6
git repository hosting
git at public.arvados.org
Tue Feb 13 15:45:17 UTC 2024
at 8c33c582e6fa415dbdca5204f5926fe09de34e3b (commit)
commit 8c33c582e6fa415dbdca5204f5926fe09de34e3b
Author: Brett Smith <brett.smith at curii.com>
Date: Tue Feb 13 10:44:45 2024 -0500
21453: Update minimum Python version in SDK documentation
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>
diff --git a/doc/sdk/python/sdk-python.html.textile.liquid b/doc/sdk/python/sdk-python.html.textile.liquid
index 9d88b5e1dc..4a6ba029fe 100644
--- a/doc/sdk/python/sdk-python.html.textile.liquid
+++ b/doc/sdk/python/sdk-python.html.textile.liquid
@@ -19,7 +19,7 @@ If you are logged in to an Arvados VM, the Python SDK should be installed.
To use the Python SDK elsewhere, you can install it "from an Arvados distribution package":#package-install or "from PyPI using pip":#pip-install.
{% include 'notebox_begin_warning' %}
-As of Arvados 2.2, the Python SDK requires Python 3.6+. The last version to support Python 2.7 is Arvados 2.0.4.
+As of Arvados 3.0, the Python SDK requires Python 3.8+.
{% include 'notebox_end' %}
h2(#package-install). Install from a distribution package
commit 54e625fd4fc61268d9a291b378bec3015ab35604
Author: Brett Smith <brett.smith at curii.com>
Date: Tue Feb 13 10:43:11 2024 -0500
21453: Clean up documentation note for unsupported distribution
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>
diff --git a/doc/sdk/python/sdk-python.html.textile.liquid b/doc/sdk/python/sdk-python.html.textile.liquid
index e97b768bad..9d88b5e1dc 100644
--- a/doc/sdk/python/sdk-python.html.textile.liquid
+++ b/doc/sdk/python/sdk-python.html.textile.liquid
@@ -32,10 +32,6 @@ First, configure the "Arvados package repositories":../../install/packages.html
{% include 'install_packages' %}
-{% include 'notebox_begin_warning' %}
-If you are on Ubuntu 18.04, please note that the Arvados packages that use Python depend on the python-3.8 package. This means they are installed under @/usr/share/python3.8@, not @/usr/share/python3 at . You will need to update the commands below accordingly.
-{% include 'notebox_end' %}
-
The package includes a virtualenv, which means the correct Python environment needs to be loaded before the Arvados SDK can be imported. You can test the installation by doing that, then creating a client object. Ensure your "@ARVADOS_API_HOST@ and @ARVADOS_API_TOKEN@ credentials are set up correctly":{{site.baseurl}}/user/reference/api-tokens.html. Then you should be able to run the following without any errors:
<notextile>
commit 554f319d6a198a6e0907203f4398aa34bef45c02
Author: Brett Smith <brett.smith at curii.com>
Date: Tue Feb 13 10:36:54 2024 -0500
21453: Install Python package virtualenvs under /usr/lib
Installing under /usr/share/python3 is a remnant from when we built
library packages that integrated with the distribution. Now that we
install an entire virtualenv that's tied to a specific minor version of
Python, installing under /usr/lib makes more sense.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>
diff --git a/build/run-library.sh b/build/run-library.sh
index 650fdde031..a395db8b77 100755
--- a/build/run-library.sh
+++ b/build/run-library.sh
@@ -702,7 +702,7 @@ fpm_build_virtualenv_worker () {
cd $WORKSPACE/$PKG_DIR
rm -rf dist/*
- local venv_dir="dist/build/usr/share/python$PYTHON3_VERSION/dist/$PYTHON_PKG"
+ local venv_dir="dist/build/usr/lib/$PYTHON_PKG"
echo "Creating virtualenv..."
if ! "$PYTHON3_EXECUTABLE" -m venv "$venv_dir"; then
printf "Error, unable to run\n %s -m venv %s\n" "$PYTHON3_EXECUTABLE" "$venv_dir"
@@ -865,17 +865,18 @@ fpm_build_virtualenv_worker () {
# make sure the systemd service file ends up in the right place
# used by arvados-docker-cleaner
if [[ -e "${systemd_unit}" ]]; then
- COMMAND_ARR+=("usr/share/python$PYTHON3_VERSION/dist/$PKG/share/doc/$PKG/$PKG.service=/lib/systemd/system/$PKG.service")
+ COMMAND_ARR+=("$sys_venv_dir/share/doc/$PKG/$PKG.service=/lib/systemd/system/$PKG.service")
fi
COMMAND_ARR+=("${fpm_args[@]}")
- # Make sure to install all our package binaries in /usr/bin.
- # We have to walk $WORKSPACE/$PKG_DIR/bin rather than
- # $WORKSPACE/build/usr/share/$python/dist/$PYTHON_PKG/bin/ to get the list
- # because the latter also includes all the python binaries for the virtualenv.
- # We have to take the copies of our binaries from the latter directory, though,
- # because those are the ones we rewrote the shebang line of, above.
+ # Make sure to install all our package binaries in /usr/bin. We have to
+ # walk $WORKSPACE/$PKG_DIR/bin rather than $venv_dir/bin to get the list
+ # because the latter also includes scripts installed by all the
+ # dependencies in the virtualenv, which may conflict with other
+ # packages. We have to take the copies of our binaries from the latter
+ # directory, though, because those are the ones we rewrote the shebang
+ # line of, above.
if [[ -e "$WORKSPACE/$PKG_DIR/bin" ]]; then
for binary in `ls $WORKSPACE/$PKG_DIR/bin`; do
COMMAND_ARR+=("$sys_venv_dir/bin/$binary=/usr/bin/")
diff --git a/doc/admin/upgrading.html.textile.liquid b/doc/admin/upgrading.html.textile.liquid
index c802098225..64a113b6f8 100644
--- a/doc/admin/upgrading.html.textile.liquid
+++ b/doc/admin/upgrading.html.textile.liquid
@@ -32,6 +32,30 @@ h2(#main). development main
"previous: Upgrading to 2.7.1":#v2_7_1
+h3. Virtual environments inside distribution Python packages have moved
+
+The distribution packages that we publish for Python packages include an entire virtualenv with all required libraries. In Arvados 3.0 these virtualenvs have moved from @/usr/share/python3/dist/PACKAGE_NAME@ to @/usr/lib/PACKAGE_NAME@ to prevent conflicts with distribution packages and better conform to filesystem standards.
+
+If you only run the executables installed by these packages, you don't need to change anything. Those are still installed under @/usr/bin@ and will use the new location when you upgrade. If you have written your own scripts or tools that rely on these virtualenvs, you may need to update those with the new location. For example, if you have a shell script that activates the virtualenv by running:
+
+<pre><code class="shell">source /usr/share/python3/dist/python3-arvados-python-client/bin/activate</code></pre>
+
+You must update it to:
+
+<notextile>
+<pre><code class="shell">source <span class="userinput">/usr/lib/python3-arvados-python-client</span>/bin/activate</code></pre>
+</notextile>
+
+If you have a Python script with this shebang line:
+
+<pre><code class="shell">#!/usr/share/python3/dist/python3-arvados-python-client/bin/python</code></pre>
+
+You must update it to:
+
+<notextile>
+<pre><code class="shell">#!<span class="userinput">/usr/lib/python3-arvados-python-client</span>/bin/python</code></pre>
+</notextile>
+
h3. WebDAV service uses @/var/cache@ for file content
@keep-web@ now stores copies of recently accessed data blocks in @/var/cache/arvados/keep@ instead of in memory. That directory will be created automatically. The default cache size is 10% of the filesystem size. Use the new @Collections.WebDAVCache.DiskCacheSize@ config to specify a different percentage or an absolute size.
diff --git a/doc/sdk/python/sdk-python.html.textile.liquid b/doc/sdk/python/sdk-python.html.textile.liquid
index fd1ff36c5e..e97b768bad 100644
--- a/doc/sdk/python/sdk-python.html.textile.liquid
+++ b/doc/sdk/python/sdk-python.html.textile.liquid
@@ -39,7 +39,7 @@ If you are on Ubuntu 18.04, please note that the Arvados packages that use Pytho
The package includes a virtualenv, which means the correct Python environment needs to be loaded before the Arvados SDK can be imported. You can test the installation by doing that, then creating a client object. Ensure your "@ARVADOS_API_HOST@ and @ARVADOS_API_TOKEN@ credentials are set up correctly":{{site.baseurl}}/user/reference/api-tokens.html. Then you should be able to run the following without any errors:
<notextile>
-<pre>~$ <code class="userinput">source /usr/share/python3/dist/python3-arvados-python-client/bin/activate</code>
+<pre>~$ <code class="userinput">source /usr/lib/python3-arvados-python-client/bin/activate</code>
(python-arvados-python-client) ~$ <code class="userinput">python</code>
Python 3.7.3 (default, Jul 25 2020, 13:03:44)
[GCC 8.3.0] on linux
@@ -53,7 +53,7 @@ Type "help", "copyright", "credits" or "license" for more information.
Alternatively, you can run the Python executable inside the @virtualenv@ directly:
<notextile>
-<pre>~$ <code class="userinput">/usr/share/python3/dist/python3-arvados-python-client/bin/python</code>
+<pre>~$ <code class="userinput">/usr/lib/python3-arvados-python-client/bin/python</code>
Python 3.7.3 (default, Jul 25 2020, 13:03:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list