[ARVADOS] created: 2.1.0-2036-gfd04088f4
Git user
git at public.arvados.org
Mon Mar 7 18:35:19 UTC 2022
at fd04088f4cd977b911818789781c13a61ff00ecc (commit)
commit fd04088f4cd977b911818789781c13a61ff00ecc
Author: Ward Vandewege <ward at curii.com>
Date: Mon Mar 7 13:21:45 2022 -0500
18766: because our Ubuntu 18.04 Python packages now depend on
python-3.8, the path for the Python executables in our packages
has changed. Put a note in the documentation, make the package
testing script able to handle that, and for good measure, also
make sure the arvados/jobs Dockerfile can handle it.
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward at curii.com>
diff --git a/build/package-testing/test-package-python3-arvados-python-client.sh b/build/package-testing/test-package-python3-arvados-python-client.sh
index 69f728c10..1e294fe0a 100755
--- a/build/package-testing/test-package-python3-arvados-python-client.sh
+++ b/build/package-testing/test-package-python3-arvados-python-client.sh
@@ -7,7 +7,9 @@ set -e
arv-put --version >/dev/null
-/usr/share/python3/dist/python3-arvados-python-client/bin/python3 << EOF
+PYTHON=`ls /usr/share/python3*/dist/python3-arvados-python-client/bin/python3 |head -n1`
+
+$PYTHON << EOF
import arvados
print("Successfully imported arvados")
EOF
diff --git a/doc/admin/upgrading.html.textile.liquid b/doc/admin/upgrading.html.textile.liquid
index 943bc3e0e..99d4371c9 100644
--- a/doc/admin/upgrading.html.textile.liquid
+++ b/doc/admin/upgrading.html.textile.liquid
@@ -32,6 +32,12 @@ h2(#main). development main (as of 2022-02-10)
"previous: Upgrading to 2.3.0":#v2_3_0
+h3. Ubuntu 18.04 Arvados Python packages now depend on python-3.8
+
+Ubuntu 18.04 ships with Python 3.6 as the default version of Python 3. Ubuntu also ships a version of Python 3.8, and the Arvados Python packages (@python3-arvados-cwl-runner@, @python3-arvados-fuse@, @python3-arvados-python-client@, @python3-arvados-user-activity@ and @python3-crunchstat-summary@) now depend on the @python-3.8@ system package.
+
+This means that they are now installed under @/usr/share/python3.8@ (before, the path was @/usr/share/python3@). If you rely on the @python3@ executable from the packages (e.g. to load a virtualenv), you may need to update the path to that executable.
+
h3. Anonymous token changes
The anonymous token configured in @Users.AnonymousUserToken@ must now be 32 characters or longer. This was already the suggestion in the documentation, now it is enforced. The @script/get_anonymous_user_token.rb@ script that was needed to register the anonymous user token in the database has been removed. Registration of the anonymous token is no longer necessary. If the anonymous token in @config.yml@ is specified as a full V2 token, that will now generate a warning - it should be updated to list just the secret (i.e. the part after the last forward slash).
diff --git a/doc/sdk/python/sdk-python.html.textile.liquid b/doc/sdk/python/sdk-python.html.textile.liquid
index 435f70e7b..56f032804 100644
--- a/doc/sdk/python/sdk-python.html.textile.liquid
+++ b/doc/sdk/python/sdk-python.html.textile.liquid
@@ -65,6 +65,10 @@ Type "help", "copyright", "credits" or "license" for more information.
If you installed from a distribution package (option 2): the package includes a virtualenv, which means the correct Python environment needs to be loaded before the Arvados SDK can be imported. This can be done by activating the virtualenv first:
+{% include 'notebox_begin_warning' %}
+If you are on Ubuntu 18.04, please note that the Arvados packages that use Python depend on the python-3.8 package. This means they are installed under @/usr/share/python3.8@, not @/usr/share/python3 at . You will need to update the commands below accordingly.
+{% include 'notebox_end' %}
+
<notextile>
<pre>~$ <code class="userinput">source /usr/share/python3/dist/python3-arvados-python-client/bin/activate</code>
(python-arvados-python-client) ~$ <code class="userinput">python</code>
diff --git a/docker/jobs/Dockerfile b/docker/jobs/Dockerfile
index 8da58a682..1b75e1342 100644
--- a/docker/jobs/Dockerfile
+++ b/docker/jobs/Dockerfile
@@ -26,8 +26,8 @@ RUN apt-get update -q
RUN apt-get install -yq --no-install-recommends python3-arvados-cwl-runner=$cwl_runner_version
# use the Python executable from the python-arvados-cwl-runner package
-RUN rm -f /usr/bin/python && ln -s /usr/share/python3/dist/python3-arvados-cwl-runner/bin/python /usr/bin/python
-RUN rm -f /usr/bin/python3 && ln -s /usr/share/python3/dist/python3-arvados-cwl-runner/bin/python /usr/bin/python3
+RUN PYTHON=`ls /usr/share/python3*/dist/python3-arvados-cwl-runner/bin/python|head -n1` && rm -f /usr/bin/python && ln -s $PYTHON /usr/bin/python
+RUN PYTHON3=`ls /usr/share/python3*/dist/python3-arvados-cwl-runner/bin/python3|head -n1` && rm -f /usr/bin/python3 && ln -s $PYTHON3 /usr/bin/python3
# Install dependencies and set up system.
RUN /usr/sbin/adduser --disabled-password \
commit 55e6de311c7a4bae272b5a64b40b82f3798b0a87
Author: Ward Vandewege <ward at curii.com>
Date: Mon Mar 7 12:31:45 2022 -0500
18766: add --force-test and --force-build options to the
run-build-test-packages-one-target.sh script.
Arvados-DCO-1.1-Signed-off-by: Ward Vandewege <ward at curii.com>
diff --git a/build/run-build-test-packages-one-target.sh b/build/run-build-test-packages-one-target.sh
index e36c4e88c..aa4acb6a2 100755
--- a/build/run-build-test-packages-one-target.sh
+++ b/build/run-build-test-packages-one-target.sh
@@ -15,6 +15,11 @@ Syntax:
Build only a specific package (or ONLY_BUILD from environment)
--arch <arch>
Build a specific architecture (or ARCH from environment, defaults to native architecture)
+--force-build
+ Build even if the package exists upstream or if it has already been
+ built locally
+--force-test
+ Test even if there is no new untested package
--upload
If the build and test steps are successful, upload the packages
to a remote apt repository (default: false)
@@ -48,7 +53,7 @@ if ! [[ -d "$WORKSPACE" ]]; then
fi
PARSEDOPTS=$(getopt --name "$0" --longoptions \
- help,debug,upload,rc,target:,only-build:,arch:,build-version: \
+ help,debug,upload,rc,target:,force-test,only-build:,force-build,arch:,build-version: \
-- "" "$@")
if [ $? -ne 0 ]; then
exit 1
@@ -72,6 +77,12 @@ while [ $# -gt 0 ]; do
--target)
TARGET="$2"; shift
;;
+ --force-test)
+ FORCE_TEST=1
+ ;;
+ --force-build)
+ FORCE_BUILD=1
+ ;;
--only-build)
ONLY_BUILD="$2"; shift
;;
@@ -107,6 +118,14 @@ if [[ -n "$ONLY_BUILD" ]]; then
build_args+=(--only-build "$ONLY_BUILD")
fi
+if [[ -n "$FORCE_BUILD" ]]; then
+ build_args+=(--force-build)
+fi
+
+if [[ -n "$FORCE_TEST" ]]; then
+ build_args+=(--force-test)
+fi
+
if [[ -n "$ARCH" ]]; then
build_args+=(--arch "$ARCH")
fi
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list