[arvados] created: 2.1.0-3079-g2168b29d2
git repository hosting
git at public.arvados.org
Fri Nov 25 17:03:13 UTC 2022
at 2168b29d23424d624bcc27170cd619ed547aca79 (commit)
commit 2168b29d23424d624bcc27170cd619ed547aca79
Author: Brett Smith <brett.smith at curii.com>
Date: Fri Nov 25 11:56:44 2022 -0500
19791: Expand Python SDK examples into API client overview
This change takes the skeleton of the existing examples page and expands
it into a fuller overview of the API client provided by the Python
SDK. It provides more explanation of how the client works; how it
corresponds to the Arvados API; more concrete examples of real tasks you
might actually want to do with the client; and a few different examples
to demonstrate how you might use different client code patterns for
different situations.
Arvados-DCO-1.1-Signed-off-by: Brett Smith <brett.smith at curii.com>
diff --git a/doc/_config.yml b/doc/_config.yml
index 5c8d77382..498434b88 100644
--- a/doc/_config.yml
+++ b/doc/_config.yml
@@ -78,7 +78,7 @@ navbar:
- sdk/index.html.textile.liquid
- Python:
- sdk/python/sdk-python.html.textile.liquid
- - sdk/python/example.html.textile.liquid
+ - sdk/python/api-client.html.textile.liquid
- sdk/python/python.html.textile.liquid
- sdk/python/arvados-fuse.html.textile.liquid
- sdk/python/arvados-cwl-runner.html.textile.liquid
diff --git a/doc/sdk/python/api-client.html.textile.liquid b/doc/sdk/python/api-client.html.textile.liquid
new file mode 100644
index 000000000..7d13a9467
--- /dev/null
+++ b/doc/sdk/python/api-client.html.textile.liquid
@@ -0,0 +1,193 @@
+---
+layout: default
+navsection: sdk
+navmenu: Python
+title: Arvados API Client
+...
+{% comment %}
+Copyright (C) The Arvados Authors. All rights reserved.
+
+SPDX-License-Identifier: CC-BY-SA-3.0
+{% endcomment %}
+
+The Arvados Python SDK provides a complete client interface to the "Arvados API":{{site.baseurl}}/api/index.html. You can use this client interface directly to send requests to your Arvados API server, and many of the higher-level interfaces in the Python SDK accept a client object in their constructor for their use. Any Arvados software you write in Python will likely use these client objects.
+
+This document explains how to instantiate the client object, and how its methods map to the full "Arvados API":{{site.baseurl}}/api/index.html. Refer to the API documentation for full details about all available resources and methods. The rest of the Python SDK documentation after this covers the higher-level interfaces it provides.
+
+h2. Initializing the API client
+
+In the simplest case, you can import the @arvados@ module and call its @api@ method with an API version number:
+
+{% codeblock as python %}
+import arvados
+arv_client = arvados.api('v1')
+{% endcodeblock %}
+
+This will connect to the Arvados API server using the @ARVADOS_API_HOST@, @ARVADOS_API_TOKEN@, and @ARVADOS_API_HOST_INSECURE@ settings from environment variables or @~/.config/arvados/settings.conf at . You can alternatively pass these settings as arguments:
+
+{% codeblock as python %}
+import arvados
+arv_client = arvados.api(
+ 'v1',
+ host='api.arvados.example.com',
+ token='ExampleToken',
+ insecure=False,
+)
+{% endcodeblock %}
+
+Either way, you can now use the @arv_client@ object to send requests to the Arvados API server you specified, using the configured token. The client object queries the API server for its supported API version and methods, so this client object will always support the same API the server does, even when there is a version mismatch between it and the Python SDK.
+
+h2. Resources, methods, and requests
+
+The API client has a method that corresponds to each "type of resource supported by the Arvados API server":{{site.baseurl}}/api/ (listed in the documentation sidebar). You call these methods without any arguments. They return a resource object you use to call a method on that resource type.
+
+Each resource object has a method that corresponds to each API method supported by that resource type. You call these methods with the keyword arguments and values documented in the API reference. They return an API request object.
+
+Each API request object has an @execute()@ method. You may pass a @num_retries@ integer argument to retry the operation that many times, with exponential back-off, in case of temporary errors like network problems. If it ultimately succeeds, it returns the kind of object documented in the API reference for that method. Usually that's a dictionary with details about the object you requested. If there's a problem, it raises an exception.
+
+Putting it all together, basic API requests usually look like:
+
+{% codeblock as python %}
+arv_object = arv_client.resource_type().api_method(
+ argument=...,
+ other_argument=...,
+).execute(num_retries=3)
+{% endcodeblock %}
+
+The following sections detail how to call "common resource methods in the API":{{site.baseurl}}/api/methods.html with more concrete examples. Additional methods may be available on specific resource types.
+
+h2. get method
+
+To fetch a single Arvados object, call the @get@ method of the resource type. You must pass a @uuid@ argument string that identifies the object to fetch. The method returns a dictionary with the object's fields.
+
+{% codeblock as python %}
+# Get a workflow and output its Common Workflow Language definition
+workflow = api.workflows().get(uuid='zzzzz-7fd4e-12345abcde67890').execute()
+print(workflow['definition'])
+
+# Get a workflow and output its name and description.
+# We pass a list of fields to select to avoid de/serializing the workflow
+# definition, which might be large and we're not going to use.
+workflow = api.workflows().get(
+ uuid='zzzzz-7fd4e-12345abcde67890',
+ select=['name', 'description']
+).execute()
+print(f"## {workflow['name']} ##\n\n{workflow['description']}")
+
+# ERROR: This raises a KeyError because we didn't load this field in
+# the `select` argument.
+workflow['created_at']
+{% endcodeblock %}
+
+h2. list method
+
+To fetch multiple Arvados objects of the same type, call the @list@ method for that resource type. The list method takes a number of arguments. Refer to the "list method API reference":{{site.baseurl}}/api/methods.html#index for details about them. The method returns a dictionary also documented at the bottom of that section. The most interesting field is @'items'@, which is a list of dictionaries where each one corresponds to an Arvados object that matched your search. To work with a single page of results:
+
+{% codeblock as python %}
+# Output the exit codes of the 10 most recently run containers.
+container_list = arv_client.containers().list(
+ limit=10,
+ order=['finished_at desc'],
+).execute()
+for container in container_list['items']:
+ print(f"{container['uuid']}: {container['exit_code']}")
+{% endcodeblock %}
+
+If you need to retrieve all of the results for a list, you need to call the list method multiple times with the same search criteria and increasing @offset@ arguments until no more items are returned. For example:
+
+{% codeblock as python %}
+def project_data(arv_client, project_uuid):
+ """Iterate portable data hashes of all collections in one project."""
+ # We create a dictionary to store all the keyword arguments we want to pass
+ # to the list method, so they're easy to repeat.
+ list_kwargs = {
+ # The actual search criteria we want to use.
+ 'filters': [['owner_uuid', '=', project_uuid]],
+ # The default offset is 0 if not specified, but initializing it here
+ # simplifies the loop code below.
+ 'offset': 0,
+ # This order helps ensure we don't miss objects that are added
+ # while our requests are running.
+ 'order': ['created_at asc'],
+ }
+ while True:
+ list_page = arv_client.collections().list(**list_kwargs).execute(num_retries=3)
+ page_collections = list_page['items']
+ if not page_collections:
+ break
+ for collection in page_collections:
+ yield collection['portable_data_hash']
+ list_kwargs['offset'] += len(page_collections)
+{% endcodeblock %}
+
+When you list many objects, the following can help improve performance:
+
+* Call the list method with @count='none'@ to avoid the overhead of counting all results with each request.
+* Call the list method with a @select@ argument to only request the data you need. This cuts out some overhead from de/serializing and transferring data you won't use.
+
+h2. create method
+
+To create a new Arvados object, call the @create@ method for that resource type. You must pass a @body@ argument with a dictionary of fields for the new object. The key-value pairs which may appear in the dictionary are determined by the type of resource you're creating. The method returns a dictionary with the new object's fields.
+
+If the resource type has a @name@ field, you may pass an @ensure_unique_name@ boolean argument. If true, the method will automatically update the name of the new object to make it unique if necessary.
+
+{% codeblock as python %}
+# Create a new project and output its UUID.
+project = arv_client.groups().create(
+ body={
+ 'name': 'Python SDK Test Project',
+ 'group_class': 'project',
+ },
+ ensure_unique_name=True,
+).execute()
+print(project['uuid'])
+{% endcodeblock %}
+
+h2. update method
+
+To modify an existing Arvados object, call the @update@ method for that resource type. You must pass a @uuid@ string argument that identifies the object to update, and a @body@ dictionary argument with the fields you want to update. The key-value pairs which may appear in @body@ are determined by the type of resource you're creating. The method returns a dictionary with the updated object's fields.
+
+If the resource type has a @name@ field, you may pass an @ensure_unique_name@ boolean argument. If true, the method will automatically update the name of the new object to make it unique if necessary.
+
+{% codeblock as python %}
+# Update the name of a container request,
+# and finalize it to submit it to Crunch for processing.
+submitted_container_request = arv_client.container_requests().update(
+ uuid='zzzzz-xvhdp-12345abcde67890',
+ body={
+ 'name': 'Python SDK Test Container Request',
+ 'state': 'Committed',
+ },
+ ensure_unique_name=True,
+).execute()
+{% endcodeblock %}
+
+h2. delete method
+
+To modify an existing Arvados object, call the @delete@ method for that resource type. You must pass a @uuid@ string argument that identifies the object to delete. The method returns a dictionary with the deleted object's fields.
+
+{% codeblock as python %}
+# Delete a collection and output its portable data hash
+deleted_collection = arv_client.collections().delete(
+ uuid='zzzzz-4zz18-12345abcde67890',
+).execute()
+print(deleted_collection['portable_data_hash'])
+{% endcodeblock %}
+
+For resource types that support being trashed, note that the @delete@ method simply sets the object's @trash_at@ field to the present, and its @delete_at@ field in the future. If you want more control over those times, you can use the @update@ method to set them yourself.
+
+{% codeblock as python %}
+# Schedule a collection to go to the trash tomorrow,
+# and to be permanently deleted three days later.
+import datetime
+now = datetime.datetime.utcnow()
+trash_at = now + datetime.timedelta(days=1)
+delete_at = now + datetime.timedelta(days=4)
+updated_collection = arv_client.collections().update(
+ uuid='zzzzz-4zz18-12345abcde67890',
+ body={
+ 'delete_at': f"{delete_at.isoformat()}Z",
+ 'trash_at': f"{trash_at.isoformat()}Z",
+ },
+).execute()
+{% endcodeblock %}
diff --git a/doc/sdk/python/example.html.textile.liquid b/doc/sdk/python/example.html.textile.liquid
deleted file mode 100644
index edcdba549..000000000
--- a/doc/sdk/python/example.html.textile.liquid
+++ /dev/null
@@ -1,70 +0,0 @@
----
-layout: default
-navsection: sdk
-navmenu: Python
-title: Examples
-...
-{% comment %}
-Copyright (C) The Arvados Authors. All rights reserved.
-
-SPDX-License-Identifier: CC-BY-SA-3.0
-{% endcomment %}
-
-In these examples, the site prefix is @aaaaa at .
-
-See also the "cookbook":cookbook.html for more complex examples.
-
-h2. Initialize SDK
-
-{% codeblock as python %}
-import arvados
-api = arvados.api("v1")
-{% endcodeblock %}
-
-h2. create
-
-{% codeblock as python %}
-result = api.collections().create(body={"collection": {"name": "create example"}}).execute()
-{% endcodeblock %}
-
-h2. delete
-
-{% codeblock as python %}
-result = api.collections().delete(uuid="aaaaa-4zz18-ccccccccccccccc").execute()
-{% endcodeblock %}
-
-h2. get
-
-{% codeblock as python %}
-result = api.collections().get(uuid="aaaaa-4zz18-ccccccccccccccc").execute()
-{% endcodeblock %}
-
-h2. list
-
-{% codeblock as python %}
-result = api.collections().list(filters=[["uuid", "=", "aaaaa-bbbbb-ccccccccccccccc"]]).execute()
-{% endcodeblock %}
-
-h2. update
-
-{% codeblock as python %}
-result = api.collections().update(uuid="aaaaa-4zz18-ccccccccccccccc", body={"collection": {"name": "update example"}}).execute()
-{% endcodeblock %}
-
-h2. Get current user
-
-{% codeblock as python %}
-result = api.users().current().execute()
-{% endcodeblock %}
-
-h2. Get the User object for the current user
-
-{% codeblock as python %}
-current_user = arvados.api('v1').users().current().execute()
-{% endcodeblock %}
-
-h2. Get the UUID of an object that was retrieved using the SDK
-
-{% codeblock as python %}
-my_uuid = current_user['uuid']
-{% endcodeblock %}
-----------------------------------------------------------------------
hooks/post-receive
--
More information about the arvados-commits
mailing list