[ARVADOS] created: 1.3.0-2055-g21b4ab9d1

Git user git at public.arvados.org
Mon Jan 13 15:48:29 UTC 2020


        at  21b4ab9d1db2cddeb77908c6fe575be041d9ff6c (commit)


commit 21b4ab9d1db2cddeb77908c6fe575be041d9ff6c
Author: Tom Clegg <tom at tomclegg.ca>
Date:   Mon Jan 13 10:47:06 2020 -0500

    15823: Add "cloud dispatcher" API doc page.
    
    Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom at tomclegg.ca>

diff --git a/doc/_config.yml b/doc/_config.yml
index 310f776f8..0f7684f83 100644
--- a/doc/_config.yml
+++ b/doc/_config.yml
@@ -129,6 +129,8 @@ navbar:
       - api/methods/container_requests.html.textile.liquid
       - api/methods/containers.html.textile.liquid
       - api/methods/workflows.html.textile.liquid
+    - Management (admin/system):
+      - api/dispatch.html.textile.liquid
     - Jobs engine (legacy):
       - api/crunch-scripts.html.textile.liquid
       - api/methods/jobs.html.textile.liquid
diff --git a/doc/api/dispatch.html.textile.liquid b/doc/api/dispatch.html.textile.liquid
new file mode 100644
index 000000000..1ca1279ee
--- /dev/null
+++ b/doc/api/dispatch.html.textile.liquid
@@ -0,0 +1,134 @@
+---
+layout: default
+navsection: api
+navmenu: API Methods
+title: "cloud dispatcher"
+
+...
+{% comment %}
+Copyright (C) The Arvados Authors. All rights reserved.
+
+SPDX-License-Identifier: CC-BY-SA-3.0
+{% endcomment %}
+
+The cloud dispatcher provides several management/diagnostic APIs, intended to be used by a system administrator.
+
+These APIs are not normally exposed to external clients. To use them, connect directly to the dispatcher's internal URL (see Services.DispatchCloud.InternalURLs in the cluster config file). All requests must include the cluster's management token (@ManagementToken@ in the cluster config file).
+
+Example:
+
+<notextile><pre><code>curl -H "Authorization: Bearer $management_token" http://localhost:9006/arvados/v1/dispatch/containers</code></pre></notextile>
+
+These APIs are not available via @arv@ CLI tool.
+
+h3. List containers
+
+ at GET /arvados/v1/dispatch/containers@
+
+Return a list of containers that are either ready to dispatch, or being started/monitored by the dispatcher.
+
+The response provides @uuid@, @priority@, and @state@ fields for each container. Other fields of the container records are not loaded by the dispatcher, and will have empty/zero values here (e.g., @{...,"created_at":"0001-01-01T00:00:00Z","command":[],...}@).
+
+Example response:
+
+<notextile><pre>{
+  "items": [
+    {
+      "container": {
+        "uuid": "zzzzz-dz642-xz68ptr62m49au7",
+        ...
+        "priority": 562948375092493200,
+        ...
+        "state": "Locked",
+        ...
+      },
+      "instance_type": {
+        "Name": "Standard_E2s_v3",
+        "ProviderType": "Standard_E2s_v3",
+        "VCPUs": 2,
+        "RAM": 17179869184,
+        "Scratch": 32000000000,
+        "IncludedScratch": 32000000000,
+        "AddedScratch": 0,
+        "Price": 0.146,
+        "Preemptible": false
+      }
+    },
+    ...
+  ]
+}</pre></notextile>
+
+h3. Terminate a container
+
+ at POST /arvados/v1/dispatch/containers/kill?container_uuid={uuid}&reason={string}@
+
+Make a single attempt to terminate the indicated container on the relevant cloud VM. (The caller can implement a delay-and-retry loop if needed.)
+
+A container terminated this way will end with state @Cancelled@ if its docker container had already started, or @Queued@ if it was terminated while setting up the runtime environment.
+
+The provided @reason@ string will appear in the dispatcher's log, but not in the user-visible container log.
+
+If the provided @container_uuid@ is not scheduled/running on a worker node, the response status will be 404.
+
+h3. List instances
+
+ at GET /arvados/v1/dispatch/instances@
+
+Return a list of cloud instances.
+
+Example response:
+
+<notextile><pre>{
+  "items": [
+    {
+      "instance": "/subscriptions/abcdefab-abcd-abcd-abcd-abcdefabcdef/resourceGroups/zzzzz/providers/Microsoft.Compute/virtualMachines/compute-abcdef0123456789abcdef0123456789-abcdefghijklmno",
+      "address": "10.23.45.67",
+      "price": 0.073,
+      "arvados_instance_type": "Standard_DS1_v2",
+      "provider_instance_type": "Standard_DS1_v2",
+      "last_container_uuid": "zzzzz-dz642-vp7scm21telkadq",
+      "last_busy": "2020-01-13T15:20:21.775019617Z",
+      "worker_state": "running",
+      "idle_behavior": "run"
+    },
+    ...
+}</pre></notextile>
+
+The @instance@ value is the instance's identifier, assigned by the cloud provider. It can be used with the instance APIs below.
+
+The @worker_state@ value indicates the instance's capability to run containers.
+* @unknown@: instance was not created by this dispatcher, and a boot probe has not yet succeeded (this state typically appears briefly after the dispatcher restarts).
+* @booting@: cloud provider says the instance exists, but a boot probe has not yet succeeded.
+* @idle@: instance is idle and ready to run a container.
+* @running@: instance is running a container.
+* @shutdown@: cloud provider has been instructed to terminate the instance.
+
+The @idle_behavior@ value determines what the dispatcher will do with the instance when it is idle; see hold/drain/run APIs below.
+
+h3. Hold an instance
+
+ at POST /arvados/v1/dispatch/instances/hold?instance_id={instance}@
+
+Set the indicated instance's idle behavior to @hold at . The instance will not be shut down automatically. If a container is currently running, it will be allowed to continue, but no new containers will be scheduled.
+
+h3. Drain an instance
+
+ at POST /arvados/v1/dispatch/instances/drain?instance_id={instance}@
+
+Set the indicated instance's idle behavior to @drain at . If a container is currently running, it will be allowed to continue, but when the instance becomes idle, it will be shut down.
+
+h3. Resume an instance
+
+ at POST /arvados/v1/dispatch/instances/run?instance_id={instance}@
+
+Set the indicated instance's idle behavior to @run@ (the normal behavior). When it becomes idle, it will be eligible to run new containers. It will be shut down automatically when the configured idle threshold is reached.
+
+h3. Shut down an instance
+
+ at POST /arvados/v1/dispatch/instances/kill?instance_id={instance}&reason={string}@
+
+Terminate the indicated instance.
+
+If a container is running on the instance, it will be killed too; no effort is made to wait for it to end gracefully.
+
+The provided @reason@ string will appear in the dispatcher's log.

commit 39138c85883383830cc0a93032a77b100c672e55
Author: Tom Clegg <tom at tomclegg.ca>
Date:   Mon Jan 13 10:46:21 2020 -0500

    15823: Skip slow parts of doc rendering if NO_SDK env var is set.
    
    Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom at tomclegg.ca>

diff --git a/doc/Rakefile b/doc/Rakefile
index 71d86c32d..dca19a334 100644
--- a/doc/Rakefile
+++ b/doc/Rakefile
@@ -22,7 +22,7 @@ task :generate => [ :realclean, 'sdk/python/arvados/index.html', 'sdk/R/arvados/
 end
 
 file "sdk/python/arvados/index.html" do |t|
-  if File.exists? "no-sdk"
+  if ENV['NO_SDK'] || File.exists?("no-sdk")
     next
   end
   `which epydoc`
@@ -35,7 +35,7 @@ file "sdk/python/arvados/index.html" do |t|
 end
 
 file "sdk/R/arvados/index.html" do |t|
-  if File.exists? "no-sdk"
+  if ENV['NO_SDK'] || File.exists?("no-sdk")
     next
   end
   `which R`
@@ -102,7 +102,7 @@ EOF
 end
 
 file "sdk/java-v2/javadoc/index.html" do |t|
-  if File.exists? "no-sdk"
+  if ENV['NO_SDK'] || File.exists?("no-sdk")
     next
   end
   `which java`

commit deff3d5c88bd150225b6e6d04d173f0f7417b9b6
Author: Tom Clegg <tom at tomclegg.ca>
Date:   Mon Jan 13 10:45:05 2020 -0500

    15823: Fix API endpoint connected to wrong handler.
    
    Arvados-DCO-1.1-Signed-off-by: Tom Clegg <tom at tomclegg.ca>

diff --git a/lib/dispatchcloud/dispatcher.go b/lib/dispatchcloud/dispatcher.go
index 19c8f6e0b..4023896f7 100644
--- a/lib/dispatchcloud/dispatcher.go
+++ b/lib/dispatchcloud/dispatcher.go
@@ -148,7 +148,7 @@ func (disp *dispatcher) initialize() {
 	} else {
 		mux := httprouter.New()
 		mux.HandlerFunc("GET", "/arvados/v1/dispatch/containers", disp.apiContainers)
-		mux.HandlerFunc("POST", "/arvados/v1/dispatch/containers/kill", disp.apiInstanceKill)
+		mux.HandlerFunc("POST", "/arvados/v1/dispatch/containers/kill", disp.apiContainerKill)
 		mux.HandlerFunc("GET", "/arvados/v1/dispatch/instances", disp.apiInstances)
 		mux.HandlerFunc("POST", "/arvados/v1/dispatch/instances/hold", disp.apiInstanceHold)
 		mux.HandlerFunc("POST", "/arvados/v1/dispatch/instances/drain", disp.apiInstanceDrain)

-----------------------------------------------------------------------


hooks/post-receive
-- 




More information about the arvados-commits mailing list