[ADP-357] Long Running Job

Overview

This document outlines the guidelines and best practices for handling long-running jobs in RESTful APIs. Long-running jobs are operations that may take a significant amount of time to complete and cannot be processed within the typical request-response cycle.

Guidance

MUST return HTTP Status 202 Accepted for a successful job creation
MUST return Location in the response header to provide the client with information about the created job
There are two options for providing additional result information.
- MAY return Location when the job is completed, generating a new resource with a URI different from the job.
http
```
HTTP/1.1 200 OK
Location: https://my.dev/v1/jobs/job-id/results
Content-Location: https://my.dev/v1/jobs/job-id

{
  "jobId": "123e4567-e89b-12d3-a456-426614174000",
  "status": "COMPLETED",
  "createdAt": "2023-04-01T12:00:00Z",
  "updatedAt": "2023-04-01T12:10:00Z"
}
```
- MAY include a response field within the job resource.
TIP
This approach is inspired by opensearch and elasticsearch, which both directly embed the response within the search job. The design rationale is to enable polling of the search job and provide the API user with updates on partial search results. Consider your use case and decide accordingly. If your goal is to offer an API similar to elasticsearch/opensearch, you MAY adopt this design. However, their design does not fully align with the current ADP. Remember the best practice principle: if you (as the API designer) believe it's an industry standard, adopt it. If you're not satisfied with the design, strive to find a better one and contribute back here.
MUST include a status field in the job resource
SHOULD use a unique identifier (e.g., UUID) for each job
MAY attach Sunset header to indicate when the job resource will be removed after completion
SHOULD provide a problem details object (compatible with RFC 9457) or error JSON if the job fails
MAY provide a mechanism for clients to cancel ongoing jobs as necessary
- DRAFT MAY employ the DELETE operation to remove a job, thereby indicating cancellation.
http
```
DELETE /jobs/{job-id}
```
MAY consider implementing a notification system (e.g., webhooks) to inform clients when jobs are completed
SHOULD implement proper security measures to ensure that clients can only access their own job resources
DRAFT MAY attach Retry-After for processing job result.

Implementation Details

Job Creation

When a client initiates a long-running job, the server should:

Accept the request and create a job resource
Return a 202 Accepted status code
Include a Location header with the URI of the created job resource

Example response:

http

HTTP/1.1 202 Accepted
Location: https://api.my.dev/v1/jobs/123e4567-e89b-12d3-a456-426614174000

Job Resource

The job resource should include at least the following information:

A unique identifier
The current status of the job
Timestamps for creation and last update

Example job resource:

json

HTTP/1.1 200 OK
Retry-After: 60

{
  "jobId": "123e4567-e89b-12d3-a456-426614174000",
  "status": "RUNNING",
  "createdAt": "2023-04-01T12:00:00Z",
  "updatedAt": "2023-04-01T12:05:00Z"
}

Job Status

The job status should be one of the following:

PENDING: The job is queued but not yet started
RUNNING: The job is currently being processed
COMPLETED: The job has finished successfully
FAILED: The job has encountered an error and could not complete

Error Handling

If a job fails, provide detailed error information using a problem details object:

http

GET /jobs/123e4567-e89b-12d3-a456-426614174000 HTTP/1.1

HTTP/1.1 200 OK
Content-Type: application/json

{
  "jobId": "123e4567-e89b-12d3-a456-426614174000",
  "status": "FAILED",
  "createdAt": "2023-04-01T12:00:00Z",
  "updatedAt": "2023-04-01T12:10:00Z",
  "problem": {
    "type": "https://example.com/problems/job-execution-error",
    "title": "Job Execution Error",
    "status": 500,
    "detail": "An unexpected error occurred during job execution"
  }
}

OpenAPI 3.1.0 Example

Here's an example of how to document a long-running job API using OpenAPI 3.1.0:

yaml

openapi: 3.1.0
info:
  title: Long Running Job API
  version: 1.0.0

paths:
  /jobs:
    post:
      summary: Create a new job
      operationId: createJob
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/JobRequest'
      responses:
        '202':
          description: Accepted
          headers:
            Location:
              schema:
                type: string
              description: URI of the created job resource
  
  /jobs/{jobId}:
    get:
      summary: Get job status
      operationId: getJobStatus
      parameters:
        - name: jobId
          in: path
          required: true
          schema:
            type: string
            format: uuid
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Job'

components:
  schemas:
    JobRequest:
      type: object
      properties:
        # Define job request properties
    
    Job:
      type: object
      properties:
        jobId:
          type: string
          format: uuid
        status:
          type: string
          enum: [PENDING, RUNNING, COMPLETED, FAILED]
        createdAt:
          type: string
          format: date-time
        updatedAt:
          type: string
          format: date-time
        problem:
          $ref: '#/components/schemas/Problem'
    
    Problem:
      type: object
      properties:
        type:
          type: string
          format: uri
        title:
          type: string
        status:
          type: integer
        detail:
          type: string

References

Changelog

2024.10.30 Add more reference; add result design for long running task

[ADP-357] Long Running Job ​

Overview ​

Guidance ​

Implementation Details ​

Job Creation ​

Job Resource ​

Job Status ​

Error Handling ​

OpenAPI 3.1.0 Example ​

References ​

Changelog ​