[ADP-362] Concurrency Control

reviewing phase 1

Add references

Overview

This specification covers two primary concurrency control mechanisms: Optimistic Locking and Pessimistic Locking.

Guidance

• If necessary, MUST ensure data consistency and integrity during concurrent operations.
• SHOULD use an appropriate concurrency control mechanism based on the application’s requirements and characteristics.
• MUST handle conflicts gracefully and inform clients appropriately.

Optimistic Locking

Concept: Optimistic locking assumes that conflicts are rare and checks for conflicts only at the time of committing the transaction. It uses ETags to detect conflicts.

Implementation Guidelines:

1. Resource Versioning with ETags:

   • MUST include an ETag header in the resource representation to act as a version identifier.
   • SHOULD use a strong ETag for resources that require precise control.

   Example Resource Representation:

   GET /users/123 HTTP/1.1
   Host: example.com
   
   HTTP/1.1 200 OK
   Content-Type: application/json
   ETag: "abc123"
   
   {
     "id": 123,
     "email": "user@example.com",
     "phone": "+1234567890"
   }

2. Including ETag in Update Requests:

   • MUST require clients to include the ETag in update requests using the If-Match header.
   • SHOULD reject the update if the If-Match header is missing.

   Example Update Request with ETag:

   PATCH /users/123 HTTP/1.1
   Host: example.com
   Content-Type: application/json-patch+json
   If-Match: "abc123"
   
   [
     { "op": "replace", "path": "/email", "value": "newemail@example.com" }
   ]

3. Conflict Detection:

   • MUST compare the provided ETag with the current ETag of the resource before applying the update.
   • MUST reject the update if the ETags do not match, indicating that the resource has been modified by another client.

4. Error Responses:

   • MUST return a 412 Precondition Failed status code when a conflict is detected.
   • SHOULD use the HTTP Problem Details format for error responses.

   Example Conflict Response:

   HTTP/1.1 412 Precondition Failed
   Content-Type: application/problem+json
   
   {
     "type": "https://example.com/problems/concurrent-update",
     "title": "Precondition Failed",
     "status": 412,
     "detail": "The resource has been modified by another client.",
     "instance": "/users/123",
     "currentETag": "def456"
   }

5. Updating the ETag:

   • MUST generate a new ETag each time the resource is successfully updated.
   • SHOULD include the updated ETag in the response headers.

   Example Successful Update Response:

   HTTP/1.1 200 OK
   Content-Type: application/json
   ETag: "def456"
   
   {
     "id": 123,
     "email": "newemail@example.com",
     "phone": "+1234567890"
   }

Pessimistic Locking

Concept: Pessimistic locking assumes that conflicts are common and locks the resource for the duration of the operation to prevent other clients from modifying it.

Implementation Guidelines:

1. Lock Acquisition:

   • MUST lock the resource before performing any modifications.
   • SHOULD use a locking mechanism; we introduce a WebDAV-like mechanism here.
   • SHOULD return Lock-Token in the response header when the lock request succeeds.
   • SHOULD specify Timeout in the response header.

2. Operation during lock:

   • SHOULD specify Lock-Token in the response header.

3. Lock Release:

   • MUST release the lock once the operation is completed.
   • SHOULD ensure that locks are released even if the operation fails, typically using a try-finally block, or a server timeout mechanism.
   • SHOULD specify Lock-Token in the request header

4. Handling Lock Failures:

   • MUST return a 423 Locked status code if the resource is already locked by another client.
   • SHOULD use the HTTP Problem Details format for error responses.

   Example Lock Failure Response:

   HTTP/1.1 423 Locked
   Content-Type: application/problem+json
   
   {
     "type": "https://example.com/problems/locked",
     "title": "Locked",
     "status": 423,
     "detail": "The resource is currently locked by another operation.",
     "instance": "/users/123"
   }

5. Long-Running Operations:

   • SHOULD minimize the duration of locks to avoid long wait times for other clients.
   • MAY use a lock timeout mechanism to release locks that have been held for too long.

Best Practices

1. Clear Documentation:
   • SHOULD clearly document the concurrency control mechanism used, including how to include ETags in requests and handle conflicts.
2. Client Handling:
   • SHOULD advise clients on how to handle 412 Precondition Failed and 423 Locked responses, including retry strategies.
3. Consistency:
   • MUST ensure that the ETag is consistently updated and checked across all instances of the resource.
4. Idempotent Operations:
   • SHOULD design update operations to be idempotent, so that retrying an operation with the same ETag does not result in inconsistent states.

Example Workflow for Optimistic Locking

1. Client Fetches Resource:

   • Client retrieves the resource and gets the current ETag.

   GET /users/123 HTTP/1.1
   Host: example.com

   Response:

   HTTP/1.1 200 OK
   Content-Type: application/json
   ETag: "abc123"
   
   {
     "id": 123,
     "email": "user@example.com",
     "phone": "+1234567890"
   }

2. Client Sends Update Request:

   • Client includes the ETag in the update request using the If-Match header.

   PATCH /users/123 HTTP/1.1
   Host: example.com
   Content-Type: application/json-patch+json
   If-Match: "abc123"
   
   [
     { "op": "replace", "path": "/email", "value": "newemail@example.com" }
   ]

3. Server Checks ETag:

   • Server checks if the provided ETag matches the current ETag of the resource.
   • If ETags match, the server applies the update and generates a new ETag.
   • If ETags do not match, the server returns a 412 Precondition Failed response.

4. Successful Update Response:

   • If the update is successful, the server responds with the updated resource and new ETag.

   HTTP/1.1 200 OK
   Content-Type: application/json
   ETag: "def456"
   
   {
     "id": 123,
     "email": "newemail@example.com",
     "phone": "+1234567890"
   }

5. Conflict Response:

   • If a conflict is detected, the server responds with a 412 Precondition Failed status code and the current ETag of the resource.

   HTTP/1.1 412 Precondition Failed
   Content-Type: application/problem+json
   
   {
     "type": "https://example.com/problems/concurrent-update",
     "title": "Precondition Failed",
     "status": 412,
     "detail": "The resource has been modified by another client.",
     "instance": "/users/123",
     "currentETag": "def456"
   }

Example Workflow for Pessimistic Locking

1. Client Requests Lock:

   • Client requests a lock on the resource before performing the update.

   POST /users/123/lock HTTP/1.1
   Host: example.com
   Timeout: Second-60

   Response:

   HTTP/1.1 200 OK
   Content-Type: application/json
   Lock-Token: abc123
   
   {
     "lockId": "abc123",
     "resource": "/users/123",
     "locked": true
   }

2. Client Sends Update Request:

   • Client sends the update request, including the lock identifier.

   PATCH /users/123 HTTP/1.1
   Host: example.com
   Content-Type: application/json-patch+json
   Lock-Id: "abc123"
   
   [
     { "op": "replace", "path": "/email", "value": "newemail@example.com" }
   ]

3. Server Applies Update:

   • Server verifies the lock identifier, applies the update, and then releases the lock.
   • If the lock identifier is invalid or the resource is locked by another client, the server returns a 423 Locked response.

4. Successful Update Response:

   • If the update is successful, the server responds with the updated resource.

   HTTP/1.1 200 OK
   Content-Type: application/json
   
   {
     "id": 123,
     "email": "newemail@example.com",
     "phone": "+1234567890"
   }

5. Lock Failure Response:

   • If the resource is already locked by another client, the server responds with a 423 Locked status code.

   HTTP/1.1 423 Locked
   Content-Type: application/problem+json
   
   {
     "type": "https://example.com/problems/locked",
     "title": "Locked",
     "status": 423,
     "detail": "The resource is currently locked by another operation.",
     "instance": "/users/123"
   }

6. Client Unlocks the resource

   POST /users/123/lock HTTP/1.1
   Host: example.com
   Lock-Token: abc123

This specification provides a comprehensive framework for implementing concurrency control in RESTful APIs using both optimistic and pessimistic locking mechanisms. The use of ETags ensures standardized version control, and the HTTP Problem Details format ensures standardized and informative error responses.

Reference

• https://sookocheff.com/post/api/optimistic-locking-in-a-rest-api/
• https://datatracker.ietf.org/doc/html/rfc4918