Idempotency and concurrency

Mutations must remain safe when a client times out, a worker restarts, or two actors modify the same record. The owning operation—not the client library—defines the guarantee.

Idempotency

An idempotency key identifies one intended mutation. Where an operation supports idempotency:

Generate a cryptographically random key for the business action, not for each network attempt.
Keep the same key and byte-equivalent semantic payload when retrying an unknown outcome.
Use a new key for a genuinely new action.
Persist the key with your local operation record until the server's replay window has elapsed.
Never derive keys from customer data, sequential IDs, or secrets.
Treat key reuse with a different payload as a conflict, not a retryable failure.

Where the key goes and what a replay returns differs by App:

App	Where the key goes	Format	Exact replay	Same key, different body
CRM	body field `idempotencyKey`	16 to 160 chars `A-Z a-z 0-9 . _ : -`	`200` with `replayed: true`	`409 IDEMPOTENCY_CONFLICT`
To-Do	body field `idempotencyKey`	16 to 160 chars `A-Z a-z 0-9 . _ : -`	`200` with `replayed: true`	`409` with field issue `idempotency_conflict`
Calendar	`Idempotency-Key` header, required on every `*-commands` route	16 to 160 chars	stored status with `replayed: true`	`409 IDEMPOTENCY_CONFLICT`
Accounting	`Idempotency-Key` header, required on commands	16 to 160 chars `A-Z a-z 0-9 . _ : -`	stored response	`409 idempotency_conflict`
Directory	`Idempotency-Key` header, optional on every write	16 to 160 chars `A-Z a-z 0-9 . _ : -`	the original status and body, byte for byte	`409 idempotency_conflict`

Keys are scoped to the workspace. A UUID v4 satisfies every App's format. No retention window is promised yet: keep keys for at least 24 hours of retries and do not reuse one for a different action, ever.

Replay outcomes

Situation	Safe client behavior
First request succeeds and response arrives	Persist the receipt/result; do not replay
First request times out before any response	Reconcile by receipt or replay with the same idempotency key if the operation permits it
Same key and same payload	Accept the prior result or replay response as the authoritative outcome
Same key and different payload	Stop; surface an idempotency conflict and investigate the caller
No idempotency mechanism declared	Treat the outcome as unknown; do not automatically retry a consequential mutation
Server says retryable and supplies delay	Retry within a bounded attempt and time budget, preserving the same action identity

A transport failure does not prove the server performed no work. That small distinction has funded many duplicate invoices throughout history.

Optimistic concurrency

Where a resource exposes recordVersion, expectedVersion, ETag, or another version token:

Read the current resource.
Build the mutation against that exact state.
Send the expected version using the operation's declared field or header.
On a stale-version conflict, read again and present or apply a deliberate merge.
Never overwrite by silently replacing the expected version with the newest one.

CRM controlled commands and Accounting draft/settings flows use optimistic version concepts in implementation and contract evidence. Calendar command conflicts protect event, hold, recurrence, and attendance state. The exact field, status, and conflict body are operation-specific.

Command receipts

A command receipt should be treated as durable evidence of acceptance and identity. Depending on the operation, it may contain:

command or receipt reference;
resulting resource reference;
disposition or state;
resulting record version;
whether the response is an exact replay;
correlation identifier;
timestamps safe for audit and reconciliation.

Do not equate receipt creation with completion of external provider work unless the response explicitly states final completion. For asynchronous work, poll or consume the documented completion event rather than repeating the command.

Retry decision

Did a response arrive?
├─ yes
│  ├─ 2xx: accept the documented result or receipt
│  ├─ conflict: reconcile state or idempotency history; do not blind-retry
│  ├─ 429: wait for Retry-After, add jitter, remain inside retry budget
│  └─ 5xx: retry only when the error model marks it retryable
└─ no
   ├─ idempotent read: retry with bounded exponential backoff
   ├─ mutation with accepted idempotency: same key and same payload
   └─ mutation without accepted idempotency: outcome unknown; reconcile manually

Recommended client state for a consequential mutation:

{
  "localOperationId": "op_01J6EXAMPLE00000000000000",
  "idempotencyKey": "b92c835a-8bea-4b94-a82d-4a16f0cb93de",
  "contractVersion": "recorded-with-request",
  "attempts": 1,
  "state": "outcome_unknown",
  "remoteReceipt": null
}

Do not log the full private payload alongside this state. Store only what your recovery and audit policy permits.