Logo von nextlevels
DEEN
Request a project

Idempotency

Last updated on

In short

Idempotency describes a property of operations that is decisive for the stability of distributed systems: an operation is idempotent if executing it several times has exactly the same effect as executing it once. You can repeat it as often as you like without anything changing in the end state.

That sounds academic, but it has very tangible consequences: idempotency is the mechanism that prevents a customer from being charged twice by mistake when the network drops out briefly during a payment. For anyone running an API that handles money, orders or other critical processes, idempotency is not an optional extra but a must.

Why idempotency matters so much on the network

Distributed systems have a fundamental problem: when a client sends a request and receives no response, it does not know whether the request arrived and was processed or whether it was lost on the way. The response could have been lost just as well as the request itself. The client faces an unpleasant choice: not send again (and risk that the operation was never carried out) or send again (and risk carrying it out twice). This is exactly where idempotency resolves the dilemma: if the operation is idempotent, the client can safely repeat it until it receives a confirmation. In the worst case, the operation was received several times but took effect only once.

A distinction is often made between three delivery guarantees: "at-most-once" (at most once, but possibly not at all), "at-least-once" (at least once, but possibly twice) and "exactly-once" (exactly once). Genuine "exactly-once" is extremely hard, if not impossible, to guarantee at network level. The pragmatic route, and the one predominantly used in the industry, is therefore "at-least-once" delivery combined with idempotent processing. For the user the result feels like "exactly-once", but technically it is much simpler and more reliable to implement. Idempotency is thus the practical key to a guarantee that would otherwise be almost unattainable.

This pattern is found everywhere in reliable software: in automatic retry mechanisms, in message queues with "at-least-once" delivery, in webhook receivers and in payment pipelines. Without idempotency, each of these repetition mechanisms sooner or later leads to duplicate bookings, duplicate orders or inconsistent data.

A concrete scenario from online retail

Imagine a customer who clicks "Buy now" at checkout. The request reaches your server, the order is created and the payment is triggered, but at exactly that moment the customer's mobile connection drops before the success message reaches them. The customer sees a loading spinner, gets impatient and clicks again. Without a protective mechanism, your system now creates a second order and charges a second time. The result: an annoyed customer, a support ticket, a refund and, in the worst case, a negative review. With idempotency, on the other hand, the server recognises the repetition, returns the result of the first order and the customer simply sees their order confirmation, as if nothing had happened. It is precisely this invisible protection that distinguishes robust systems from fragile ones.

Idempotent and non-idempotent HTTP methods

In the context of a REST API, it is precisely defined which HTTP methods are supposed to be idempotent:

  • GET, HEAD, OPTIONS: idempotent and additionally "safe" (they do not change the state at all).
  • PUT: idempotent; overwriting the same resource with the same data always leads to the same result.
  • DELETE: idempotent; deleting a resource that has already been deleted changes nothing more in the state.
  • POST: not idempotent; sent twice, two records are created. This is exactly what makes POST dangerous with repetitions.
  • PATCH: not guaranteed to be idempotent; it depends on how the update is formulated (an absolute "set the price to 10 euros" is idempotent, a relative "increase the price by 10 per cent" is not).

This definition comes from the HTTP specification and is not a recommendation but a contract: clients, proxies and caches rely on it. A proxy may automatically repeat a failed GET request because GET counts as idempotent; with POST that would be fatal.

The idempotency key: the practical tool

But what if you want to make an inherently non-idempotent operation such as "create a new order" or "execute a payment" safely repeatable? This is where the idempotency key comes into play. The principle: before the request, the client generates a unique key (usually a UUID) and sends it along as a header. The server remembers this key together with the result of the operation. If a second request arrives with the same key, the server does not execute the operation again but returns the stored result of the first execution.

The most prominent example is the Stripe API: it supports an Idempotency-Key header for all POST requests. If your application sends a payment with a key and receives no response because of a timeout, it can repeat the identical request with the same key, and Stripe guarantees that the customer is charged only once. PayPal, the Adyen API and many other payment service providers also offer comparable mechanisms. In e-commerce, for example in a Shopware integration, this header is the standard means of preventing duplicate orders on shaky connections.

Important for understanding: the idempotency key always refers to one specific intention. If the client generates a new key for each genuine order attempt but the same key for every repetition of the same attempt, the protection works as intended. If it accidentally uses the same key for two genuinely different orders, the second would wrongly be recognised as a repetition and suppressed. Correct generation of the key on the client side is therefore just as much part of the design as the check on the server side; both halves have to be thought through together.

How an idempotency key works on the server

The server-side implementation follows a clear sequence that you can imagine as a lock:

  1. A request with an idempotency key arrives.
  2. The server checks whether the key is already stored.
  3. If it is new: execute the operation, store the result together with the key, return the result.
  4. If it is known: return the stored result without executing the operation again.
  5. If a first request with the key is still running, the second is blocked or rejected with a conflict status in order to avoid race conditions.

It is important that storing the key and the actual operation are linked transactionally; otherwise a window opens in which the operation ran but the key had not yet been stored. The stored keys usually get an expiry time (for example 24 hours) so that the storage does not grow indefinitely.

Implementing idempotency in your own back end

You do not have to wait for ready-made services to get idempotency: in your own back end (for example based on NestJS or Node.js), various strategies can be combined. Which one fits depends on the use case:

Common strategies for making operations idempotent and their typical areas of use
StrategyHow it worksTypical use
Idempotency keyClient sends a unique key, server stores the result per keyPayments, orders, critical POSTs
Natural key / unique constraintThe database prevents duplicates via a unique columnCreating records with a business ID
Upsert (INSERT ... ON CONFLICT)Insert or update in one stepSynchronisation, imports
State checkOnly execute the operation if the target state has not yet been reachedStatus changes, workflows

In practice, the combination of an idempotency key for the external interface and a unique constraint in the database as a second line of defence is particularly robust. Even if the application logic makes a mistake once, the database still catches the duplicate. This defence in depth is exactly the kind of care that pays off in production, usually precisely when you least expect it.

Idempotency beyond HTTP

The concept is older than the web and comes from mathematics: a function is idempotent if f(f(x)) = f(x) holds. The absolute value of a number is a simple example; taking it twice changes nothing. In software you encounter idempotency far beyond REST: when processing messages from a queue (Kafka, RabbitMQ and SQS often deliver "at least once", so the consumer has to cope with duplicate messages), when receiving webhooks (most providers resend when in doubt), with database upserts (INSERT ... ON CONFLICT) and with infrastructure-as-code tools such as Terraform or Ansible, whose entire promise consists of establishing the same desired state on every run.

This common thread, "repeat safely until it has definitely worked", is one of the most important thinking patterns for robust, distributed software of all. Anyone who designs systems so that as many operations as possible are idempotent makes the whole system more fault-tolerant, easier to operate and easier to test. Idempotent operations can be repeated without hesitation, which drastically simplifies retry logic, monitoring and restarting after disruptions. In architecture consulting, "Is this operation idempotent?" is therefore one of the first questions we ask when designing critical interfaces.

A technical deep dive into the definition can be found on Wikipedia.

Frequently asked questions about idempotency

What does idempotent mean, explained simply?
An operation is idempotent if it does not matter whether you execute it once or several times: the result stays the same. Deleting a file is idempotent; debiting money without a protective mechanism is not.

Is GET idempotent?
Yes. GET is even "safe", meaning it does not change the server state at all. Reading several times returns the same result (apart from changes made in the meantime).

Why is POST not idempotent?
Because every POST creates a new resource by definition. Sent twice, two records are created. With an idempotency key, however, POST can be made repeatable.

What is an idempotency key?
A unique key (usually a UUID) that the client sends with the request. The server executes the operation only once per key and returns the stored result for repetitions.

Do I really need idempotency?
As soon as requests move money, create orders or have other irreversible effects and run over unreliable networks: yes. It is the cheapest insurance against duplicate executions.

Next step

Robust interfaces that absorb duplicate calls

From interface architecture to third-party integration: custom software from one provider.

Changelog

  1. · Created · Jan Schumann