In high-volume messaging applications (like WhatsApp, Telegram, or webchats), it's common for a user to send multiple messages in sequence (message A followed by message B within a short interval) before the first response is generated and delivered.
This article presents a complete case study on concurrent state management in conversational AI Agents. In high-volume messaging applications (like WhatsApp, Telegram, or webchats), it's common for a user to send multiple messages in sequence (message A followed by message B within a short interval) before the first response is generated and delivered. Without architectural concurrency control, the system suffers from duplicated token consumption in LLMs, context desynchronization, and delivery of contradictory or out-of-order responses.
Source Code Full repository on GitHub containing the evolutionary branches, simulation scripts, and Docker Compose instructions: 🔗 https://github.com/marcelo3macedo/gerenciamento-estado-concorrente-no-adk
In conversational systems based on Artificial Intelligence agents, users' actual behavior on messaging channels (like WhatsApp, Telegram, or Webchat) differs drastically from HTTP's traditional synchronous request-response model. Users interact fluidly and frequently send messages in short, sequential bursts (rapid-fire messaging). For example:
When the agent's architecture operates in a decoupled, asynchronous way — where a webhook receives each message in isolation, triggers LLM inference, and queues the response in a message broker with operational delay — a race condition and state-management failure emerges.
The problem manifests across three critical dimensions:
Semantic Inconsistency and Contradictory Responses Since message A has already started its lifecycle in the agent, the system generates the cancellation confirmation and sends it to the outbound queue. When message B arrives 1.5 seconds later, the agent generates the address-change response and queues it right after. On the user's channel, however, the cancellation response for A is delivered after the customer has already given up on the cancellation, creating noise and eroding trust.
Computational and Financial Waste The agent consumes significant compute resources and input/output tokens on the model's (LLM) API to process and generate the response to message A, unaware that the dialogue context was completely invalidated seconds later by message B.
Broken User Experience (UX) Out-of-order chat bubbles overlapping violates the conversational expectations of enterprise-grade virtual assistants.
To resolve the race condition, the architecture uses Redis as a distributed lock manager tied to the session identifier, and RabbitMQ as an asynchronous dispatch queue. Whenever a new message enters the system while a previous response is still being processed or is waiting to be sent from the outbound queue, Redis signals an active lock. This lock intercepts the RabbitMQ consumer, preventing the in-flight message from being dispatched to the user until an evaluator agent determines whether the previous response should be released, canceled, or merged with the new input.
Across three branches, order-service progressively gained the layers needed to treat this concurrency as a first-class problem, not a bug to hide:
| Branch | What was added |
|---|---|
feature/01-naive-fastapi-rabbitmq-delay | Baseline scenario: webhook + ADK agent + outbound queue with operational delay, no concurrency protection whatsoever |
feature/02-redis-lock-rabbitmq-exponential-backoff | Redis session lock + exponential-backoff retry ladder on the consumer via RabbitMQ DLX |
feature/03-google-adk-intent-evaluator-triage | Intention Evaluator decides RELEASE_FIRST / CANCEL_FIRST / MERGE |
feature/04-e2e-simulative-tests-pytest-testcontainers | E2E test suite with testcontainers |
This is the core of the case: a message published to outbound_messages isn't just "delivered or not delivered" — it moves through a small set of states, tracked in two Redis structures and checked by the consumer on every dispatch attempt.
The lock and the per-message status answer different questions:
While the lock is active, the consumer doesn't poll or hold the message in memory — it NACKs and republishes it to a delay queue with a TTL that grows with each attempt:
The timeline below is real data, pulled straight from the logs of a suite run: a lock that never gets released causes the message to exhaust all 3 attempts and land in outbound_messages.parked, never reaching the user and never retrying forever:
NACK (rejection) and places the message on a retry (backoff) queue with a 0.3s delay.outbound_messages.parked queue.This prevents stuck messages from being reprocessed indefinitely, which would needlessly consume server CPU and memory.
Detailed information on the main decisions in this case:
| Decision | Alternative Considered | Why It Was Chosen? | Accepted Trade-off |
|---|---|---|---|
| Explicit lock release at the end of triage, TTL only as a safety net | Rely solely on the lock's TTL (5s) to release the session | Holding the lock until the TTL expires would make every in-flight message wait up to 5s even when triage finishes in milliseconds — unnecessary in most cases. | If the webhook process dies between activating and releasing the lock, the message keeps retrying (backoff) until the lock expires via TTL |
Decision matrix via a 2nd ADK agent (output_schema=IntentionDecision, no tools, ephemeral session) | Deterministic heuristic (keywords, text similarity) | Distinguishing "cancel" from "don't cancel, never mind" requires understanding the sentence's intent, not just the presence of keywords. | Every collision (B arriving while A is still in flight) adds an extra LLM call to B's latency, plus the cost of a second ADK session per evaluation. |
| Retry with backoff capped at 3 attempts + parking queue | Infinite retry until the lock releases | A lock that never releases (webhook crash) can't turn into a message retrying forever and consuming the consumer. | Parked messages aren't automatically redelivered; they require manual intervention or a separate reprocessing job. |
Asynchronous delivery via RabbitMQ (webhook publishes, consumer dispatches) instead of a synchronous response in POST /webhook/message | Return the agent's response directly in the HTTP response body | Necessary to simulate and test a real channel's operational delay, and for the lock to make sense — without a queue, there's no "in-flight message" to re-evaluate. | Additional latency (the operational delay) between generating the response and delivering it, even on the happy path with no collision at all. |
scripts/simulate_intention_triage.py reproduces the 3 decision paths via HTTP, each in its own session. The times below are real, pulled from the E2E suite logs, with A and B separated by ~0.3s, within the 0.6s operational-delay window configured for the tests:
Decision 1 — RELEASE_FIRST (B doesn't conflict with A) A = "What are your business hours?", B = "And do you accept PIX?". Nothing is flagged in Redis; both responses reach the user, A first (606ms), B shortly after (891ms).
Decision 2 — CANCEL_FIRST (B nullifies A)
A = "I want to cancel my order", B = "Actually don't cancel it, I just received it". A is flagged CANCELLED; the consumer discards A (reason=cancelled, 607ms after the webhook) without dispatching it to the user. Only B's response arrives (892ms).
Decision 3 — MERGE (B complements A)
A = "Add a pepperoni pizza", B = "And a 2L Coke too". A is flagged SUPERSEDED; the consumer discards A (reason=superseded, 608ms). Since the agent's session already has A in memory, the second turn produces a single response covering both items — the user receives one combined reply (893ms), never two.
Extra case — Parking
Message published, lock activated and never released. The retry ladder exhausts all 3 attempts (0.6s + 0.3s + 0.6s + 1.2s) and the message is moved to outbound_messages.parked at 2.71s, never reaching the user and never retrying forever.
RELEASE_FIRST (Release): When the new message doesn't conflict with the previous one (e.g., a question about business hours followed by a question about payment methods), the system releases the held response and processes the next one in the queue, ensuring order and fluidity.CANCEL_FIRST (Cancellation): When the second message nullifies the first (e.g., "I want to cancel the order" followed by "never mind, don't cancel"), sending the first response is aborted. This generates direct token savings (FinOps) by avoiding redundant LLM interactions and generations, and it also prevents a stale cancellation confirmation from reaching the user.SUPERSEDED (Unification / Merge): When the new message complements the previous one (e.g., "add a pizza" followed by "and a 2L soda"), the first message is flagged as superseded. The agent generates a single consolidated response with both items, eliminating the visual clutter of multiple chat bubbles.outbound_messages.parked queue. This preserves the ecosystem, isolates the problem, and keeps the consumer free to serve other users.The complete code, with all four branches and the test suite, is at: