← All stories

Completion monitoring · Controlled experiment

Your systems are running.
Did the work finish?

Clutta recorded 24 introduced delivery failures while leaving 100,000 successfully completed operations unflagged.

Clutta · 5 min read

Share by email

The completion question

For engineering teams responsible for business workflows, the important question is whether each operation reached its expected outcome.

Consider an order that has been paid for but still has no confirmation. The payment step succeeded. The person who placed the order is still waiting for the next thing they were promised.

The same completion problem appears elsewhere. A report can be generated without reaching its recipient. An account can be created before access is granted. An AI answer can be produced while delivery remains incomplete.

Clutta monitors business workflows for that expected completion.

What Clutta demonstrated

In a six-hour controlled test of AI answer delivery, Clutta recorded all 24 failures we deliberately introduced. Across another 100,000 operations that completed successfully, none was wrongly flagged.

Each operation represented one AI answer moving toward its expected delivery outcome. We checked the results against the application's own records.

Six hours. Two questions.

100,000successful operations.
None wrongly flagged.
Includes 4,967 retries that finished on time.
24 / 24introduced failures.
All recorded.
9 missing completion · 15 late findings

Simulated responses · Test delivery · Checked against application records

Measured groups, not a proportional chart. Successful retries are included in the 100,000, not additional operations.
Six-hour AI-delivery test
What happenedWhat Clutta recorded
24 failures deliberately introducedAll 24 recorded
100,000 operations completed successfullyNone wrongly flagged
4,967 of those successful operations retried and finished on timeNone wrongly flagged

Both sides matter. A team needs visibility into affected work, and it needs confidence that successful work can be left alone.

A successful step leaves a question open

A business workflow is a set of steps that produces an outcome someone needs. Each step has its own result. A successful payment confirms that the payment step worked. The wider operation still has an expected outcome to reach.

Follow the outcome

A successful step is not a finished workflow.

  1. 01 · PersistAnswer saved
  2. 02 · SendDelivery attempted
  3. 03 · ConfirmAcceptance expected

Same operation. Required outcome. Agreed deadline.

Health: is the service running?
Completion: did this operation reach its expected outcome?

Illustrative delivery contract, not a screenshot or trace from the test. Destination acceptance is distinct from a person reading the answer.

Availability answers a related question: is a service running and responding? That information is valuable. It leaves a further question about each operation passing through the service: did the expected work finish?

That is the completion question Clutta brings into monitoring.

A retry can still end in success

A retry means an application makes another attempt after a temporary interruption. A service may temporarily limit how many requests it accepts. The application waits and tries again. The work can still finish within its allowed time.

In this test, 4,967 of the 100,000 successful operations encountered rate-limit retries and still completed on time. Clutta did not wrongly flag any of them.

A recovered interruption should be understood in the context of the final outcome.

Which work needs attention?

When the expected outcome has not been confirmed, the team has a concrete question to investigate. If it arrived late, the timing also matters.

9recorded as missing completion
15recorded as late after processing delays

Some findings therefore became visible after delays. The result establishes that all 24 introduced failures were recorded; it does not establish immediate visibility for every case.

This distinction helps engineering teams set an accurate expectation of what the evidence demonstrates.

About this result

This was a six-hour controlled AI-delivery test using simulated model responses and test delivery. All 24 failures were deliberately introduced. Results were checked against application records.

The successful group represents repeated work over six hours, including thousands of operations that recovered through retries. The absence of wrongly flagged successful operations describes this measured result, not a guarantee for every deployment. Processing delays affected when some findings were recorded.

The measured example is AI answer delivery. Orders, reports and account provisioning illustrate the wider completion problem; they are not additional measured cases from this test. Test delivery does not establish that a real customer received or read a message.

This controlled test is separate from our 121-answer live messaging experiment, where the four unconfirmed deliveries occurred naturally.

From reading to testing

Pick one promised outcome.

Start with the documented playground scenarios: observe healthy completion, introduce a controlled failure, then verify recovery. The public playground is an introduction, not a reproduction of this six-hour benchmark.

Explore the playground →

Explore Clutta

Over to you

What does finished mean in your system?

Which customer-facing outcome is hardest to confirm? Tell us about the handoff, or question the experiment's assumptions.

Send us your perspective →

This opens your email app. Your reply goes to the Clutta team, not a public comment feed.

Up is not done.