Back to lab notesBuild notes
Build notes3 min read

Good automation has an off switch.

A workflow is not finished when the happy path runs. It also needs a safe retry, a visible result, and a way to stop.

An automation looks convincing when every step succeeds. The more revealing demonstration is what happens when the connection drops halfway through.

Consider a workflow that reads a selected set of notes and creates tasks in a project board. It sends a request, waits, and receives no reply. Did the task fail to appear, or did it appear while the response went missing?

Sending the same request again might be the right move. It might also produce a duplicate.

The interesting work is not just connecting the two systems. It is deciding what the connection should do when the result is uncertain.

A retry is another action

Amazon’s Builders’ Library explains this problem in Making retries safe with idempotent APIs. A request can succeed even when the caller does not receive the response. A supported idempotency mechanism lets a retry refer to the same intended operation instead of creating another one.

For the task-board example, keep a stable operation identifier for the intended task creation and reuse it when the destination supports that mechanism. Treat a materially changed request as a different operation. Do not assume that every API provides the same guarantees or remembers an identifier indefinitely.

Where the destination offers no such protection, inspect the current state and define a reconciliation step before retrying uncertain writes. A queue item labeled “needs review” is a better outcome than a confident guess that creates more work.

Bound the effort

Not every failure deserves the same response. A temporary service interruption may justify a retry. Invalid input usually needs correction instead. Revoked permission needs to be restored by the person who controls it.

Retries should also have a stopping point. Amazon’s discussion of timeouts, retries, and backoff describes why uncontrolled retries can add load to an already struggling service.

For a small workflow, I would set a finite retry budget, space retry attempts, and surface the unresolved item when that budget is exhausted. The person using the system should not have to discover the failure through an unexpectedly large queue or a duplicate project board.

Define what pause actually means

An off switch needs a clear meaning. Does it stop accepting new work? Prevent queued items from starting? Cancel a request already in progress?

Those are different promises. A request already accepted by another service may still finish after the local workflow is paused.

A useful interface might say that new runs are paused, two tasks completed, and one request is still awaiting confirmation. That is less tidy than a single “Stopped” badge, but it tells the truth about what remains.

Resuming deserves the same care. Decide whether it restarts unfinished work, retries failures, or begins only with new inputs.

Leave a record someone can use

A result summary should explain what the automation attempted and what is known to have happened. Keep links or identifiers for created items, separate skipped inputs from failures, and avoid putting secrets or unnecessary private content into logs.

Test the boring cases deliberately: a repeated trigger, an unavailable service, a missing permission, and a pause during a run. Those exercises tell you more about the workflow’s usefulness than another perfect demonstration.

The goal is not to remove every possible interruption. It is to make interruptions understandable, bounded, and recoverable. That is how a system earns the right to run with less supervision.