A lot of automations look impressive when everything works.
Something happens in one platform, an AI model does something clever with it, data gets moved somewhere else, and the result appears almost instantly.
The more interesting question is what happens when one of those steps fails.
Consider a relatively simple support request automation:
React form → backend API → queue → n8n → OpenAI → HubSpot
A user submits a support request. The backend validates it and sends it to a workflow which saves the request to a queue. A second workflow runs on a schedule, picks up waiting requests, sends them to OpenAI for classification, and creates tickets in the correct HubSpot pipeline.
The happy path is simple.
Making sure it behaves predictably when things go wrong is where the engineering starts.
A Failed Request Doesn't Mean Nothing Happened
Imagine the workflow sends a request to HubSpot:
Create ticket #123.
HubSpot receives it and creates the ticket.
But something goes wrong before the response makes it back to the workflow. Maybe the connection drops. Maybe the request times out.
As far as the automation is concerned, the request failed.
So what should it do?
The obvious answer is retry it.
Except HubSpot already created the ticket.
Now it exists twice.
It's easy to think of failure as binary:
It worked / it didn't work.
Once multiple systems are communicating, it's not that simple. What the automation knows happened and what actually happened aren't necessarily the same thing.
Reliable automation needs to account for that uncertainty.
Using a Queue as Persistent State
Every support request is written to a queue before the main automation processes it.
The entry contains the request itself, but also information about what has happened to it.
A new entry begins waiting to be processed.
The scheduled workflow selects those entries and attempts to process them. Once an entry has successfully made it through the workflow and into HubSpot, it's marked as processed and the HubSpot ticket ID is saved against it.
Future runs ignore processed entries.
This means the workflow isn't relying on an individual execution to remember what happened. The state survives after that execution finishes.
If something fails, the entry remains available for another attempt.
If it succeeds, it isn't touched again.
That makes retries much easier.
But retries introduce their own problems.
Retrying Without Making Things Worse
Before creating a HubSpot ticket, the workflow checks HubSpot to see whether that queue entry already exists there.
This deals with the awkward situation from earlier.
If HubSpot created the ticket but the workflow never received confirmation, the next run doesn't blindly create another one. It checks first.
If the ticket exists, the workflow knows the external operation has already happened.
If it doesn't, it can try again.
This is an idempotency problem: the workflow needs to be capable of repeating an operation without repeatedly producing the same side effect.
Without this check, retries can actually make an unreliable system worse.
Retries Need a Limit
Suppose the HubSpot credentials expire.
The workflow runs.
It fails.
Then it runs again.
It fails.
Then again.
Without some kind of boundary, the automation has become a machine for repeatedly sending requests that are never going to work.
Each queue entry therefore has an attempts field, beginning at 0.
When processing fails, an attempt is added.
The scheduled workflow only picks up entries below the retry limit. Once an entry reaches five failed attempts, automatic processing stops and the developer is notified that it needs investigation.
Transient problems get several chances to resolve themselves.
Permanent problems don't retry forever.
Failure Should Tell You Something
There isn't much value in a system that reports:
Workflow failed.
Why did it fail?
An authentication failure isn't the same as malformed data. Malformed data isn't the same as an unavailable external service. Different failures can require completely different responses.
HTTP requests therefore continue into error-handling branches when they fail. The response can be inspected, the reason recorded, and the appropriate action taken.
If authentication to an external service has broken, for example, repeatedly sending the same request isn't going to fix the credentials.
More importantly, whoever maintains the system should know that authentication is the problem.
The goal isn't just failure handling.
It's observable failure handling.
A workflow silently retrying something in the background isn't particularly useful if nobody can tell why it keeps failing.
Validate Both Sides of AI
Adding an AI model creates another boundary that needs to be controlled.
Before anything is sent to OpenAI, the system checks that the data going in has the structure it expects.
But checking the input isn't enough.
Suppose the model is being asked to classify a ticket into one of three departments:
- Technical
- Billing
- Sales
Technical is useful.
Customer Assistance might be a perfectly reasonable classification in normal English, but if the next part of the automation doesn't have a route for Customer Assistance, the workflow has a problem.
So validation happens on both sides:
Is the AI receiving valid data?
and:
Did the AI return something the rest of the system understands?
AI can make decisions inside an automation, but the systems surrounding it still need deterministic boundaries.
The Happy Path Is the Easy Part
The final workflow isn't particularly complicated.
Someone submits a form. Their request gets classified. A HubSpot ticket gets created.
The happy path could be built relatively quickly.
Most of the engineering exists around it.
- What if the API receives an array instead of the object it's expecting?
- What if a required field is missing?
- What if OpenAI returns an unexpected classification?
- What if HubSpot is temporarily unavailable?
- What if authentication fails?
- What if HubSpot creates the ticket but the workflow never receives the response?
- What if the same request gets processed twice?
- What if something keeps failing forever?
When everything works perfectly, most of the mechanisms described here do absolutely nothing.
That's the point.
If a run succeeds, its state is persisted and that request isn't processed again.
If a run fails, the reason is visible, its state remains available, and it can be retried safely.
If it keeps failing, eventually the system stops trying and asks for a human.
Getting data from A to B is automation.
Making sure the system still knows what to do when it gets stuck somewhere in between is engineering.