AI that holds together outside the demo.

• An AI feature that works in a demo and has not been trusted with real users yet.
• An agent that needs to take actions in real systems, where the question is what it must never be able to do.
• A workflow with genuine repetition in it, where the judgment stays with a person and the mechanics should not.
• An existing AI surface that changes behaviour unpredictably when a prompt or a model is updated.
• A product where retrieval quality, not model choice, is what actually decides whether the answers are good.

A reference architecture, meaning the shape the work is designed to rather than a record of systems delivered. It names layers and responsibilities and never products: what runs at a given layer is a decision taken with you during scoping, not one taken here in advance.

• Agent and workflow orchestration: the steps, the state between them, and what happens when one fails.
• Explicit tool and permission boundaries: what the system may read, write and call, decided and written down.
• Context and retrieval design, built against the questions the system actually has to answer.
• An evaluation plan and harness: the cases it is measured against, and the procedure for checking a change before it ships.
• Guardrails and defined failure paths, including what the system does when it is not confident.
• The data movement between the systems this touches, with the boundaries the data crosses made deliberate.

• Shipping a change to an AI surface on the strength of it looking better in a demo.
• An agent that can reach a system nobody intended it to reach, because it inherited a credential rather than being granted one.
• Behaviour that changes without warning when a model version moves underneath you.
• A failure path discovered by a user rather than designed by an engineer.
• Automation that moves the work somewhere else instead of removing it.

• A demo intended to raise money rather than serve users. It is a legitimate thing to want and it is not what this is.
• Model training or fine-tuning as the headline. The work here is the system around the model; where a fine-tune is genuinely the answer, that gets said during scoping.
• An AI feature with no defined success condition, where nobody can say what a good answer looks like. That is a product question, and it comes first.
• Replacing judgment that should stay with a person. Automation is aimed at the repetitive mechanics, not at the decision.
• Anything that needs a benchmark number published before the system exists to measure.

1. 01

   The evaluation plan, and the harness that runs it

   What the system is allowed to do, what it must never do, the cases it is measured against, and the procedure for checking a prompt or model change. Your team runs it after we are gone; that is the point of it.

2. 02

   The permission model, written down

   What every agent, job and integration may read, write and call, and why. A control that can be traced back to its reason can be reviewed. One that cannot gets guessed at and eventually widened.

3. 03

   Prompts and context in source

   Versioned in your repositories alongside the code, not pasted into a console. A prompt is a behavioural decision and it belongs under the same review as any other one.

4. 04

   The known limitations

   Where the system is weak, which inputs it handles badly, and what was deliberately left out of scope. Stated plainly, because the alternative is your team rediscovering each one.

• DocumentsSample artifactsThe AI evaluation plan is one of the eight formats, shown with what it records and the risk it removes.
• BuildA natural-language analytics systemThe data and inference build: carrying raw events to a number worth acting on without failing silently.
• ServiceSecurity and systems depthWhere the agent boundary work continues: least privilege, credential handling and the blast radius of a mistake.