OXHN Partners / Decision resource

Print or save this page as PDF. Download the blank CSV records below; no signup or upload is needed.

Manual template, existing AI tool, or custom workflow?

Compare the whole job, not just the draft.

A tool can produce a fast draft and still leave your team with more work. Before commissioning a custom workflow, compare it with a clearer manual template and an existing suitable tool. Use the same inputs, the same review standard and the same definition of “finished.”

The example below is entirely hypothetical. It contains invented effort and cost assumptions, not client data, tool-test results, an OXHN quote or a savings forecast. No AI system was run for this comparison.

Start with the result you need: for example, a requirements list a proposal lead can check against a brief and its amendments. A list with fluent wording is not finished if it omits an obligation, invents a deadline or hides a conflict.

Inspect the synthetic requirements-review example for the source-to-output task. This comparison does not resolve that example's open source dependencies.

Step 1 — Give simpler alternatives a fair chance

“Buy” can mean a subscription you already have. Record only incremental costs in this comparison and identify any existing subscription separately. Free software can still require model usage, staff time and maintenance.

Could you continue without this tool?

Before committing, try a small exit rehearsal with material you are allowed to use. A download is not enough: can another responsible person inspect the work and continue the task in the intended alternative?

• Recover the useful record. Export or copy the output, source references, unresolved questions and review decisions needed for the task. Record anything the tool cannot preserve.
• Open it elsewhere. In an approved document or spreadsheet environment, check that the content is readable, source references still identify the supporting material, and unresolved items remain visible. Use permitted source copies where a link depends on the original account.
• Continue one case. Have the responsible person complete the next review or handoff using that record and the manual fallback. Record the time, missing information and any reconstruction needed.

An exported output does not recreate the original model, instructions or integrations. Moving to another AI tool requires a separate evaluation; a successful file export does not establish equivalent performance.

Compare the value of the tool with the effort and limitations of leaving it. Some dependency may be worth accepting; custom software also has dependencies. Record the tradeoff, owner and evidence that would trigger reconsideration. Use the existing trial-record notes to keep the rehearsal findings separate from normal case-running effort.

Basis: GOV.UK, Managing technical lock-in in the cloud, checked 16 October 2026. The exit rehearsal is OXHN's practical adaptation for document workflows, not a certification or a government requirement for your team.

Step 2 — Apply a quality gate before the money comparison

Agree what would make an output unusable before seeing which approach is fastest.

For the linked teaching task:

• Unsupported deadlines, hidden contradictory limits or false readiness approval fail the gate.
• Missing requirements and unsupported additions require correction before use.
• A source reference must support the claim; a working link alone is not enough.
• Check the entire source packet for omissions, not only the output's citations.
• Open source questions stay open until a responsible owner receives appropriate evidence.

Keep the raw draft and the corrected final separately. Record whether the raw draft failed, whether correction was possible, and how much work correction took. A slow but correct manual result is not defeated by a fast unusable draft.

Passing a small trial only supports the next bounded decision. It does not establish production reliability or eliminate human judgment.

Step 3 — Count all the active work

For each case, measure:

Active minutes = prepare inputs + operate/generate + verify + correct + handle exceptions + hand off.

Measure elapsed turnaround separately, including queue and approval delays. If a model runs while a person does other work, record its machine time separately; do not double-count it as human labor. If multiple staff work at once, sum their person-minutes for labor and retain wall-clock time as a separate measure.

Count abandoned runs, retries and manual fallback. Label unmeasured fields “not measured,” not zero. If corrections are already inside measured case time, do not add the same correction expense again.

Then record one-time setup/training and recurring maintenance outside per-case labor. Record direct incremental software/model costs separately.

Step 4 — Work through one hypothetical comparison

Assume:

• 40 eligible briefs each month; the unchanged process uses 70 active minutes per brief.
• 30 briefs/month use whichever candidate is being evaluated; the other 10 remain at 70 minutes. The same 30-case volume applies to all four alternatives below for comparability.
• Labor is valued at a hypothetical $60/hour capacity equivalent.
• Per-case active minutes include preparation, verification, correction, exceptions and handoff.
• Monthly maintenance is additional staff labor, not already included in per-case minutes.
• Setup hours include configuration/integration, initial evaluation, training and handoff. No external setup fees or hardware purchases are assumed; add them if applicable.
• Monthly cash is incremental software/model expense, not maintenance labor.
• All cases are assumed to reach the agreed quality standard for this arithmetic only. Actual gate outcomes remain unmeasured.
• The year has 12 full months at the stated operating level; no ramp, discounting, inflation or changed case mix. This simplifying assumption can overstate early benefit.

Every entry is an invented scenario input, not a vendor benchmark.

Formulas:

• Net monthly hours released = assisted cases × (baseline minutes − assisted minutes) ÷ 60 − maintenance hours.
• Monthly capacity equivalent after recurring cash = net hours × hourly value − incremental monthly cash.
• First-year capacity equivalent after setup = 12 × monthly equivalent − setup hours × hourly value − any additional one-time cash costs.

This is a comparison of labor capacity valued at an assumed rate and cash outlays. It is not cash profit, realized ROI or a payback promise. If released hours cannot be used productively or reduce actual expense, their financial benefit may be zero while tool and setup costs remain.

Example calculation for the adapted workflow: 30 × (70 − 40) ÷ 60 − 4 = 11 hours/month. Then 11 × $60 − $80 = $580/month. Across 12 months, $580 × 12 − 60 × $60 = $3,360.

What this says—and does not say: under these invented assumptions the adapted workflow has the highest first-year capacity equivalent, but only by $480 over the improved manual template. The custom option has the fastest assumed case processing but does not recover its larger setup/operating burden in the assumed year. That narrow $480 margin is a reason to test the uncertain inputs, not a recommendation to buy or build.

Step 5 — Change one assumption and see the comparison change

Keep every other input unchanged but halve the assisted volume to 15 briefs/month. The other 25 of the original 40 remain on the unchanged process.

The manual template now leads. Fixed maintenance and setup costs matter more when fewer cases benefit.

At 30 assisted cases, just two extra active minutes per adapted case reduce its first-year capacity equivalent by 30 × 2 ÷ 60 × $60 × 12 = $720. Its assumed $480 advantage disappears. That is why review time must be measured rather than guessed.

None of these economic comparisons overrides a failed quality gate. If the adapted approach cannot preserve source meaning, its apparent capacity advantage is irrelevant until the defect is corrected and retested.

Step 6 — Run a small comparison you can learn from

1. Name the decision and owner. What specific evidence would justify keeping the template, configuring an existing tool, adapting it or stopping?
2. Make a reference before comparing candidates. A responsible reviewer identifies required items, supporting source locations and unresolved questions. Resolve reviewer disagreements explicitly; do not hide them in a score.
3. Separate configuration from evaluation. Use different examples to tune instructions and to assess the final candidate. Include ordinary cases and the variations you actually see: amendments, missing attachments, tables or poor scans. If that case mix is unknown, measure it instead of inventing weights.
4. Keep the task comparable. Give each candidate the same input pack and completion standard. Have comparable operators perform the work; record experience and tool versions. Rotate order or use independent matched cases to reduce learning effects. Reusing the exact packet with the same person can make later approaches appear faster simply because they remember the answer.
5. Log the whole attempt. Include failed/abandoned cases and fallback work. Save the actual output, source references, gate outcome and person-minutes. Note any drift in task scope.
6. Look at case-level results, not just an average. Show critical errors, omissions and unsupported additions separately; review normal and exception cases separately. Small samples are diagnostic, not a precise population estimate.
7. Make a bounded decision. Continue only when relevant quality and effort conditions hold. Revise the specific weak point, or stop if the template works well enough. Set the next review date and who owns maintenance.

A practical planning example, not a statistical guarantee: begin with 20 evaluation packets covering the workflow's known variations, separate from configuration examples. Four approaches would require 80 case-runs, plus reference preparation, operator onboarding and adjudication. If the average run took an assumed 45 active minutes, case-running alone would take 60 person-hours; it is not a 15-minute “test.” Screen out unsuitable options before the full comparison, reduce the initial scope when resources are limited, and report the smaller evidence base honestly.

If evidence is insufficient, the correct decision can be not enough information yet. Do not invent a success threshold after seeing the preferred candidate.

Can AI grade its own work?

An AI-generated score is another output to check—not independent proof that a workflow is reliable. Start with a responsible person's reference decisions and a clear rubric. Use simple checks for things such as required fields; use source review for meaning, omissions and unresolved conflicts. A field being present does not mean its contents are correct.

Before relying on automated grading, check it against examples a reviewer has already judged, including acceptable outputs and deliberately flawed ones. In our synthetic requirements-review example, an invented deadline or a concealed page-limit conflict must fail even if the rest reads well. These are hand-written teaching errors, not observed model failures.

Record where the grader and reviewer disagree, inspect the source evidence and reconcile the judgment. Keep unresolved judgments visible. If you adjust the rubric or grader using those examples, check the revised version on separate examples before relying on it. An automated “pass” does not close an unresolved source question or replace the responsible professional's approval.

Is one good run enough?

No. A generated result can change between attempts with the same inputs. Before testing, decide which cases to repeat, how many attempts to retain and how failures affect the decision. Keep the inputs and configuration version fixed for that comparison; save every attempt rather than selecting the best-looking output.

For example, three attempts on each of two selected cases produce six attempts, not six distinct cases. This is an illustration of recordkeeping—not a minimum sample size or reliability guarantee. Give each attempt a unique trial-log ID; record its shared source-case ID and attempt number in the notes. Keep raw and corrected outputs linked and count only the additional work at each stage.

Report both how many distinct cases were covered and how many attempts passed, failed or remained unresolved. Include abandoned attempts and retry effort. Repeating one easy packet does not replace testing different kinds of work, and one passing retry does not erase an earlier critical error. Use the evidence to make a bounded go, revise or stop decision—not a claim of production reliability.

Method references: OpenAI, Evaluation best practices and Anthropic, Demystifying evals for AI agents, checked 16 October 2026. The recordkeeping example is OXHN Partners' editorial illustration, not a measured result.

Take the comparison to your team

These original, ungated CSVs open in a spreadsheet or text editor. The trial log and assumptions record contain headers only: no observations or outcomes are prefilled. Keep a separate assumptions row for each option and scenario; identify the measurement period, owner and evidence behind each input.

• Download blank trial log (CSV)
• Download blank assumptions record (CSV)
• Download hypothetical inputs — not tested (CSV)
• Download record definitions and formulas (text)

Use the records without losing the caveats

Use non-identifying case IDs, not client names. Source packs, drafts and corrected outputs stay in your team’s approved environment; this site accepts no uploads and does not store your records.

Record quality_gate as pass, fail, unresolved or not_assessed; a blank cell is not a pass. Use separate linked rows for raw and corrected outputs, mark the stage in exclusions_or_limitations, and count only the additional labor at each stage. Record missing observations as “not measured,” not zero. Zero critical errors in 20 cases does not guarantee a zero error rate.

Active person-minutes include prep_min + operate_min + verify_min + correction_min + exception_min + handoff_min, plus fallback_min_not_already_counted only when it is additional labor. Exception and fallback time must not overlap. Machine time and end-to-end elapsed time are separate; neither is automatically human labor. Sum concurrent staff person-minutes, not just elapsed minutes.

For assumptions, input_status is measured, estimate or hypothetical. Record mixed evidence or exclusions in evidence_reference; split scenarios when assumptions change. Direct cash uses a consistent currency (USD in the hypothetical inputs). Record existing subscription costs separately and count only incremental cash in the formulas. Every hypothetical row is invented, not client data, an OXHN quote, a forecast or tool-test evidence. No AI system was run.

The workflow-fit worksheet helps scope the decision; the synthetic requirements-review example shows a reviewable output and unresolved source dependencies.

Discuss a bounded workflow pilot: scope and evaluation. A non-sensitive description is enough. Do not upload client documents through this resource.

Planning for changing AI capabilities? Use the Super Intelligence preparedness decision brief to separate present evidence from future scenarios.