Setting up a proof of concept: what it must prove
An AI proof of concept is not a small build, it is a test of a hypothesis. It answers a question that must be sharply phrased before the start: does what we suspect actually work on our real data and in our real flow. The most important lever is the pair of success criterion and abort threshold, both fixed before the first move. The success criterion says when the assumption is proven. The abort threshold says when you stop, instead of throwing good money after bad. Testing happens on real cases, not on toy data, otherwise the PoC only proves that something runs in the lab. At the end there is not another meeting but a yes or a no.
What a Proof of Concept Must Prove
A proof of concept is not a shrunk product. It is a test, and a test has a question. Before anyone builds anything, that question must fit on one sentence: We believe AI reads the amount off our supplier invoices reliably enough to replace the manual check. That is an assumption that can hold or fall. That is the whole point.
The most common mistake is to start the PoC without that one question. Then a team builds something that works in principle, and at the end nobody knows whether that was a yes. A PoC that shows everything a little proves nothing. A PoC that tests one assumption hard delivers a decision.
So the first step is not technical. It is a wording: which single thing must be true for the further path to be worth it. If you cannot say that question in one sentence, you do not yet have a PoC, you have a statement of intent.
Success Criterion and Abort Threshold
Once the assumption stands, it needs two numbers or two clear statements, fixed before the build. The first is the success criterion: the point at which the assumption counts as proven. The second is the abort threshold: the point at which you stop, because the case does not hold in this form.
The abort threshold is the part almost everyone forgets. Without it, a PoC runs on as long as budget lasts, because there is always one more knob to turn. With it, an open experiment becomes a test with an ending. A PoC you cannot let fail is not a test.
A proof of concept without an abort threshold is not a test, it is a project that is never allowed to say no.
Both limits belong to the real task, not to a gut feeling. If one person checks eighty invoices a day today and barely makes mistakes, the question is not whether the AI is perfect but whether it is good enough to change that work meaningfully. The success criterion hangs on the baseline, the measured current state. Without a baseline, every number floats in the air.
Real Conditions, Not the Lab
A PoC only proves something if it runs under real conditions. Real data, real exceptions, the real flow around it. Tested on neatly chosen examples, a PoC proves the tool works in the lab, and that was rarely the question. The question was whether it copes with your actual, messy cases.
This is exactly where a defensible PoC parts from a pretty demo. The demo shows the normal case, polished and fast. The defensible PoC deliberately hunts the hard, rare and unpleasant cases, because that is where the truth sits. What fails on the hard case while it is cheap costs many times more later in operation.
| Vague wish | Testable success criterion | Abort threshold |
|---|---|---|
| The AI should read invoices | amount correct on 95 of 100 real invoices | below 80 of 100, then stop |
| The assistant should answer requests | the ten most common questions correct with a source | no reliable source possible |
| The classification should save time | faster than the baseline at equal hit rate | slower or less accurate than today |
| The extraction should be clean enough | exceptions clearly flagged as uncertain | errors silent, without a warning |
The right column matters most. It forces you to name failure in advance, and that turns a wish into a test.
The Setup Checklist
Before you start a PoC, answer these five points. Get stuck on one and you have found the reason the test would not deliver a decision later.
- The one assumption: does it fit on a sentence, checkable and not vague?
- The baseline: what does today's flow look like measured, in time, volume and quality?
- The success criterion: from what point is the assumption proven, in numbers against the baseline?
- The abort threshold: from what point do you stop, instead of tuning further?
- The real cases: are there enough real, including hard, examples, cleared and usable?
Core sentence: a PoC is in small part a build and in large part a matter of wording. What you fix as a criterion beforehand becomes a decision at the end. What you leave open becomes another meeting.
Common Misconceptions
- A PoC is a small version of the finished product.
- No. A PoC is a test of an assumption, not a scale model. It may be ugly and leave a lot out, as long as it tests the one question cleanly.
- If the PoC runs, it can go into operation.
- Rarely. A PoC proves the assumption holds. Between it and operation lie permissions, monitoring, an error path and named accountability. The PoC decides whether that path is worth it, it does not replace it.
Questions and Responses
- How does a PoC differ from a pilot?
- A PoC tests an assumption, often outside live operation, on a clearly bounded case. A pilot already runs a solution in real operation, bounded in time and scope, to see whether it holds in daily use. The PoC comes first, the pilot builds on a PoC that passed.
- What if we do not know the success criterion beforehand?
- Then that is the real first task. A missing criterion usually hides a missing baseline: you do not yet know exactly how good today's flow is. That can be measured, and the measurement becomes the criterion. Without that step, any PoC stays arbitrary.
- Is a failed PoC wasted money?
- No, if it was cheap and decided early. A PoC that returns a clear no after two weeks has spared you a project that would have failed on the same point after months. That is precisely what the abort threshold is for.
Sources and Evidence
No external statistics or client results are asserted as verified. The example numbers in the tables are chosen for illustration and do not replace a case-specific definition.