One of these is easy to read. That's not the same as being good.
I made both of these up to show you the shape of the problem. They're the two folders every screening committee has seen.
Two applications, one screening tool
Before anyone opens them: which one will the screen prefer?
Application one
Subscription analytics, fourth into the category
Application two
A process with no category to sit in
Familiar, and it shows its work
A subscription analytics tool in a category that already has four players. Companies it can point at that sold well. A market size with a footnote. Numbers that look like other numbers that worked out.
Everything a scoring tool wants, in the format it wants it.
Protected, and completely unfamiliar
A new process with a granted patent and something they know how to do that nobody else does. A young team, well trained, already backed by people who know them well.
There's nothing to compare it to. The category doesn't exist yet, so nothing anybody has ever written describes what happens next.
Looks great
High confidence. Which makes sense, because a machine that learned from a million business plans is going to feel good about a business plan that looks like a million business plans.
Looks risky
Low confidence, flagged. Not because there's anything weak in the folder, but because the folder has no relatives. Hard to recognize gets read as unlikely to work.
Ask it to judge, and what comes back is the past's opinion
Authors Farsan Madjdi and Bernd Wurth argue that a system trained on history hands out a kind of vote of confidence. How much that vote matters depends on whether it sits beside a decision or quietly becomes one. Their name for the setup, tools plus institutions plus habits, is a plausibility regime.
My everyday version, not the paper's: a mortgage application. The bank isn't asking whether you'll pay it back. It's asking how closely you resemble people who did. Someone self-employed with a better income and a lumpier record scores worse, and everyone knows the score is a proxy, right up until it becomes the decision.
One property, two opposite mistakes, depending on the job you hand it. Used to invent, it can talk up ideas that merely sound right. Used to screen, it can talk down anything it hasn't seen. The authors call these risks, not certainties, and the tool isn't the villain.
Farsan kept returning to what that leaves the founder holding.
… entrepreneurial judgment is becoming very important, precisely because the relevant evidence does not yet exist.Farsan Madjdi, in conversation
Some questions arrive before the evidence can
I pushed them on this first, because it's the first thing anyone asks about the paper. Wurth walked through why the usual method doesn't reach.
… the literature on experiments is very thin, and just because it's difficult to do.Bernd Wurth, in conversation
The reason is structural. You can't rerun a funding round, and no two applications, committees or years line up cleanly enough to compare. Section three gets into why, with jars.
A familiar shape if you've worked on any big messy system. Nobody runs a controlled trial on a city's transport network either. You build the argument, look for the traces it would leave, and go carefully. That's what this paper does.
One thing to keep straight: when the paper says the machine gets it wrong, it means it treated I don't recognize this as evidence of this won't work. A claim about the reasoning, not the outcome.
It's answering a different question than the one you asked.
You want to know if this will work. It's telling you how much this looks like things that worked. For a familiar venture those two are close enough to be useful. The further you get from the familiar, the further apart they drift.
The shelf it learned from
Try all threeClick an idea
Cycling on its own until you pick one.
How much this looks like things I've seen
This is the number that comes back. It moves.
Whether this is going to work
This is the number you wanted. It's blank, and it always was.
Slots straight in.
Thousands of near-identical plans on that shelf. The confidence is real information, but it is information about how crowded the category is. Not about whether yours wins it.
Close enough to reason about.
Enough of a match to borrow from, enough difference to matter. This is the sweet spot, and it's also where you're most likely to mistake a smooth answer for an informed one.
Nothing on the shelf to measure against.
The low score isn't a judgment about your idea. It's a report on the empty space around it. Same output either way, and nothing tells you which one you're looking at.
The teal line is the only thing the machine can actually measure. Both bars underneath would sit next to each other on a dashboard looking equally official, and only one of them was ever filled in.
The same machinery that rewards resemblance punishes its absence
There's no separate channel for "unprecedented and good." Two ventures get low marks and high marks for reasons that have nothing to do with each other, and the output looks the same either way.
Something that could look plausible … resembles what has worked before, while another venture can look implausible simply because there is little precedent for it.Farsan Madjdi, in conversation
The paper says this plainly: on well-trodden ground, resemblance is a reasonable thing to measure. It's real information about a market that exists. The trouble starts when it gets read as information about one that doesn't.
Imagine asking someone very well read who has never met your customers. You get a confident answer assembled from what other people wrote down. Worth something. Not worth one honest conversation with a customer, and much easier to get, which is the whole trouble.
This landed on me directly. I'd been reading the model's answer about my own ideas as a verdict. It was a distance measurement, and I'd confused the two for months.
Which raises the obvious follow-up: if distance is all it can measure, what exactly is it that can't be measured? Two jars.
There's not knowing the answer, and then there's the answer not existing yet.
This distinction is doing most of the work in the paper. I found it easier to picture than to define, so here are two jars.
One jar, two kinds of not knowing
Drag, count, then drawDrag a marble down into the jar. Drop one outside and it rolls away. The jar holds ten.
Add or remove marbles and the odds follow.
That's risk, and it's the easy one.
You don't know which marble comes out, but you know what's in there, so the odds fall out of counting. Loads of business decisions look like this. How many customers churn next quarter, with three years of history behind it. Which of three sites the new shop should go in.
Author Bernd Wurth used that second one himself as an example of where these tools genuinely earn their money.
If you have data sets about population density, what other stores are there and everything, AI will easily find you the top three locations …Bernd Wurth, in conversation
That's two weeks saved, and nobody in this argument has a problem with any of it.
That's uncertainty, and it's a different animal.
Same jar. You built it a minute ago. Now you can't see in, and here's the part that matters: something keeps topping it up while you play, so the mix drifts. Your last twenty draws tell you nothing about the next one.
That's the friction. No number to find, so nothing to be right about. You can't compute the odds or audit the decision afterward, because the thing you decided against has already changed. The economist Frank Knight described this a century ago, and a lot of new ventures live there.
Ask in 2007 whether people will let strangers sleep in their spare room, and you aren't facing an estimation problem. The answer hadn't been decided yet. Airbnb's founders didn't find it. They built it, coaxing people into trusting each other until the whole thing felt normal.
Airbnb is one of the paper's own examples, p. 4.Medicine gets a control group
Drug trials work because you can hold a group back and compare. That's how a fuzzy question turns into a number you can act on.
… that's why you do RCTs, like a gold standard in medicine, because you establish the counterfactual.Bernd Wurth, in conversation
An RCT is a randomised controlled trial: split people into two groups at random, treat one and not the other, and the gap between them is your answer. The counterfactual is the group you held back, the version of events where you didn't do the thing. It's the whole basis of knowing that a drug works.
Funding decisions don't
You see something once and your judgment has already moved. Present it differently and you get a different answer. No two founders are the same input, and no two panels are either.
You can't give an investor the same venture over and over.Bernd Wurth, in conversation
So the founder has to spend money before anyone can know. That's the job.
… entrepreneurs have to commit resources before the relevant evidence exists.Farsan Madjdi, in conversation
A big hump, and a long thin tail.
The paper has two figures worth your time and they're really one thing. The curve shows where the money goes. The table sorts ventures by where they sit on it. Click a part of the curve.
Where you sit on the line, and what follows from it
Fig. 1, p. 4 and Table 1, p. 5Click part of the curve
Cycling on its own until you pick one.
Plenty like it, and the data knows how those went.
- What it looks like
- Stable, repeated patterns that are well covered in what's been written. You really can work out how it's likely to go from things that already happened.
- Which tools
- Prediction, scoring, forecasting.
- What goes wrong
- Mild overconfidence. A close comparison gets treated as a promise rather than a hint.
- Who ends up funded
- Ventures that fit the existing criteria, efficiently. Little effect on the range of what gets backed.
- What to do
- Use it as an aid. Show how confident it is and how close the comparison really was. Flag where the match is only partial.
Genuinely new, but you can still find something to hold it against.
- What it looks like
- Real novelty, with partial analogies available. Comparable markets, visible demand, something to reason from.
- Which tools
- Generative models, analytics, market intelligence.
- What goes wrong
- False positives. Something that hangs together gets mistaken for something that's understood.
- Who ends up funded
- More ideas get considered, but attention can drift back to familiar recombinations rather than departures from them.
- What to do
- Treat what it produces as a provocation, not an endorsement. Make somebody establish the feasibility separately before anyone commits money.
Nothing like it, and somebody has to make the call.
- What it looks like
- Whether it works depends on experiments nobody has run, rules that don't exist yet, or a category that has to be built. This is where the rogue ideas live.
- Which tools
- Prediction and screening, where the miscalibration is worst. Generative models are usable here only for provocation, never for a verdict.
- What goes wrong
- False negatives. Hard to recognize gets read as unlikely to work.
- Who ends up funded
- Where the score gets the last word, not these. They lose credibility and face a higher bar, and the tail thins out.
- What to do
- The score doesn't decide. Carve out room in your screening for things with no comparison set, and put a human argument next to every number.
And then the score stops advising and starts deciding.
This is the dashed line, and it's the paper's sketch of a risk rather than a forecast. Same ideas, same budget, different shape. The peak grows because coherent-sounding plans get talked up, the tail thins because unfamiliar ones get talked down, and the total underneath barely moves. Which is why it's easy to miss.
- What it looks like
- Scores baked into funding, accelerator intake, hiring or whatever gets shown to whom, whatever kind of venture is being judged.
- Which tools
- Rankings, recommenders, automated screening, decision-support pipelines.
- What goes wrong
- Everything above, amplified. The number stands in for the judgment instead of feeding it.
- Who ends up funded
- The recognizable, across every category at once, unless something is deliberately protecting the range.
- What to do
- Cap what the score is allowed to weigh. Require a human to sign off on any low-precedent rejection. Audit the spread of what you funded, once a year.
The right-hand column is the authors' own advice, not mine. Author Farsan Madjdi's version of what the dashed line does:
… generative AI can expand the search space while … anchoring your imagination to historically available material.Farsan Madjdi, in conversation
Which changes what, exactly? Not the number of ideas.
… not simply more or fewer ideas … change the composition of the ideas that appear plausible.Farsan Madjdi, in conversationCurve redrawn from Fig. 1, p. 4. The four cases and the closing column are Table 1, p. 5, put into plainer words.
Everything in the paper hangs off one measurement
How much has already been written about the thing you're proposing. That's the shelf gap from earlier, the x-axis here, and what sorts the four cases. It says nothing about whether your idea is good. How much it decides depends on how much weight the room gives the score.
The authors treat this as a risk, not a rule. The symptom isn't a smaller budget. It's that everything you fund starts to look alike. Which brings up the obvious objection.
Somebody still has to say no.
Here's the pushback I'd expect at a board table, and the reason I don't think it lands where people think it does.
Selection is the job
Money runs out. Every round turns down most of what comes in, and it always did. If the unfamiliar applications get turned down more often than the familiar ones, that might be nothing more sinister than a committee doing what committees are for.
There's a stronger version of it too, and you'll hear it in the room. Most new ventures fail. Most of the strange ones fail harder. A screen that's tough on things nobody has tried is being sensible, not narrow-minded.
Two problems with that
The first is uncomfortable, and I only noticed it while writing this up. "Most of them would have failed anyway" assigns odds to the exact set where the argument is that you can't. Nobody has that denominator. Nobody tracks what happens to unfamiliar ideas that get turned down, and the fate of the never-funded isn't observable anyway. The claim feels like evidence and is really a feeling of familiarity, the same move the machine makes.
The second: the paper isn't disputing any particular no. It's asking whether the reason behind the no has quietly become the same reason everywhere. One committee reaching for what it recognizes is judgment. Every committee reaching for the same thing, because they consult tools trained on the same past, is something else.
You can't run the study. You can instrument your own room.
The paper closes by naming the research it wants done: watch whether money drifts toward familiar-looking ventures, and whether the spread of what gets funded narrows where these tools are built in. A hard study to run, for all the reasons above.
You don't need a literature to see your own desk. Log the decisions. Note when a human overrides the score, and why. Check the spread of what you funded once a year. That's paperwork, not a research program, and it answers the question for the one room you control. The draft policy at the bottom does exactly that.
Four applications. Pick one to fund.
Two of these are famous, which is the trap. The fourth one is why the exercise is honest.
What the score can and can't see
Have a goThere's so many things that changed and that had to change for Uber to be successful.Bernd Wurth, in conversation
Pick one to fund
Four folders, no outcomes attached.
What you get
Whatever a precedent-trained score can see in each one, and whatever it can't.
What you won't get
A right answer. Two of these are famous and one exists to stop that mattering.
Nothing to compare it to, and a social rule in the way.
What the score sees
A business that needs people to host strangers overnight, in a market with no equivalent and legal trouble waiting in every city. The safest thing it can suggest is that you make it more like a hotel booking site.
What it can't see
That the rule was about to change, partly because of this. The paper's account is that the founders had to build the trust, make the practice feel normal and drag a whole category into existence. Nobody detected that it would work. They went and made it work.
The energy industry thought it was a hobby.
What the score sees
Power gets made in big expensive plants. A village-owned turbine has no match anywhere in the operating data and nowhere obvious to sit in the system.
What it can't see
That the people behind the Danish wind industry did the boring institutional work alongside the engineering: the rules, the grid connections, the ownership models. All of that had to be invented too.
The one that scores best.
What the score sees
Comparable exits, a citable market, numbers that rhyme with companies that worked. It all lines up, and the coherence is real.
Something I wondered, which isn't in the paper
Being easy to recognize might cut both ways. A pattern clear enough for a machine to spot is clear enough for four other teams to copy. The authors don't make this argument and I've got no evidence for it. File it as a question worth asking.
The control. Nothing like it, and it doesn't work.
What the score sees
No match, no analog, weak signal. Exactly what it saw in A and B.
Why this is the important one
The machine was right this time, and nothing in the folder told it apart from the other two. That's the shape of the problem. The score can't separate "new" from "quietly doesn't work," and neither can you, not reliably.
And nobody knows how often D happens. Unfunded ideas leave no record of what they would have become. You can't even say D is the common case, which is why any policy here has to work without knowing.
Everything changes when the score stops being a suggestion.
Same output. Different spot in the process. Completely different consequences.
Sitting beside you
It widens the search. Author Farsan Madjdi took the paradox head on: the range of ideas you consider genuinely grows, through combinations you wouldn't have reached. What shifts is which ones feel worth taking seriously.
Sitting at the door
The same output inside an investment committee or an accelerator's intake becomes the thing that decides, rather than a thing that informs.
… you're being pushed towards the consensus …Bernd Wurth, in conversation
The past is the training set
This is where the bias conversation stops being a separate topic. If evaluation has historically favored certain founders and certain ways of writing, that preference is sitting in the data.
AI is trained on all the past.Bernd Wurth, in conversation
Asking nicely doesn't fix it
Author Bernd Wurth's point is that telling a system to be fair doesn't make it fair, because the habit is in what it learned rather than in what you typed.
Just by asking AI, you tell AI to be impartial. It's not automatically impartial because these biases are in there.Bernd Wurth, in conversation
So "pushes toward consensus" and "repeats old preferences" turn out to be the same behavior with two names, and an unusual founder can get caught by both at once.
Every application gets better
This is my first question, and the first half of the answer is yes. You use it to sharpen a grant application and your odds go up. I've felt that. Then Farsan asked the thing I hadn't.
… if everyone else is doing it, what does that then actually mean? So where do we end then?Farsan Madjdi, in conversation
And the signal goes flat
Once every application is buffed by the same sort of tool, buffing stops telling you anything. Bernd mentioned the parallel in research funding, where one proposal is to stop ranking above a quality bar and draw lots instead.
… above that, it basically becomes a lottery because it's so difficult and there's so much variation, like who your reviewer is …Bernd Wurth, in conversation
Hiring looks to be running the same experiment right now. Written by a machine, screened by a machine, and a stack of good candidates who all read the same.
What a reviewer sees after a year of this
Round one, before any of this. Three applications came in. Pick the strongest.
Round two, a year later. Five came in. Pick the strongest.
None of this is automatic
If you take one thing from the paper, take this. The authors don't argue these tools are restrictive by nature. The claim is conditional: it turns on what you use the tool for, how legible your venture is in prior data, and how much authority the number gets.
This effect is conditional rather than inevitable, where AI outputs are treated as advisory, variance may be preserved or even widened, whereas where they become authoritative gatekeeping mechanisms they redistribute legitimacy toward historically validated trajectories.Madjdi and Wurth, p. 4
Read the middle clause twice. Treated as advice, the variety of what gets funded may hold steady or even widen; the narrowing shows up at the door, not in the tool. The paper is just as direct the other way: where a venture is legible in prior data, AI “may usefully augment judgment,” and generative systems “may expand the range of ideas entrepreneurs consider.”Quoted phrases, p. 3. The conditional statement, p. 4.
So my first answer: it helps, it helps everyone, and an advantage everyone holds cancels out. The last thing it does is close the gap between an application that reads well and one that reads badly. If your edge was writing better than the next person, that edge is going.
So how do you tell brilliant from daft?
This is the question I came into the call wanting answered. Both cases look identical from outside: no precedent, no comparison, nobody's done it. Neither author pretended it resolves cleanly, and I respected them more for saying so.
Evidence you go and make
You can't run the trial, but you can run a pilot. A small group of real people who actually paid. Letters of intent. Recordings of customers describing the problem in their own words. None of that is in any training set, which is precisely why it's worth something.
The people, and what they've already pulled off
Author Farsan Madjdi turned the question back on the institutions, and it's the one I'd most want an incubator director to sit with.
Should incubators, accelerators emphasize the idea that potential applicants come up with, or … the profile of applicants … people who show or have agency and who are capable of execution?Farsan Madjdi, in conversation
Now that anyone can have their idea written up beautifully, proof that someone gets things done is proof a machine didn't generate.
The bit where somebody just has to have the idea
Underneath all of this is the leap: coming up with the hypothesis before there's anything to check it against. You can't calculate your way to an idea nobody's had. Somebody has to have it, and then the testing starts.
You do a small version of it constantly. Your car makes a noise on the highway and before any diagnostic runs, you already have a guess about what it is. The guess came from somewhere, it isn't a calculation, and the mechanic will confirm or kill it later.
The two ledges above are the difference in miniature. Left, a cab you can book ahead: easy to explain to a committee, harder than it looks because the market already exists. Right, the thing that worked, which needed a rule about strangers to change first. You can walk to the left idea. You have to jump to the right one.
Farsan's example was Steve Jobs at Xerox PARC, the research lab where much of the modern computer interface got built by a company that never worked out what it had. The technology was sitting there in the building. Seeing what it was for was a separate act entirely.
His wider point was that the things that move a society along have rarely arrived with the evidence already assembled.
It worked based on visions, on imaginations of entrepreneurs … They weren't in line with these plausibility signals and this expert consensus. These people actually had to go around it, or they even had to go against it.Farsan Madjdi, in conversation
Which raises something we chewed on for a while, and it's speculation rather than anybody's finding: how much one of these tools can help you spot something new might depend on how much you already know. It's not much use being hit on the head by an apple if you're not Newton.
My second question, more or less answered. These tools hand you the adjacent thing, the ledge you can walk to, and they can widen the search. They can't make the jump for you. And if you don't carry enough of the domain to recognize the far ledge, a wider search just gives you more places to stand on the near one.
A score tells you what the past makes of your idea. It can't tell you what the future will do with it.
Don't ask it for a verdict. Ask it for arguments.
The practical suggestion from the interview, and then a draft policy you can scribble on.
Stop asking one machine one question
Author Farsan Madjdi's instinct was to treat it the way you'd treat money.
… potentially you would use like a portfolio in a way. It's kind of like … how you should invest.Farsan Madjdi, in conversation
Author Bernd Wurth made it concrete by pointing at something a lot of us already do daily, in a completely different context.
… when you use them for coding … they do the implementation, they do a review and then they typically do an adversarial review. So they purposely try and find errors.Bernd Wurth, in conversation
Nobody ships code on one pass from one model. That's what most of us do when we ask a model about a business plan.
Four reviews of one unfamiliar application
Open each oneClick to open
Believe every word. Does it hold together?
Do the claims agree with each other? Do the numbers agree with the claims? It's the cheapest genuinely useful thing these tools do, and it's a tidiness check. Passing means the application isn't muddled. That's all it means.
Now assume everything they predict comes true.
Push it as far as it goes. If the world moves the way they say it will, what is this thing? Are we funding it at the right size, or should we be doing a great deal more? This is the pass that stops you under-reacting to something big.
Now try to knock it over.
Bernd put it as a question you ask deliberately, one assumption at a time.
What if individual assumptions don't hold? Do you still have a business, or is the whole thing falling apart?Bernd Wurth, in conversation
In the Uber case that assumption is people getting into cars driven by strangers. Naming it tells you the cheap experiment you should run before you write a big cheque.
Is this a new field, or a graveyard?
There's nothing to compare it to. Fine. But is that because nobody has tried, or because plenty tried and nobody wrote it up? Opposite answers, same score. You have to ask this one on purpose, because nothing else in your process will. This pass is my addition, not theirs.
Four reports, possibly all agreeing, and the call is still yours. If you still think there's something in it, that's not a bug. That's the part of the job nobody was ever going to do for you.
Neither of them thinks you should stop using LLMs
Both said so directly and more than once. The argument is about where the output sits, how much authority you hand it, and whether anyone has bothered to check what it can't see.
… it's helpful to compare it to certain patterns, but that shouldn't be the only criteria.Bernd Wurth, in conversation
The paper's acknowledgements say the authors used Claude to help with copyediting and checked every change themselves. Which is, more or less, the policy the paper recommends, applied to the paper.
I went in with two questions about my own work. Both answers were smaller, and more useful, than the ones I'd imagined.
On whether these tools help my funding odds: yes, and so does everyone else's copy of them. The lift stops being an advantage the moment everyone has it. What survives is the part I can't outsource, so I've stopped polishing and started collecting: a pilot with people who paid, customers describing the problem in their own words, the assumption that would sink the whole thing written down before anyone asks.
On whether they blunt my ability to find something new: not exactly, but they pull. Generous with the adjacent idea, quiet about the one nobody has tried. The quiet is easy to mistake for a verdict. It's a report on how much has been written, and for anything genuinely new that's close to nothing.
So I've changed what I ask for. Not whether an idea is good, because it will say yes and I'll believe it, since I wanted to. I ask it to take the idea apart. Then I go find the thing the past couldn't tell me.
The uneasy feeling hasn't gone. It's gotten more specific, which is the most I can ask of an hour with two people who've thought about this longer than I have. I know what I'm reaching for the tool to do now, and what it was never going to do.
Who did the work
Two researchers at Glasgow who came into academia from different directions and landed on the same question.
Farsan Madjdi
Assistant Professor Adam Smith Business School, University of GlasgowSpent about twenty years in industry first, in management, business development and innovation roles, which he describes as a long run of dealing with what comes next. He now researches how founders spot and judge opportunities, how their identities shape what they chase, and how social ventures get built.
ORCID 0000-0002-2187-6568Bernd Wurth
Senior Lecturer Adam Smith Business School, University of GlasgowTrained as an engineer and works on complex systems, building simulations of how startup ecosystems behave. His route to this paper, in his own words, was a windy road: systems work, then conversations about AI, then what these tools are quietly doing to the way decisions get made.
ORCID 0000-0002-6688-4270