What an AI readiness assessment actually covers

Most AI proposals arrive with a model already chosen. By the time the conversation reaches data quality, governance and integration, the expensive decisions have been made in reverse.

An assessment runs before any of that. Its job is to establish whether a problem is genuinely suited to machine learning, and whether the institution can defend an automated decision when challenged.

Start with the decision, not the data

Ask one question first: what decision are you trying to change, and how would you know it worked?

If the answer is a metric without a baseline — "reduce defaults", "improve efficiency" — the assessment will stall. A target decision with a measurable current value is the only thing that makes the rest of the work falsifiable.

Lenders usually have one of three:

• a manual committee approving applications in days
• a rules engine that is over-flagging and rejecting good customers
• a call centre spending most of its time on document checks

Each implies a different starting point, and only one of them is usually a modelling problem.

The four questions that decide whether to proceed

Is the outcome predictable enough to learn? If the process depends on judgement that experienced staff cannot articulate, a model will not capture it either. This is the most common reason AI pilots fail quietly and get quietly abandoned.

Is there enough labelled data? Not volume — labelled examples tied to a known outcome. A lender with two years of decisions and a stable policy usually has more usable signal than one with ten years and three policy regimes.

Is the cost of being wrong higher than the cost of not acting? In credit this is almost always yes, which is precisely why a human override with a retained reason matters more than model accuracy.

Who is accountable when it is wrong? If the answer is "the vendor", the assessment has not finished. Accountability has to sit with a named internal role before go-live, not after the first disputed decision.

What "ready" looks like in practice

Institutions that pass readiness share a few unglamorous properties:

• A decision log. Every automated decision retains its inputs, model version, and outcome, queryable by date and by applicant.
• A deterministic fallback. When confidence is low or an input is missing, the case routes to a human. Not as an error path — as the designed, expected path.
• Reproducibility. Given the same inputs and the same model version, the same output can be regenerated. This is what an examiner will ask for, and it is much harder to retrofit than to build.
• Data ownership clarity. Someone internally can say where each field comes from, how fresh it is, and who may correct it.

None of these are about model sophistication. They are about whether the organisation can operate and explain the thing after it ships.

When the honest answer is "not AI"

A meaningful proportion of assessments conclude that AI is the wrong instrument. Rules with better coverage of edge cases beat a model on well-defined inputs. A process redesign that removes the re-keying step often delivers more than any model built on top of it.

That outcome is a success, not a failure. A paid assessment you own outright — whether or not you continue with the vendor who produced it — should be able to end in "do not build this". If a provider cannot say that, they are selling something.

What to ask a provider before you start

• Can you name a case where you recommended against building?
• Who owns the model after go-live, and what are they paid to do?
• What happens to the decision log if you leave?
• How do you retrain, and who approves a new version into production?

If the answers are vague, the risk is not the model. It is the dependency.