Explainable Underwriting: Proving Your AI Didn't Discriminate

A customer is declined. They ask why. What do you say?

"Our model scored you below threshold" is not an answer. It's not an answer to the customer, and it is emphatically not an answer to a regulator. Yet it's functionally what a lot of AI underwriting can produce today.

With the NAIC Model Bulletin adopted across 23 US jurisdictions and the EU AI Act classifying insurance pricing and risk assessment as high-risk, "the model decided" has stopped being an awkward answer and started being a compliance failure. Here's what explainable underwriting actually requires — in engineering terms, not legal ones.

Three different questions people conflate

"Explainability" gets used for three distinct requirements. They need different solutions, and mixing them up is why teams flounder.

1. Why this decision? (Adverse action.) The customer was declined or surcharged — which specific factors drove that? This is per-decision and legally required.

2. How does the model work? (Model transparency.) What features does it use, how was it trained, what are its limits? This is per-model and documented once.

3. Is it fair? (Bias testing.) Does the model produce systematically worse outcomes for a protected class? This is per-cohort and statistical.

You need all three. Only the second is about the model itself. The other two are data problems.

Reason codes: the thing you can't retrofit

Adverse-action requirements mean you must state the principal factors behind a negative decision. In practice, that means every decision needs to carry reason codes — a small, human-readable set of drivers ("roof age over 15 years", "three prior claims in 5 years", "property in a high-risk flood zone").

Two hard rules follow:

Reason codes must be generated at decision time, not reconstructed later. You cannot re-run a model six months on and claim that's why it decided — the data has changed, the model may have changed, and the reconstruction is a guess. Write them when the decision happens or they don't exist.

They must be genuine drivers, not decoration. Attaching plausible-sounding reasons that weren't actually what moved the score is worse than nothing. If it's a gradient-boosted model, use SHAP or an equivalent and map contributions to codes honestly.

The architecture that makes this survivable

The pattern that works — and it's the one we build for regulated lines — is to keep the model advisory and the decision deterministic:

• The model produces a risk score with feature attributions.
• An approved rules/rating engine converts that score plus explicit business rules into the actual decision and price.
• Every step writes to an immutable decision log: inputs, model version, attributions, rules fired, outcome, timestamp.

This separation matters enormously. When the price comes from approved rating tables and the decision comes from inspectable rules, you can explain any outcome exactly — because the explanation is the rule, not an interpretation of a black box. The model informs; it doesn't rule.

Carriers that let the model output the price directly find themselves unable to answer the simplest regulatory question, and end up rebuilding this way anyway. Save the round trip.

Proving it wasn't discriminatory

Here's the trap. Most teams assume that not using protected attributes means the model can't discriminate. That's wrong, and regulators know it's wrong.

Proxies. Postcode correlates with ethnicity. Occupation correlates with gender. Your model can reproduce a protected-class pattern perfectly while never seeing the attribute — this is proxy discrimination, and it's the main thing bias testing exists to catch.

Which produces an awkward requirement: to test for disparate impact, you need the cohort data you're not allowed to price on. So you hold that data in a governed, access-controlled space used strictly for fairness testing — never as a model feature. That's a data-architecture decision, and it's one you must make deliberately, with legal, before you need it.

Then run the tests as a routine report, not a fire drill: outcome rates by cohort, over time, with thresholds that trigger review.

Why this is a data problem, not a model problem

Count what we've needed: per-decision inputs captured. Model versions tracked. Attributions computed and stored. Rules logged. Cohort data governed and sliceable. Lineage from source to feature.

Every one of those is a property of your pipeline. None is solved by choosing a more interpretable algorithm — though that helps at the margins.

This is the part carriers keep learning late: you cannot document lineage you didn't capture, or produce a decision record you never wrote. Explainability is designed in before launch, or it's absent forever.

The upside

Teams that build this properly discover something unexpected: explainable underwriting is better underwriting. Reason codes make model failure modes visible. Decision logs make drift detectable. Fairness slices surface data quality issues nobody had noticed.

The regulation is forcing work that makes the models genuinely better. That's a rare kind of compliance burden — the kind worth doing enthusiastically.

We build governed, lineage-aware, audit-ready data platforms for AI in regulated industries. More at IntelliBooks.