AI Ethics in Practice: A Bias Audit Walkthrough From a Recent Engagement

An SME I worked with had already been using an AI screening tool for months before anyone thought to check it for bias — not because they were careless, but because nobody had told them checking was something they needed to do. What the audit found wasn't a rogue algorithm making deliberately unfair decisions. It was a training dataset that quietly encoded a pattern nobody had noticed, and a proxy variable doing the discriminating on the model's behalf. (Identifying details here are changed to protect confidentiality; the pattern below is representative of what these audits consistently turn up.)

The audit method I use

A bias audit isn't a single test — it's four separate checks, run in order. First, identify every protected characteristic the system's decisions could plausibly correlate with, directly or indirectly: age, sex, ethnicity, disability, and so on. Second, run a disparate impact test comparing outcome rates across groups — the four-fifths rule (a selection rate for any group below 80% of the highest-performing group's rate) is a useful, well-established starting threshold, not a legal guarantee of fairness on its own. Third, test edge cases deliberately — the unusual applications and inputs a model rarely sees in normal operation, where bias often hides because there isn't enough routine data to average it out. Fourth, and most often skipped: check for proxy variables — fields that aren't a protected characteristic themselves but correlate strongly with one.

What the audit actually found

The disparate impact test flagged a selection rate gap that sat right at the edge of the four-fifths threshold — not obviously alarming on its own, which is exactly why it had gone unnoticed. Digging into the model's inputs, one field stood out: postcode, included as a reasonable-looking proxy for commute distance. Postcode in the UK correlates with a number of protected characteristics more strongly than most teams assume, and in this case it was quietly doing indirect discrimination the model's designers never intended and never tested for.

Nobody had acted in bad faith anywhere in this chain. The data reflected years of historic hiring decisions that were never audited for the same pattern, and the model faithfully learned to repeat them. That's the uncomfortable part of most bias audits: the algorithm isn't inventing prejudice, it's automating and scaling whatever pattern was already sitting quietly in the historic data.

The fix isn't always retraining the model

The instinctive response to a finding like this is “retrain the model without that field.” Sometimes that's right. More often, the more durable fix is procedural: remove the proxy variable, re-run the disparate impact test to confirm the gap actually closes rather than just moving to a different proxy, and put a recurring audit cadence in place so this doesn't quietly reappear as the underlying data shifts. A one-off technical patch without a recurring check is a fix that expires the moment the training data updates again.

Why this matters even if nobody's complained yet

The businesses that get hurt by this aren't the ones with an obviously reckless AI system — they're the ones who assumed reasonably-designed automation doesn't need checking, right up until an applicant, a journalist, or a regulator runs the same disparate impact test from the outside, without the benefit of context or good intent. A bias audit run proactively is a governance exercise. The same audit run reactively, after a complaint, is a crisis response — and it costs considerably more, in every sense, to run it that way.