From AI Pilots to Production in a Regulated Business
Overview
Our client had eleven successful AI pilots and nothing in production, because every one of them stopped at the same governance question nobody owned.
Ian Mccarty
Chief Technology Officer / fairsystems
Most organisations do not have an AI capability problem. They have a route-to-production problem. Pilots succeed because pilots are exempt from the controls that real systems must satisfy, and then each one arrives separately at a risk function with no framework for assessing it. fairsystems was engaged by a financial services group to build that route once, rather than negotiate it eleven times.
The Client
Our client is a financial services group offering lending and wealth products across regulated markets. Their innovation function had run a series of well-executed AI pilots covering document processing, client onboarding checks, adviser support and internal knowledge search. Every pilot demonstrated value and none had reached customers. Risk, compliance and technology each had legitimate concerns, no shared framework for resolving them, and no agreed definition of what a production-ready AI system in a regulated context needed to demonstrate.
Roadmap
1
Assess
fairsystems examined all eleven pilots and, more importantly, the path each had taken when it sought approval. Our consultants interviewed risk, compliance, technology, legal and the business sponsors to establish where each attempt stopped and why. The blockage was consistent: no one owned the question of what evidence an AI system had to produce to be considered controlled. We assessed the pilots against the group's existing model risk framework, which had been written for statistical credit models and covered the new systems only partially, and mapped precisely where the gaps sat.
2
Deliver
Our team extended the model risk framework to cover AI systems, defining proportionate evidence requirements by use case rather than treating a document classifier and a customer-facing adviser tool as equivalent risks. fairsystems established an approval path with named owners at each gate, so a system moves rather than circulates. We built the shared platform capability the pilots had each been improvising, covering evaluation, monitoring, human review and audit logging, and worked with the business to sequence the pilots by value and risk, taking the two clearest cases to production first to prove the route existed.
3
Continue
fairsystems continues to support the group's AI governance forum as further use cases enter the pipeline, and reviews the framework as regulatory expectations develop. We work with the risk function on ongoing monitoring, since an approved AI system needs continuing evidence rather than a one-time sign-off. Our consultants also help the business assess new proposals early against the framework, so that effort goes into use cases that can realistically reach production rather than into pilots that will stop at the same gate.
Solution Details
Pilots are easy because they are exempt. Production needs a route.
No definition of what a controlled AI system must demonstrate
Deliverable: fairsystems extended the group's model risk framework to AI, defining explicit and proportionate evidence requirements by use case.
Every pilot negotiating approval from first principles
Deliverable: Our consultants established a single approval path with named owners at each gate, so systems progress rather than circulate between functions.
Model risk framework written for statistical credit models
Deliverable: We mapped the gaps between the existing framework and AI system risks, extending it rather than creating a parallel and competing regime.
Identical scrutiny applied regardless of risk
Deliverable: fairsystems introduced proportionate tiers, so a document classifier is not assessed as though it were a customer-facing advisory system.
Each pilot rebuilding evaluation and monitoring
Deliverable: Our team built shared platform capability for evaluation, monitoring, human review and audit logging, removing per-pilot reinvention.
No ongoing evidence after initial approval
Deliverable: We established continuous monitoring requirements with the risk function, treating approval as a state to maintain rather than a one-time gate.
Effort invested in use cases that could never be approved
Deliverable: fairsystems introduced early framework assessment for new proposals, directing effort towards use cases that can realistically reach production.
Human oversight asserted rather than designed
Deliverable: Our consultants specified meaningful human review points with defined authority and recorded decisions, making oversight demonstrable to a regulator.
Result:
The group moved from eleven stalled pilots to a working route to production, with the first two use cases live and the remainder sequenced against a framework everyone had agreed in advance. Approval time for a new AI system fell from an open-ended negotiation to a defined path with named gate owners. Risk and compliance are engaged early rather than encountered late, and the business now assesses proposals against production readiness before investing in a pilot. The capability that mattered was the route, not the models.
0
Pilots progressed to an approved route
0
Reduction in AI approval cycle time
0
Proposal to decision time reduction
0
AI systems under continuous monitoring
0
Reduction in per-pilot engineering rework
Testimonials
How our services bring about success
Annika Nkosi
Lead Process Analyst
fairsystems’ team brought a unique combination of skills to our business assignment and was often sought out to tackle the most complex and integrated challenges.
Marie-Anne Badeaux
Programme Manager
fairsystems will challenge your thinking, provide you with alternative perspectives and, where appropriate, actively coach/mentor/advise you to generate and co-create solutions.
Justin J. Spratt
General Manager
Fairsystems consults impeccably to understand organizational needs and then brings innovative business relevant ideas and solutions to the table.
Mohamed Bin Naser
Chief Financial Officer
I am impressed with fairsystems’ team ability to grasp complex, accounting system problems and quickly find a simple, straightforward and affordable approach.
Ella-May Robinson
HR Manager
fairsystems has an incredible gift for making the complex appear simple and without being simplistic. It is a pleasure to work with them as they always deliver to an exceptionally high standard.
Jamal Aagha
Chief Executive Officer
There are some great companies in the media industry and the social space, but we have yet to see anyone take it to the level that we have with @fairsystems.
Allen Gibson
Operations Manager
The most valuable lesson we learned is that there are no shortcuts. Every minute we spent dissecting and learning was rewarded.
