Warden · AI Governance

Your model risk framework was written for models that are deterministic

An engagement for regulated institutions extending existing model risk management — SR 11-7 and equivalents — to generative and agentic systems, where validation, monitoring and challenge all need redefining.

Request a briefing Financial Services

Existing MRM assumes a testable, stable function

Validation practice was built around models with defined inputs, measurable error and stable behavior between releases. A generative system has an open input space, no single correct output and behavior that changes when the provider updates the model underneath you.

Benchmarking is not validationA score on a public benchmark says little about performance on your population and your task.
The vendor changes the modelA hosted model can be updated without a change request on your side, which is a version-control problem your framework does not anticipate.
Effective challenge needs new skillsIndependent review cannot challenge what the reviewers cannot evaluate.

How the engagement runs

Framework gap analysis

Where existing MRM policy breaks down when applied to generative and agentic systems.

Tiering

Risk tiering that reflects autonomy and consequence, not only materiality of the decision.

Validation approach

Evaluation design, adversarial testing and human-review sampling appropriate to non-deterministic output.

Ongoing monitoring

Drift, provider version change detection, and the trigger conditions for revalidation.

What you hold at the end

Revised model risk policy language covering generative and agentic systems
A tiering methodology that accounts for autonomy and consequence
Validation standards and templates for non-deterministic systems
Ongoing monitoring requirements, including provider model-change detection
A skills and staffing assessment for the independent review function

What supervisors expect

SR 11-7: US supervisory guidance on model risk management: development, implementation, use, validation and governance.
SS1/23: The PRA's model risk management principles for UK banks, with explicit expectations on model identification and tiering.
EU AI Act: Creditworthiness and certain insurance uses are named high-risk, so the AI Act obligations land on top of MRM.

What carries the evidence

TraceLockContinuous Trust. Immutable Evidence.Continuous control monitoring across nine core frameworks and fifteen regulated packs, with WORM evidence auditors can verify.ArgusAutonomous Red-Team AssessmentAttack your own agents before someone else does, then prove the fix.

Extend the framework you have

Institutions with mature MRM do not need a second framework. They need the existing one to survive contact with a system that improvises.