Structured scoring engine
The same inputs produce the same output, every time.
The problem
AI compliance has a structural blind spot.
Compliance review of AI-driven systems has a structural problem: the thing being evaluated is often the thing least visible in the policy library. Questionnaires assert. Policies describe. Vendors attest. The underlying system — models, sub-processors, data flows, controls, failure modes — sits behind a layer of documentation that traditional compliance review was not designed to penetrate.
Policies don't operate themselves
A pristine AI governance charter and an unmonitored production model can look identical on paper.
Questionnaires anchor on assertion
Vendors return a coherent control narrative. Coherence is not evidence.
Workstreams fragment
Privacy, security, AI governance, and legal review live in different workstreams. Nothing stitches them into a single compliance picture.
AI systems fail in new ways
Drift, hallucination, bias, prompt injection, untracked sub-processors, training-data provenance — traditional control frameworks weren't built to detect these.
Kaptrix is built for the moment after that gap becomes visible. It gives compliance, risk, and capital teams a structured, evidence-backed view of whether an AI system is actually controlled — and a live reasoning surface to interrogate that view as new artifacts arrive.
What Kaptrix is
Three engines. One platform.
Kaptrix combines three things that don't usually coexist in a single platform. The operator owns the score. The AI expands what the operator can see. Evidence, not opinion, is what moves anything.
Evidence engine
Turns artifacts — policies, contracts, control tests, model cards, logs — into structured, machine-readable signals.
Reasoning engine
Operates continuously on top of both, grounded in what has actually been observed about this specific system.
The core idea
Most AI compliance tools do one of two things.
They summarize
They read policies and produce a narrative. Fast, but non-defensible at audit.
They score by checklist
They apply a static questionnaire. Defensible on paper, but blind to what the system is actually doing in production.
Kaptrix does neither in isolation
A rubric-driven scoring engine runs underneath a live reasoning layer, with a strict separation between what the operator decides and what the AI contributes. The scoring logic is fixed and inspectable. The AI layer is bounded and auditable. Together they produce something both fast and defensible — a combination traditional compliance review and first-generation AI tools have not delivered.
The three layers
Coordinated. Bounded. Each with a specific job.
Understanding the separation between the three layers is the key to understanding why the platform's outputs hold up under scrutiny.
Platform architecture
03
Reasoning engine
Continuous · context-aware · grounded
02
Evidence engine
Artifact ingest · structured signal extraction
01
Scoring engine
Trust anchor · reproducible · operator-controlled
Every layer above rolls up to — and is traceable back to — the layer beneath it.
Layer 01
The Scoring Engine
Auditable. Operator-controlled.
This is the trust anchor. Every downstream output — insights, comparisons, recommendations, control-failure flags — rolls up to a score produced by a fixed, inspectable process.
Six risk dimensions
Control Credibility
Whether stated controls are actually operating, or whether the policy document is carrying the load.
Tooling & Vendor Exposure
Which models, sub-processors, and third-party AI services sit in the data path, and whether the contracts and DPAs match the regulated data crossing those boundaries.
Data & Sensitivity Risk
How regulated data is sourced, retained, processed, and disposed of in line with the regime that actually applies.
Governance & Accountability
What controls exist when the system behaves badly, who owns them, and whether oversight is proportionate to the system's risk tier.
Operational Readiness
Whether monitoring, logging, evaluation, and human-in-the-loop controls are instrumented in production, or only described in policy.
Open Validation
What has been independently verified against applicable frameworks, and what remains untested.
Four properties define how this layer behaves
Scores are reproducible
The same inputs produce the same output, every time. No model variance, no drift between runs, no “the AI felt differently today.”
Failure-weighted, not feature-weighted
A system that looks complete on the surface but is weak on control credibility cannot score its way out through strong peripheral signals.
The operator assigns every base score
The AI does not. This is a hard architectural constraint — not a configuration setting, not user-toggleable.
Every score carries rationale
Nothing is stored as a number alone. Every sub-criterion is accompanied by the reasoning that produced it.
Layer 02
The Evidence Engine
Artifact-driven, not interview-driven.
Traditional compliance review front-loads questionnaires and management interviews because there is no better way to surface hidden structure when starting from zero. Kaptrix inverts this.
Artifacts come first. The platform ingests them and extracts structured signals: claims made, controls in place, dependencies declared, gaps visible.
Artifacts ingested
Architecture diagramsModel & system cardsPolicies & standardsSub-processor lists & DPAsSOC / ISO reportsControl test resultsEval & red-team reportsAudit logs & output samples
→Extracted into structured signals feeding the scoring model.
The effect: management interviews become targeted follow-up rather than primary discovery. You walk into the control walkthrough already knowing what is missing, what is inconsistent, and what needs pressure.
Layer 03
The Reasoning Engine
Continuous. Grounded. Context-aware.
This is the layer most compliance tools lack entirely, and it is what makes Kaptrix a live system rather than a report generator.
Questions it answers in context
- Where is this system most likely to fail an audit?
- Which controls in the policy are unsupported by operating evidence?
- What is missing that the applicable framework — NIST AI RMF, ISO/IEC 42001, EU AI Act, SOC 2, HIPAA, GDPR — actually requires?
- How does this system compare to others we have assessed in the same regulatory category?
- Where has confidence in the score shifted since assessment began, and why?
Outputs are grounded. No generic answers. No hallucinated confidence. If the evidence does not support a conclusion, the engine says so — and flags it as a gap to close, not a question to paper over.
How evidence changes a score
Evidence never silently modifies a score.
When the platform ingests a new artifact and extracts a signal that affects the model, it generates a structured proposal. That proposal names the sub-criterion it affects, the direction of the pressure it creates, the rationale behind it, and the supporting artifacts.
Support
Evidence that confirms an existing score.
Contradiction
Evidence that creates downward pressure.
Augmentation
Evidence that supports an upward signal.
Gap
Missing evidence that should exist but does not.
Bounded. Reviewed. Approved.
- A single piece of evidence cannot move a score by an arbitrary amount. Evidence affecting one dimension cannot bleed into another. These are enforced at the engine level, not left to operator discipline.
- Proposals are reviewed and approved by the operator before they affect the score. Nothing enters the composite without a human decision.
- The result is a scoring model that stays live — continuously updated as new artifacts arrive — while remaining controlled.
Confidence is separate from score
A score tells you where the system landed. Confidence tells you how much to trust that landing.
Kaptrix calculates confidence independently from the score itself — based on how much of the model is covered by evidence, the quality of the sources feeding it, how recent the evidence is, and how consistent the signals are with each other. Confidence qualifies the score. It does not override it.
Soft signal
High score, low confidence
Score4.0 / 5.0
Confidence0.42
The evidence we have is positive, but we have not seen enough.
Strong signal
High score, high confidence
Score4.0 / 5.0
Confidence0.86
We have seen enough, and it holds up.
Audit committees, regulators, and counsel need to be able to tell these two states apart. Kaptrix makes that distinction visible, explicit, and reviewable.
Why this holds up under scrutiny
Every output can be walked backwards to its source.
When an auditor, an audit committee, a regulator, an LP, or counsel asks how did you arrive at this view, the answer needs to be traceable.
Trace chain
Step 01
Artifact
Source of truth
Step 02
Signal
Structured extraction
Step 03
Proposal
Named, bounded, reasoned
Step 04
Score impact
After operator approval
- Every score traces to its sub-criteria and the rationale behind them.
- Every adjustment traces to the proposal that triggered it and the artifact that supported it.
- Every AI-generated insight traces to the signals and evidence it drew from.
- No scoring logic is hidden. No adjustments happen silently. No conclusions float free of their evidence base.
The audit trail is not a feature bolted on afterward — it is the structure the platform is built on. If a conclusion cannot be traced to its source, it is not a conclusion Kaptrix will present.
What makes Kaptrix different
Two existing approaches. Each with structural limits.
Manual compliance review
- Defensible but slow
- Inconsistent across portcos and assessments
- Dependent on whichever reviewer knows AI best
- Scales poorly — every assessment starts from zero
First-gen AI compliance tools
- Fast but shallow
- Map to checklists without structure
- No audit trail behind outputs
- No operator judgment inside outputs
Kaptrix
- Structured human judgment
- Automated evidence extraction
- Rubric-driven scoring logic
- Full auditability
- Continuous reasoning that stays live throughout the assessment
Faster than traditional compliance review, more structured than ad hoc questionnaire workflows, more defensible than pure AI output.
What Kaptrix will not do
Trust also comes from knowing where a system stops.
Does not certify regulatory compliance
Produces an evidence-based compliance posture. Certification, attestation, and legal opinion remain with auditors and counsel.
Does not replace external audit or legal review
Compresses the fragmented parts of AI-specific compliance into a structured layer the rest of the program can build on.
Does not operate as a black-box autonomous evaluator
The operator decides. The platform equips that decision.
Does not accept claims it cannot trace
If the evidence is not there, the gap is surfaced — not filled with inference.
What you get
When the assessment concludes.
Composite score
Calibrated to failure-weighted compliance risk, with a full dimension-level breakdown.
Confidence signal
Qualifies how much of the model is evidence-backed.
Evidence coverage map
What has been validated and what has not, mapped against applicable frameworks.
Identified gaps & contradictions
A structured view of control failures, policy-vs-practice gaps, and risks requiring follow-up.
Complete audit trail
Linking every output to the artifacts and decisions that produced it.
Live reasoning surface
Continues to answer questions during the assessment, at audit committee, before a regulator, and after remediation.
The shift
Small on the surface. Large in practice.
Kaptrix moves the central question of AI compliance from one with no defensible answer to one that does.
Before
“Do we attest this system is compliant?”
After
“What evidence supports each control, what contradicts it, and how much regulatory risk remains?”
Kaptrix is built to answer the second question in real time, with full context, and with logic you can defend — in front of an audit committee, a regulator, a board, or an LP.
Built for high-stakes evaluation
For the moments where the answer has to be right and the reasoning has to hold up.
Portfolio AI compliance review. Pre-acquisition AI compliance assessment. Validation of vendor AI compliance claims. Assessment of internal AI initiatives where regulatory and capital exposure intersect.
Most valuable when decisions must be made quickly, documentation is incomplete or contradictory, and control claims need to be pressure-tested against operating evidence.