From risk to test, with no gaps.
Risks, controls, requirements, verification and validation for your AI system, linked and signed on top of the MLOps and DevSecOps layers your team already uses.
From your team’s work to signed evidence
Froga generates signed evidence to document your AI system and prove compliance with applicable regulatory requirements.
The Froga chain, signed
- Risks · Remaining risk: the organisation accepts it
- Controls · Do they lower the risk?
- Requirements · Validation: is it fit for its intended use?
- Design · Verification: does it meet each requirement?
The system. Your team’s MLOps and DevSecOps layers
- data
- runs and metrics
- model
- repository and continuous integration
Every risk, all the way to its test
Every risk has its controls, every control its requirement, every requirement its test and every test its result. If a link is missing, it shows.
1
Risks
What can go wrong, how much harm it would do and how likely it is.
2
Controls
Each control lowers the likelihood of the harm, its impact or both. The organisation accepts the remaining risk in writing.
3
Requirements
Each control becomes a requirement, with how to check it and the criterion for passing it.
4
Verification
Tests run in continuous integration. Each result says whether the system meets the requirement, does not meet it, or whether the test cannot settle it.
5
Validation
Checks that the system is fit for its intended use, with the data and the model of each version.
| Impact → | ||||||
|---|---|---|---|---|---|---|
| Negligible | Low | Medium | High | Critical | ||
| Likelihood → | Almost certain | Likelihood Almost certain, impact Negligible: a person’s judgement, no risks. | Likelihood Almost certain, impact Low: not acceptable, no risks. | Likelihood Almost certain, impact Medium: not acceptable, no risks. | Likelihood Almost certain, impact High: not acceptable, no risks. | Likelihood Almost certain, impact Critical: not acceptable, no risks. |
| Likely | Likelihood Likely, impact Negligible: acceptable, no risks. | Likelihood Likely, impact Low: a person’s judgement, no risks. | Likelihood Likely, impact Medium: not acceptable, no risks. | Likelihood Likely, impact High: not acceptable, 2 risks. | Likelihood Likely, impact Critical: not acceptable, no risks. | |
| Possible | Likelihood Possible, impact Negligible: acceptable, no risks. | Likelihood Possible, impact Low: a person’s judgement, no risks. | Likelihood Possible, impact Medium: a person’s judgement, no risks. | Likelihood Possible, impact High: not acceptable, no risks. | Likelihood Possible, impact Critical: not acceptable, no risks. | |
| Unlikely | Likelihood Unlikely, impact Negligible: acceptable, no risks. | Likelihood Unlikely, impact Low: acceptable, no risks. | Likelihood Unlikely, impact Medium: a person’s judgement, no risks. | Likelihood Unlikely, impact High: a person’s judgement, no risks. | Likelihood Unlikely, impact Critical: not acceptable, 1 risk. | |
| Rare | Likelihood Rare, impact Negligible: acceptable, no risks. | Likelihood Rare, impact Low: acceptable, no risks. | Likelihood Rare, impact Medium: acceptable, no risks. | Likelihood Rare, impact High: acceptable, no risks. | Likelihood Rare, impact Critical: a person’s judgement, no risks. | |
| Impact → | ||||||
|---|---|---|---|---|---|---|
| Negligible | Low | Medium | High | Critical | ||
| Likelihood → | Almost certain | Likelihood Almost certain, impact Negligible: a person’s judgement, no risks. | Likelihood Almost certain, impact Low: not acceptable, no risks. | Likelihood Almost certain, impact Medium: not acceptable, no risks. | Likelihood Almost certain, impact High: not acceptable, no risks. | Likelihood Almost certain, impact Critical: not acceptable, no risks. |
| Likely | Likelihood Likely, impact Negligible: acceptable, no risks. | Likelihood Likely, impact Low: a person’s judgement, no risks. | Likelihood Likely, impact Medium: not acceptable, no risks. | Likelihood Likely, impact High: not acceptable, no risks. | Likelihood Likely, impact Critical: not acceptable, no risks. | |
| Possible | Likelihood Possible, impact Negligible: acceptable, no risks. | Likelihood Possible, impact Low: a person’s judgement, no risks. | Likelihood Possible, impact Medium: a person’s judgement, no risks. | Likelihood Possible, impact High: not acceptable, no risks. | Likelihood Possible, impact Critical: not acceptable, no risks. | |
| Unlikely | Likelihood Unlikely, impact Negligible: acceptable, no risks. | Likelihood Unlikely, impact Low: acceptable, 1 risk. | Likelihood Unlikely, impact Medium: a person’s judgement, no risks. | Likelihood Unlikely, impact High: a person’s judgement, no risks. | Likelihood Unlikely, impact Critical: not acceptable, no risks. | |
| Rare | Likelihood Rare, impact Negligible: acceptable, no risks. | Likelihood Rare, impact Low: acceptable, no risks. | Likelihood Rare, impact Medium: acceptable, no risks. | Likelihood Rare, impact High: acceptable, 1 risk. | Likelihood Rare, impact Critical: a person’s judgement, 1 risk. | |
Example with three risks. The number in each cell shows how many risks fall in it.
Everything runs on your team’s MLOps and DevSecOps layers: metrics come from its runs and tests from its repository.
What it works with
MLOps · data and executions
| Tool | Contribution | Scope |
|---|---|---|
| Lock file fingerprint | A third party can recompute it with the repository and its dependencies. | |
| Run metrics fingerprint | The metrics remain in MLflow’s store. | |
| Execution material fingerprint | The material remains in Dagster’s store. | |
| Provenance in the execution history | Data is referenced through pointers. |
DevSecOps · repositories and continuous integration
| Tool | Contribution | Scope |
|---|---|---|
| Delivery verification | According to Froga’s continuous integration configuration. | |
| Delivery verification | According to Froga’s continuous integration configuration. | |
| Repository connection | No delivery verification. |
Who approves each version
Each change is proposed by one person and approved by another, in the repository, and it stays signed. Froga does not let a version out until the chain is complete, and its documentation can be checked without Froga.
The evidence Froga produces, standard by standard.
Fourteen standard families. Which part of each Froga addresses for an AI system and how many clauses its catalogue evaluates. What stays outside, on the same page.
DORA · 11 clauses
What the evaluation coversICT risks, protection controls, resilience testing and a dependency inventory.
EN 18286 · 11 of 39 clauses
What the evaluation coversSystem documentation, risk treatment, data, changes and a monitoring plan.
ENS · 4 clauses
What the evaluation coversLifecycle records, a monitoring plan, secure development and a component inventory.
Estatuto de los Trabajadores · 1 clause
What the evaluation coversAlgorithmic transparency towards worker representatives.
ISO 14971 · 10 clauses
What the evaluation coversRisk analysis and control, residual risk, benefit–risk analysis and risks introduced by controls.
ISO/IEC 23894 · 8 clauses
What the evaluation coversAI risk identification, assessment and treatment, with records and a monitoring plan.
ISO/IEC 27001 · 19 clauses
What the evaluation coversSystem security risks, controls and a software component inventory; monitoring and review.
ISO/IEC 42001 · 9 clauses
What the evaluation coversAI risks and impacts, objectives, operational control and system review.
Ley 15/2022 · 1 clause
What the evaluation coversBias minimisation in decision-making algorithms.
MDR · 12 clauses
What the evaluation coversMedical software safety and risks, performance, lifecycle and dependencies.
prEN 18228 · 28 clauses
What the evaluation coversAI risks throughout the lifecycle: testing, controls and residual risk evaluation.
prEN 18229-1 · 6 clauses
What the evaluation coversLifecycle records, foreseeable misuse and controls for human oversight and intervention.
prEN 18282 · 8 clauses
What the evaluation coversCybersecurity threats, resistance to attacks, integrity, access, confidentiality and monitoring.
prEN 18283 · 10 clauses
What the evaluation coversBias, data governance, measurement, mitigation and residual bias monitoring.
Scope is partial and applies to the AI system, not the entire organisation or the whole standard. Counts do not represent a compliance percentage.
If it fits so far, the next step is a conversation.
Let's talk