Skip to content
Venturalítica
Froga

From risk to test, with no gaps.

Risks, controls, requirements, verification and validation for your AI system, linked and signed on top of the MLOps and DevSecOps layers your team already uses.

From your team’s work to signed evidence

Froga generates signed evidence to document your AI system and prove compliance with applicable regulatory requirements.

Risks, controls, requirements and design go down to the system, built on your team’s MLOps and DevSecOps layers. On the way back up, verification, validation and the effectiveness of the controls check each level, up to the remaining risk.

The Froga chain, signed

  1. Risks · Remaining risk: the organisation accepts it
  2. Controls · Do they lower the risk?
  3. Requirements · Validation: is it fit for its intended use?
  4. Design · Verification: does it meet each requirement?

The system. Your team’s MLOps and DevSecOps layers

  • data
  • runs and metrics
  • model
  • repository and continuous integration

Every risk, all the way to its test

Every risk has its controls, every control its requirement, every requirement its test and every test its result. If a link is missing, it shows.

  1. 1

    Risks

    What can go wrong, how much harm it would do and how likely it is.

  2. 2

    Controls

    Each control lowers the likelihood of the harm, its impact or both. The organisation accepts the remaining risk in writing.

  3. 3

    Requirements

    Each control becomes a requirement, with how to check it and the criterion for passing it.

  4. 4

    Verification

    Tests run in continuous integration. Each result says whether the system meets the requirement, does not meet it, or whether the test cannot settle it.

  5. 5

    Validation

    Checks that the system is fit for its intended use, with the data and the model of each version.

Without controls
Without controls. Likelihood in rows and impact in columns.
Impact →
NegligibleLowMediumHighCritical
Likelihood →Almost certainLikelihood Almost certain, impact Negligible: a person’s judgement, no risks.Likelihood Almost certain, impact Low: not acceptable, no risks.Likelihood Almost certain, impact Medium: not acceptable, no risks.Likelihood Almost certain, impact High: not acceptable, no risks.Likelihood Almost certain, impact Critical: not acceptable, no risks.
LikelyLikelihood Likely, impact Negligible: acceptable, no risks.Likelihood Likely, impact Low: a person’s judgement, no risks.Likelihood Likely, impact Medium: not acceptable, no risks.Likelihood Likely, impact High: not acceptable, 2 risks.Likelihood Likely, impact Critical: not acceptable, no risks.
PossibleLikelihood Possible, impact Negligible: acceptable, no risks.Likelihood Possible, impact Low: a person’s judgement, no risks.Likelihood Possible, impact Medium: a person’s judgement, no risks.Likelihood Possible, impact High: not acceptable, no risks.Likelihood Possible, impact Critical: not acceptable, no risks.
UnlikelyLikelihood Unlikely, impact Negligible: acceptable, no risks.Likelihood Unlikely, impact Low: acceptable, no risks.Likelihood Unlikely, impact Medium: a person’s judgement, no risks.Likelihood Unlikely, impact High: a person’s judgement, no risks.Likelihood Unlikely, impact Critical: not acceptable, 1 risk.
RareLikelihood Rare, impact Negligible: acceptable, no risks.Likelihood Rare, impact Low: acceptable, no risks.Likelihood Rare, impact Medium: acceptable, no risks.Likelihood Rare, impact High: acceptable, no risks.Likelihood Rare, impact Critical: a person’s judgement, no risks.
With controls
With controls. Likelihood in rows and impact in columns.
Impact →
NegligibleLowMediumHighCritical
Likelihood →Almost certainLikelihood Almost certain, impact Negligible: a person’s judgement, no risks.Likelihood Almost certain, impact Low: not acceptable, no risks.Likelihood Almost certain, impact Medium: not acceptable, no risks.Likelihood Almost certain, impact High: not acceptable, no risks.Likelihood Almost certain, impact Critical: not acceptable, no risks.
LikelyLikelihood Likely, impact Negligible: acceptable, no risks.Likelihood Likely, impact Low: a person’s judgement, no risks.Likelihood Likely, impact Medium: not acceptable, no risks.Likelihood Likely, impact High: not acceptable, no risks.Likelihood Likely, impact Critical: not acceptable, no risks.
PossibleLikelihood Possible, impact Negligible: acceptable, no risks.Likelihood Possible, impact Low: a person’s judgement, no risks.Likelihood Possible, impact Medium: a person’s judgement, no risks.Likelihood Possible, impact High: not acceptable, no risks.Likelihood Possible, impact Critical: not acceptable, no risks.
UnlikelyLikelihood Unlikely, impact Negligible: acceptable, no risks.Likelihood Unlikely, impact Low: acceptable, 1 risk.Likelihood Unlikely, impact Medium: a person’s judgement, no risks.Likelihood Unlikely, impact High: a person’s judgement, no risks.Likelihood Unlikely, impact Critical: not acceptable, no risks.
RareLikelihood Rare, impact Negligible: acceptable, no risks.Likelihood Rare, impact Low: acceptable, no risks.Likelihood Rare, impact Medium: acceptable, no risks.Likelihood Rare, impact High: acceptable, 1 risk.Likelihood Rare, impact Critical: a person’s judgement, 1 risk.

Example with three risks. The number in each cell shows how many risks fall in it.

Everything runs on your team’s MLOps and DevSecOps layers: metrics come from its runs and tests from its repository.

What it works with

MLOps · data and executions

ToolContributionScope
DVCLock file fingerprintA third party can recompute it with the repository and its dependencies.
MLflowRun metrics fingerprintThe metrics remain in MLflow’s store.
DagsterExecution material fingerprintThe material remains in Dagster’s store.
DataLadProvenance in the execution historyData is referenced through pointers.

DevSecOps · repositories and continuous integration

ToolContributionScope
GitHubDelivery verificationAccording to Froga’s continuous integration configuration.
GitLabDelivery verificationAccording to Froga’s continuous integration configuration.
ForgejoRepository connectionNo delivery verification.

Who approves each version

Each change is proposed by one person and approved by another, in the repository, and it stays signed. Froga does not let a version out until the chain is complete, and its documentation can be checked without Froga.

The evidence Froga produces, standard by standard.

Fourteen standard families. Which part of each Froga addresses for an AI system and how many clauses its catalogue evaluates. What stays outside, on the same page.

View by sector

DORA · 11 clauses

What the evaluation coversICT risks, protection controls, resilience testing and a dependency inventory.

EN 18286 · 11 of 39 clauses

What the evaluation coversSystem documentation, risk treatment, data, changes and a monitoring plan.

ENS · 4 clauses

What the evaluation coversLifecycle records, a monitoring plan, secure development and a component inventory.

Estatuto de los Trabajadores · 1 clause

What the evaluation coversAlgorithmic transparency towards worker representatives.

ISO 14971 · 10 clauses

What the evaluation coversRisk analysis and control, residual risk, benefit–risk analysis and risks introduced by controls.

ISO/IEC 23894 · 8 clauses

What the evaluation coversAI risk identification, assessment and treatment, with records and a monitoring plan.

ISO/IEC 27001 · 19 clauses

What the evaluation coversSystem security risks, controls and a software component inventory; monitoring and review.

ISO/IEC 42001 · 9 clauses

What the evaluation coversAI risks and impacts, objectives, operational control and system review.

Ley 15/2022 · 1 clause

What the evaluation coversBias minimisation in decision-making algorithms.

MDR · 12 clauses

What the evaluation coversMedical software safety and risks, performance, lifecycle and dependencies.

prEN 18228 · 28 clauses

What the evaluation coversAI risks throughout the lifecycle: testing, controls and residual risk evaluation.

prEN 18229-1 · 6 clauses

What the evaluation coversLifecycle records, foreseeable misuse and controls for human oversight and intervention.

prEN 18282 · 8 clauses

What the evaluation coversCybersecurity threats, resistance to attacks, integrity, access, confidentiality and monitoring.

prEN 18283 · 10 clauses

What the evaluation coversBias, data governance, measurement, mitigation and residual bias monitoring.

Scope is partial and applies to the AI system, not the entire organisation or the whole standard. Counts do not represent a compliance percentage.

If it fits so far, the next step is a conversation.

Let's talk