Baselines and regression diffs¶

Baselines let CI distinguish new regressions from findings that already existed.

Single-project and portfolio baselines use separate versioned contracts. Both store only safe audit state needed for comparison; they do not store evidence text, CI/CD variable values, or raw API payloads.

Single-project baseline¶

Save:

gitlab-project-audit audit group/project \
  --save-baseline .gitlab-project-audit-baseline.json

Compare:

gitlab-project-audit audit group/project \
  --baseline .gitlab-project-audit-baseline.json

Write the diff as JSON:

gitlab-project-audit audit group/project \
  --baseline .gitlab-project-audit-baseline.json \
  --diff-output audit-diff.json

Portfolio baseline¶

The same flags work for explicit multi-project and group audits.

Save an explicit-project portfolio baseline:

gitlab-project-audit audit group/a group/b \
  --save-baseline portfolio-baseline.json

Save a filtered group baseline:

gitlab-project-audit audit --group group \
  --topic python \
  --visibility private \
  --save-baseline portfolio-baseline.json

Compare later:

gitlab-project-audit audit --group group \
  --topic python \
  --visibility private \
  --baseline portfolio-baseline.json \
  --fail-on-new

Portfolio baselines are keyed by project path. Project ordering therefore does not affect the diff.

Portfolio project classification¶

Portfolio comparison classifies each project as:

Classification	Meaning
`added`	Project exists in the current portfolio but not the baseline.
`removed`	Project existed in the baseline but is no longer selected/present.
`unchanged`	Project status and all comparable findings are unchanged.
`changed`	Project execution state or one or more findings changed.

Within completed projects, findings retain the normal rule-level classifications: new, unchanged, resolved, and changed.

Project execution errors are retained explicitly in the baseline/diff rather than being converted into findings.

Fail only on regressions¶

For either single-project or portfolio mode:

gitlab-project-audit audit group/a group/b \
  --baseline portfolio-baseline.json \
  --fail-on-new

Unchanged legacy finding failures do not return exit code 1.

A finding regression includes:

a new unsuppressed fail or error
an existing non-problem becoming a fail/error
increased severity on an existing problem
fail becoming error
a previously suppressed problem becoming unsuppressed

For portfolios:

a completed project becoming a project execution error is a regression
an added project contributes regressions only for problematic findings (or a project execution error)
a removed project is explicit but is not itself treated as a regression

Current project or rule execution errors still return exit code 3, even with --fail-on-new.

Output separation¶

The normal audit report remains on stdout. Human baseline diff output goes to stderr by default. Use --diff-output PATH to write deterministic JSON separately so JSON/SARIF audit stdout remains machine-readable.

Schemas¶

The packaged contracts are:

baseline-1.1.schema.json
diff-1.0.schema.json
portfolio-baseline-1.1.schema.json
portfolio-diff-1.0.schema.json

Baseline 1.1 artifacts persist the provenance of each completed audit. Legacy 1.0 baselines remain loadable and compare normally, but do not contain provenance.