The DevSecOps Audit Checklist for Your Pipeline

Opinions about pipeline security are cheap; evidence is not. The audit below asks for artefacts, not assurances — policies you can open, numbers you can query, logs you can grep. Run it against your own delivery setup in an afternoon, score honestly, and the two or three weakest sections will identify themselves. That is the point: an audit earns its cost only when it ends in a short, defensible list of fixes. The discipline it measures is DevSecOps, and the sections follow its natural layers.

DevSecOps audit area 1: the gate policy exists on paper

Everything downstream depends on this document, so the audit starts by asking whether it exists at all.

• Can you produce a written statement of which findings block a merge? If the answer lives in someone’s memory, the gate has no policy.
• Does blocking require all three conditions — a fresh finding, a serious rating, a reliable detector? Any two out of three produces a gate people argue with.
• Do exceptions carry a named owner and an expiry, capped around ninety days? A perpetual waiver is debt wearing a lanyard.
• Is inherited backlog explicitly excluded from blocking? It must be, or the first week of enforcement will detonate every release.
• Does the policy distinguish actions — block, warn, record — per rule, instead of one binary verdict for everything?
• When a block fires, does the author see the reason and the fix path in the same interface, or in a separate queue?
• Is the configuration of the gate itself read from the protected base branch, so a change cannot author its own admission rules?

DevSecOps audit area 2: the findings are worth the electricity

Scanner output decays into noise unless quality is measured. The numbers to pull:

• Effective false-positive rate per rule. Google’s published experience with fleet-wide analysis is blunt: a rule earns blocking status only when this approaches zero, and thirty findings from a rule is the minimum sample before trusting it.
• Override rate. Below five percent a month is the healthy band; a gate overridden more often than that has lost authority, and the fix is retiring rules, not lecturing engineers.
• Fix time for new findings, tracked as a median and a ninetieth percentile. The median hides the tail; the p90 does not.
• Escape rate — defects that shipped to production in spite of the checks, each traced back to the stage that should have intercepted it.
• Results scope: does analysis evaluate changed lines only, or rescan the entire tree and bury reviewers in duplicate output?
• Reviewer load: comments per hundred lines touched, held under ten percent.

DevSecOps audit area 3: credentials have a lifecycle

Secrets remain the cheapest attack in software, and the audit questions here are answerable in minutes:

• Does a scanner run in every developer’s pre-commit hook, and is push protection switched on server-side as well? One layer catches what the other misses.
• Do you know who can bypass push protection on your repositories? On GitHub the default answers “any user with write access”, and nobody should learn that from an incident report.
• When a credential leaks, is the runbook’s first step rotation? History rewriting comes second, never first — a scrubbed repository does not revoke a still-live key.
• Which pipeline jobs still authenticate with static cloud keys, and what is the dated migration plan to short-lived federated identity for each?
• Are fork-originating builds structurally unable to read secrets, whatever the workflow author intended?
• Is there a single inventory of where long-lived credentials live, so an incident response starts from a list rather than a treasure hunt?

DevSecOps audit area 4: the supply chain produces evidence

Provenance work fails quietly, so the audit asks for samples rather than statements:

• Pick yesterday’s production release. Produce its SBOM. If retrieval takes more than minutes, the inventory is decorative.
• Produce its signature and verify it against the signer you expect. Producing evidence and trusting evidence are separate controls; only the second detects anything.
• Produce its provenance statement, and check the build platform’s SLSA standing — Build levels run L0 to L3, Source levels L1 to L4. GitHub artifact attestations, for reference, correspond to Build L2, and reach L3 only where reusable workflows are pinned.
• Are third-party actions referenced by immutable commit hash, with version bumps reviewed rather than automatic?
• Has anyone rehearsed the response to a poisoned build dependency? The March 2025 tj-actions/changed-files compromise — leaking runner secrets through a hijacked action, with CISA advising rotation — is the canonical drill scenario.

DevSecOps audit area 5: the pipeline machinery itself

The OWASP catalogue of CI/CD weaknesses gives the hardening checks, and a self-respecting audit spot-checks rather than assumes:

• Job permissions default to read-only, widened only where a specific step justifies it.
• No workflow runs untrusted contributor code inside a privileged trigger such as pull_request_target.
• Dynamic and API testing exists against a running build, not just static analysis of source.
• Scanner output is stored centrally in a standard format — SARIF is the convention — so trends survive employee turnover.
• Pipeline overhead is measured and acknowledged. Checks that double build time get optimised or scoped, not tolerated into resentment.
• Gate and scanner configuration changes follow the same review path as code, with an audit trail of who changed what and when.

DevSecOps audit area 6: people, cadence, and the maturity ladder

The final section checks whether the programme is a living process or a shrine. The field’s own adoption arc gives the questions. Day zero artefacts — the gate policy, the SARIF store, the identity model — should already exist; a programme without them is still at the starting line whatever the slides claim. Day one was meant to be a report-only pilot with false-positive measurement and SBOM generation from the outset; if scanning went straight to blocking, expect the override data in area 2 to show the scar tissue. Day two steady state means monthly metric reviews, deliberate retirement of noisy rules, and a rehearsed compromise runbook.

Then ask the human questions. Do one or two engineers per team hold protected time — a tenth to a fifth of the week — as security champions, written into the sprint rather than borrowed from nights? Whoever fills that role needs authority, not just enthusiasm: the power to retire a noisy rule and to schedule the threat-modeling hour without escalation. Does the 45-minute threat-modeling drill run at the start of significant features, walking STRIDE and the Manifesto’s quartet of questions about what is being built, what could go wrong, what is being done, and whether the job was done well? And does leadership know the one claim this field refuses to make: nothing certifies DevSecOps, no official standard defines it, and the term — used interchangeably with SecDevOps and DevOpsSec — carries no agreed word-order meaning. The Ops Atlas page on DevSecOps is the sober reference to cite in the audit report’s appendix.

Scoring, and what to do with the result

Mark each item as evidenced, partial, or absent, then look at the pattern before the total. A common shape: strong tooling, absent policy, nobody owning exceptions. That shape predicts next quarter’s incident better than any count of installed scanners. Another recurring one: everything evidenced except area 6 — the programme works but lives in one person’s head, which is one resignation away from stopping. Fix in this order — policy first (it is free), secrets second (highest incident reduction per hour), provenance third, cadence last. If the audit surfaces monitoring gaps around escaped defects, the detection side pairs naturally with operational intelligence; see how AI is reshaping DevOps and SRE practice, the survey of AIOps use cases in monitoring and root-cause analysis, and the operational follow-on on automating incident detection and response. An audit you can rerun in an afternoon, against evidence rather than opinions, is the cheapest programme review you will ever run — and the only one that improves the moment it finds something. Diary it quarterly; the delta between runs is your actual maturity curve, and it reads better than any roadmap slide.