Reconciling Site SOP Version Mismatches Automatically

Every site running a protocol operates against a controlled set of standard operating procedures, and every staff member with a role that touches a given SOP must have read and acknowledged its current version before performing the corresponding task. The controlled set changes over the life of a trial — a safety reporting SOP gets a revision, a lab-handling procedure is superseded — and each revision resets the clock on every affected site’s acknowledgements. This walkthrough builds the reconciler that compares what a site has acknowledged against what is currently controlled and flags the gap before it becomes an activation blocker or, worse, an inspection finding of staff operating on an outdated procedure. It is one of the concrete detectors surveyed by Checklist Sync & Gap Analysis within Automated Document Ingestion & Validation Workflows, and it shares its severity vocabulary with the two-system reconciliation engine in Automating Checklist Synchronization Between EDC and CTMS. The other two activation blockers in this set are a missing delegation of authority log and an expired IRB approval.

Why naive approaches fail

Comparing “does the site have an acknowledgement on file for this SOP” is the tempting first cut, and it misses the failure modes that actually cause deviations:

  • Presence is not currency. A staff member who acknowledged version 3 of a procedure eighteen months ago satisfies a naive presence check even after version 5 supersedes it. The gap is not that nobody acknowledged the SOP — it is that the acknowledgement on file no longer matches the controlled document.
  • Not every role owes every SOP. A lab-handling procedure applies to lab staff, not to the study coordinator who never touches a sample. A reconciler that flags every staff member against every SOP produces so much noise that real gaps get lost in it.
  • A brand-new revision should not instantly fail every site. The moment a controlled SOP is revised, every site with an acknowledgement on the prior version is technically “stale,” but sites need a defensible grace window to read and acknowledge the new version before that becomes a compliance finding rather than an operational fact of life.
  • Superseded and retired SOPs are different states. An SOP that was retired outright — folded into another procedure, for instance — should not keep generating “stale acknowledgement” findings forever; the reconciler needs to know the document is no longer controlled at all, not merely that it moved versions.
  • Manual spot checks do not scale across a large site network. A program running 200 sites against 40 controlled SOPs, each with its own role applicability, is a matrix too large for a human to audit by hand on every revision.

Solving this correctly means modeling the controlled document set, the site’s staff roster with role applicability, and each staff member’s acknowledgement history as distinct, comparable data, and applying a grace window as an explicit input rather than an implicit assumption.

Architecture overview

The reconciler loads the current controlled SOP set and each site’s staff roster and acknowledgement history, matches each mandatory SOP against the roles that owe it, applies the grace window to recent revisions, and classifies each result before writing an audit record.

SOP version reconciliation pipeline Five stages left to right: controlled SOPs and roster, acknowledgement history, role-applicability match, staleness classification, and audit record with remediation finding. SOPs + roster Ack history Role match Classify Audit + fix

A stale acknowledgement that survives its grace window is treated as a hard finding by the same weighted-and-gated model described in Clinical Site Readiness Assessment Frameworks, and because the controlled set itself often changes on the heels of a protocol amendment cleared through the IRB/Ethics Workflow Mapping process, the two detectors frequently fire in the same window.

Setup and configuration

No third-party dependency is required beyond pytest for the test suite; date handling uses the standard library.

pip install "pytest>=8.0"

The grace window after a revision’s effective date, and whether a retired SOP still requires an acknowledgement, are policy decisions read from the environment:

"""Environment-driven configuration for SOP version reconciliation."""
from __future__ import annotations

import os
from dataclasses import dataclass


def _required_env(name: str) -> str:
    value = os.environ.get(name)
    if not value:
        raise RuntimeError(f"Missing required environment variable: {name}")
    return value


@dataclass(frozen=True)
class SopReconciliationConfig:
    acknowledgement_grace_days: int
    controlled_set_source: str

    @classmethod
    def from_env(cls) -> "SopReconciliationConfig":
        return cls(
            acknowledgement_grace_days=int(
                os.environ.get("SOP_ACK_GRACE_DAYS", "14")
            ),
            controlled_set_source=_required_env("SOP_CONTROLLED_SET_SOURCE"),
        )

SOP_CONTROLLED_SET_SOURCE names the system of record — typically a document-control platform — that the controlled SOP list is loaded from; keeping it in configuration means swapping document-control vendors never touches the reconciliation logic.

Full working implementation

Model the controlled document, the staff roster with role applicability, the acknowledgement record, and the finding as immutable, typed structures.

"""Reconcile site SOP acknowledgements against the current controlled set."""
from __future__ import annotations

from dataclasses import dataclass
from datetime import date, datetime, timezone
from enum import Enum


class SopLifecycleState(str, Enum):
    """Whether a controlled document is currently in force."""

    EFFECTIVE = "effective"
    RETIRED = "retired"


class AckStatus(str, Enum):
    """Result of comparing one staff member's acknowledgement to policy."""

    CURRENT = "current"
    GRACE_PERIOD = "grace_period"
    STALE = "stale"
    UNACKNOWLEDGED = "unacknowledged"
    NOT_APPLICABLE = "not_applicable"


class Severity(str, Enum):
    CRITICAL = "P0"
    HIGH = "P1"
    OK = "ok"


@dataclass(frozen=True, slots=True)
class ControlledSop:
    """One standard operating procedure in the sponsor's controlled set."""

    sop_code: str
    current_version: str
    effective_on: date
    lifecycle_state: SopLifecycleState
    applicable_roles: frozenset[str]


@dataclass(frozen=True, slots=True)
class StaffMember:
    site_id: str
    staff_id: str
    full_name: str
    role: str


@dataclass(frozen=True, slots=True)
class SopAcknowledgement:
    """One staff member's most recent acknowledgement of one SOP."""

    site_id: str
    staff_id: str
    sop_code: str
    acknowledged_version: str
    acknowledged_on: date


@dataclass(frozen=True, slots=True)
class SopMismatch:
    """One detected SOP version gap for a staff member at a site."""

    site_id: str
    staff_id: str
    sop_code: str
    status: AckStatus
    severity: Severity
    detail: str
    blocks_activation: bool


def reconcile_sop_versions(
    site_id: str,
    controlled_sops: list[ControlledSop],
    staff: list[StaffMember],
    acknowledgements: list[SopAcknowledgement],
    as_of: date,
    grace_days: int,
) -> list[SopMismatch]:
    """Return every stale, missing, or in-grace SOP acknowledgement for a site.

    Only staff whose role appears in ``applicable_roles`` are evaluated
    against a given SOP; everyone else is silently out of scope for it,
    never flagged, and never counted toward the site's gap total.
    """
    ack_by_key: dict[tuple[str, str], SopAcknowledgement] = {
        (a.staff_id, a.sop_code): a
        for a in acknowledgements
        if a.site_id == site_id
    }
    site_staff = [s for s in staff if s.site_id == site_id]

    mismatches: list[SopMismatch] = []
    for sop in controlled_sops:
        if sop.lifecycle_state is SopLifecycleState.RETIRED:
            continue  # retired documents no longer generate findings

        for member in site_staff:
            if member.role not in sop.applicable_roles:
                continue

            ack = ack_by_key.get((member.staff_id, sop.sop_code))
            if ack is None:
                mismatches.append(SopMismatch(
                    site_id, member.staff_id, sop.sop_code, AckStatus.UNACKNOWLEDGED,
                    Severity.CRITICAL,
                    f"{member.full_name} has never acknowledged {sop.sop_code}",
                    blocks_activation=True,
                ))
                continue

            if ack.acknowledged_version == sop.current_version:
                continue  # current; no finding

            days_since_effective = (as_of - sop.effective_on).days
            if 0 <= days_since_effective <= grace_days:
                mismatches.append(SopMismatch(
                    site_id, member.staff_id, sop.sop_code, AckStatus.GRACE_PERIOD,
                    Severity.HIGH,
                    f"{sop.sop_code} revised {days_since_effective} days ago; "
                    f"{member.full_name} has not yet acknowledged v{sop.current_version}",
                    blocks_activation=False,
                ))
            else:
                mismatches.append(SopMismatch(
                    site_id, member.staff_id, sop.sop_code, AckStatus.STALE,
                    Severity.CRITICAL,
                    f"{member.full_name} acknowledged v{ack.acknowledged_version}, "
                    f"current is v{sop.current_version}",
                    blocks_activation=True,
                ))

    return mismatches

The role-applicability check runs before anything else in the inner loop, which is what keeps the reconciler quiet for staff the SOP was never meant to cover — the alternative, checking every staff member against every SOP and filtering afterward, produces the same result but buries the meaningful gaps in noise during triage. The grace-period branch is the second detail worth keeping: a revision younger than the grace window downgrades to HIGH and does not block, while the same gap outside the window is CRITICAL and does.

Emit an audit record for the completed reconciliation run so a site’s SOP-currency history is queryable from the append-only audit log rather than reconstructed from document-control exports after the fact:

"""Emit an audit event for a completed SOP reconciliation run."""
from __future__ import annotations

import json
import logging

logger = logging.getLogger("checklist_sync.sop_version_reconciliation")


def record_sop_reconciliation(
    site_id: str, as_of: date, mismatches: list[SopMismatch], principal: str
) -> None:
    event = {
        "action": "sop_version_reconciliation",
        "site_id": site_id,
        "as_of": as_of.isoformat(),
        "checked_at": datetime.now(timezone.utc).isoformat(),
        "principal": principal,
        "mismatch_count": len(mismatches),
        "activation_blocked": any(m.blocks_activation for m in mismatches),
        "mismatches": [
            {"staff_id": m.staff_id, "sop_code": m.sop_code, "status": m.status.value}
            for m in mismatches
        ],
    }
    logger.info(json.dumps(event, sort_keys=True))

Validation and edge-case handling

A few shapes routinely trip up SOP reconciliation logic that looks correct on a first pass:

  • A staff member with two roles. If a coordinator is also cross-trained as a lab technician, evaluate applicability as a union across all of their roles rather than a single primary role — otherwise a genuinely applicable SOP silently never gets checked for them.
  • A version rollback. Document control systems occasionally reissue a prior version number after a correction. Compare on the full version string as recorded by the controlled set, not on a parsed numeric ordering, so a reconciler never assumes “higher number always wins” when the sponsor’s own numbering scheme does not guarantee that.
  • An SOP retired mid-cycle with open findings. Once lifecycle_state flips to RETIRED, the loop above stops generating new findings for it, but any findings already recorded in the audit log from before retirement should remain visible in historical reports — retiring a document clears future checks, not history.
  • A newly onboarded staff member. Someone added to the roster after a site’s last reconciliation run has no acknowledgement history at all; they should surface as UNACKNOWLEDGED for every applicable SOP rather than being skipped because “they’re new,” since the gap is real regardless of tenure.
  • Grace window measured from the wrong date. The grace period counts from effective_on, the date the new version takes effect, not from when a given site happened to receive notification. If notification lag is a genuine operational concern, model it as a separate, explicitly tracked field rather than silently extending the grace window per site.

Testing and verification

The behaviors worth defending with pytest are role-scoping, the grace-period downgrade, and the fact that retired SOPs stop producing findings.

"""Tests for SOP version reconciliation."""
from __future__ import annotations

from datetime import date

TODAY = date(2026, 7, 17)


def _sop(**over: object) -> ControlledSop:
    base = dict(
        sop_code="SOP-014", current_version="4", effective_on=date(2026, 6, 1),
        lifecycle_state=SopLifecycleState.EFFECTIVE,
        applicable_roles=frozenset({"coordinator"}),
    )
    base.update(over)
    return ControlledSop(**base)


def _member(role: str = "coordinator") -> StaffMember:
    return StaffMember("SITE-014", "S001", "J. Alvarez", role)


def test_out_of_scope_role_is_never_flagged() -> None:
    mismatches = reconcile_sop_versions(
        "SITE-014", [_sop()], [_member(role="lab_tech")], [], TODAY, grace_days=14,
    )
    assert mismatches == []


def test_unacknowledged_sop_is_critical() -> None:
    mismatches = reconcile_sop_versions(
        "SITE-014", [_sop()], [_member()], [], TODAY, grace_days=14,
    )
    assert mismatches[0].status is AckStatus.UNACKNOWLEDGED
    assert mismatches[0].blocks_activation is True


def test_recent_revision_is_grace_period_not_stale() -> None:
    ack = SopAcknowledgement("SITE-014", "S001", "SOP-014", "3", date(2026, 5, 1))
    mismatches = reconcile_sop_versions(
        "SITE-014", [_sop(effective_on=date(2026, 7, 10))], [_member()], [ack],
        TODAY, grace_days=14,
    )
    assert mismatches[0].status is AckStatus.GRACE_PERIOD
    assert mismatches[0].blocks_activation is False


def test_old_revision_past_grace_is_stale() -> None:
    ack = SopAcknowledgement("SITE-014", "S001", "SOP-014", "3", date(2025, 1, 1))
    mismatches = reconcile_sop_versions(
        "SITE-014", [_sop(effective_on=date(2026, 1, 1))], [_member()], [ack],
        TODAY, grace_days=14,
    )
    assert mismatches[0].status is AckStatus.STALE
    assert mismatches[0].blocks_activation is True


def test_retired_sop_produces_no_findings() -> None:
    sop = _sop(lifecycle_state=SopLifecycleState.RETIRED)
    mismatches = reconcile_sop_versions(
        "SITE-014", [sop], [_member()], [], TODAY, grace_days=14,
    )
    assert mismatches == []

Run with pytest -q. The grace-period pair of tests is the one to keep especially close, because it is the boundary most likely to shift when a sponsor renegotiates its SOP rollout policy — a wider grace window is a one-line configuration change, and these tests prove the change only affects timing, not whether the underlying gap is ever detected at all.

FAQ

Does an acknowledgement of any version satisfy the requirement?

No. An acknowledgement only satisfies the requirement if it matches the SOP’s current controlled version. Presence of an old acknowledgement is treated as either a grace-period finding, if the revision is recent enough that the sponsor’s rollout window still applies, or a stale finding that blocks activation once that window has passed.

Why does a retired SOP stop generating findings?

Once a controlled document is retired — commonly because its content was folded into another procedure — there is no longer a current version for staff to be out of date against, so continuing to flag it would be a false positive that never resolves. Historical findings recorded before retirement remain in the audit log; only new checks stop being generated.

How is this different from delegation-of-authority gap detection?

Both are activation blockers modeled the same way — typed records, a deterministic detector, and an audit-logged finding — but they check different regulatory obligations. Delegation of authority verifies that a named, qualified person is authorized to perform a specific trial task; SOP reconciliation verifies that the staff performing any task are current on the procedures governing how it is done. A site can pass one and fail the other.