Document Classification & Routing for Clinical Site Packets

A site activation packet does not arrive as a labeled stack of forms. It arrives as an undifferentiated stream of files from a CTMS export, an eTMF drop folder, or a scanning vendor’s batch job: scan0042.pdf, Form1572_signed.pdf, CV_Investigator_final_v2.docx, a folder of IRB approval letters named after the board rather than the site. Before any of it can reach the right check, something has to decide what each file is — a Form 1572, an investigator CV, a financial disclosure, a delegation log, an IRB approval letter, a lab certification — so it can be handed to the validator built for that document type. Get the classification wrong and a well-formed CV gets run through the FDA 1572 schema and fails for reasons that have nothing to do with its actual content; get it right, silently, and a document lands on a validator that was never designed to catch its specific regulatory requirements.

This guide is the routing layer between raw extraction and the compliance gate: it sits downstream of the parsing and recognition work and upstream of Schema Validation & Error Categorization, inside Automated Document Ingestion & Validation Workflows. By the time a document reaches this stage it already has extracted text — either directly from a native PDF/DOCX or from the OCR pipeline — plus whatever container metadata the file itself carries. The job here is narrow and mechanical: decide the document type with enough confidence to route correctly, and if that confidence is not there, say so rather than guess.

Why this is a rules problem, not a modeling problem #

It is tempting to treat document typing as a text-classification exercise and reach for a trained model. Resist it. A model trained on labeled examples produces a probability distribution over classes with no fixed, inspectable reason for its answer, and a regulatory routing decision has to be reconstructable months later from a specific, named rule — not a weight matrix. A rules-first classifier evaluates a fixed, version-controlled set of deterministic signals — a filename pattern, a PDF metadata field, an AcroForm value, a keyword or regex match in the extracted text — and combines them into a confidence score with a documented formula. When an inspector or a regulatory-affairs reviewer asks why a document was routed a particular way, the answer is “rule FDA1572-FN-03 matched the filename and rule FDA1572-KW-07 matched the phrase ‘Statement of Investigator’ in the body text, for a combined confidence of 0.92” — not “the model was fairly sure.”

That does not mean probabilistic signals have no place. A similarity score or a trained model can assist a human reviewer working the low-confidence queue, surfacing a best guess to speed up manual triage. What it must never do is make the accept/route decision autonomously. This mirrors the same non-negotiable boundary the ingestion pipeline draws around OCR confidence and schema validation: probabilistic output informs, deterministic rules and humans decide.

Decision flow #

The classifier is a short, linear pipeline with one branch. Every signal source is evaluated independently, the matches are combined into a single confidence number, and a threshold gate decides between automatic routing and a human-reviewed fallback. Nothing about the flow depends on the content of any single document type — new document types are added by adding rules, not by changing the pipeline shape.

Three signal sources feed the score, in ascending order of how much they depend on upstream extraction quality. Filename patterns are cheap and available before any parsing runs at all. PDF metadata and AcroForm field values are strong when present — a fillable Form 1572 template usually carries its own form title in a field — but not every document is a fillable form. Keyword and regex signatures over the extracted body text catch everything else, including scanned documents that only have a name like IMG_20260114_003.pdf and no useful container metadata.

Library and tooling landscape #

The building blocks are ordinary and, deliberately, unglamorous. A document router does not need anything beyond the standard library’s re module and a maintained PDF library to read structural metadata; the discipline is in how the signals are combined and audited, not in the tooling.

Deprecated dependency: PyPDF2 reached end of life and its final releases simply re-export pypdf. Any code path that reads a PDF’s /Title, /Subject, or AcroForm field values for classification should import pypdf directly and pin it in pyproject.toml, exactly as recommended for the parsing and assembly stages elsewhere in this platform. A stale import PyPDF2 in a classification rule is a sign the rule predates the current toolchain and should be migrated.

Step-by-step implementation #

The classifier is five small, independently testable stages: model the rule and result types, extract the three signal sources, score the matches, and apply the routing threshold.

1. Model the document type, the rule, and the result #

A closed enumeration of document types and a frozen rule dataclass keep the rule table declarative and reviewable without reading the scoring code.

"""Typed model for a document type, a classification rule, and its result."""
from __future__ import annotations

from dataclasses import dataclass
from enum import Enum


class DocumentType(str, Enum):
    FDA_1572 = "FDA_1572"
    INVESTIGATOR_CV = "INVESTIGATOR_CV"
    IRB_APPROVAL_LETTER = "IRB_APPROVAL_LETTER"
    FINANCIAL_DISCLOSURE = "FINANCIAL_DISCLOSURE"
    DELEGATION_LOG = "DELEGATION_LOG"
    LAB_CERTIFICATION = "LAB_CERTIFICATION"
    UNKNOWN = "UNKNOWN"


class SignalSource(str, Enum):
    FILENAME = "filename"
    PDF_METADATA = "pdf_metadata"
    ACROFORM_FIELD = "acroform_field"
    KEYWORD_TEXT = "keyword_text"


@dataclass(frozen=True)
class ClassificationRule:
    """One deterministic signal: a pattern that, if matched, votes for a type."""

    rule_id: str
    doc_type: DocumentType
    source: SignalSource
    pattern: str   # regex, matched case-insensitively
    weight: float  # contribution to the confidence score, 0.0-1.0


@dataclass(frozen=True)
class ClassificationResult:
    doc_id: str
    doc_type: DocumentType
    confidence: float
    matched_rule_ids: tuple[str, ...]
    routed_to: str

2. Extract the filename, PDF metadata, and AcroForm signals #

Each extractor is a small, pure function over the file on disk. None of them decide anything on their own; they only surface raw signal for the scorer.

"""Extract deterministic signals from a document's filename and PDF structure."""
from __future__ import annotations

from pathlib import Path

from pypdf import PdfReader


def filename_signal(path: Path) -> str:
    """Return the filename stem, lowercased, for regex matching."""
    return path.stem.lower()


def pdf_metadata_signal(path: Path) -> dict[str, str]:
    """Return the PDF DocumentInformation dictionary as plain strings.

    pypdf exposes /Title, /Subject, /Author and similar keys through
    PdfReader.metadata; any of them may be absent on a given file.
    """
    reader = PdfReader(str(path))
    info = reader.metadata or {}
    return {str(key).lstrip("/"): str(value) for key, value in info.items() if value}


def acroform_field_signal(path: Path) -> dict[str, str]:
    """Return AcroForm field name/value pairs, e.g. from a fillable Form 1572.

    Returns an empty mapping for documents with no interactive form —
    that absence is itself informative and is handled by the scorer.
    """
    reader = PdfReader(str(path))
    fields = reader.get_fields() or {}
    return {name: str(field.value) for name, field in fields.items() if field.value}

3. Score every rule against its signal and combine the votes #

Combining signals is a single pass over the rule table. Each rule looks only at the haystack for its own source, so adding a new document type never touches the scoring loop.

"""Evaluate every classification rule and combine matches into a score."""
from __future__ import annotations

import re
from collections import defaultdict


def classify(
    doc_id: str,
    filename_stem: str,
    pdf_metadata: dict[str, str],
    acroform_fields: dict[str, str],
    body_text: str,
    rules: list[ClassificationRule],
) -> ClassificationResult:
    """Evaluate every rule against its signal and return the best-scoring type."""
    votes: dict[DocumentType, float] = defaultdict(float)
    matched: dict[DocumentType, list[str]] = defaultdict(list)
    haystacks = {
        SignalSource.FILENAME: filename_stem,
        SignalSource.PDF_METADATA: " ".join(pdf_metadata.values()),
        SignalSource.ACROFORM_FIELD: " ".join(acroform_fields.values()),
        SignalSource.KEYWORD_TEXT: body_text,
    }

    for rule in rules:
        haystack = haystacks[rule.source]
        if re.search(rule.pattern, haystack, re.IGNORECASE):
            votes[rule.doc_type] += rule.weight
            matched[rule.doc_type].append(rule.rule_id)

    if not votes:
        return ClassificationResult(doc_id, DocumentType.UNKNOWN, 0.0, (), "unrouted")

    best_type = max(votes, key=lambda candidate: votes[candidate])
    confidence = min(votes[best_type], 1.0)
    return ClassificationResult(doc_id, best_type, confidence, tuple(matched[best_type]), "unrouted")

4. Apply the confidence threshold and resolve the destination #

The threshold is read from the environment so a staging deployment can run stricter or looser than production without a code change, and the routing table is a plain dictionary rather than a chain of conditionals.

"""Apply the confidence threshold and resolve the destination validator queue."""
from __future__ import annotations

import os

CONFIDENCE_THRESHOLD = float(os.environ.get("CLASSIFIER_CONFIDENCE_THRESHOLD", "0.75"))

_ROUTING_TABLE: dict[DocumentType, str] = {
    DocumentType.FDA_1572: "validator.fda-1572",
    DocumentType.INVESTIGATOR_CV: "validator.investigator-cv",
    DocumentType.IRB_APPROVAL_LETTER: "validator.irb-approval",
    DocumentType.FINANCIAL_DISCLOSURE: "validator.fin-disclosure",
    DocumentType.DELEGATION_LOG: "validator.delegation-log",
    DocumentType.LAB_CERTIFICATION: "validator.lab-cert",
}


def route(result: ClassificationResult) -> ClassificationResult:
    """Return a copy of result with routed_to resolved by confidence and type."""
    if result.confidence < CONFIDENCE_THRESHOLD or result.doc_type is DocumentType.UNKNOWN:
        destination = "queue.manual-classification-review"
    else:
        destination = _ROUTING_TABLE.get(result.doc_type, "queue.manual-classification-review")
    return ClassificationResult(
        result.doc_id, result.doc_type, result.confidence, result.matched_rule_ids, destination,
    )

5. Emit the audit record #

The classification decision is a regulatory fact — which rules fired, what confidence resulted, and where the document went — and belongs in the same hash-chained log as every other stage in the platform.

"""Record a classification decision in the shared audit trail."""
from __future__ import annotations

import json
import logging

logger = logging.getLogger("ingestion.classification")


def record_classification(result: "ClassificationResult") -> None:
    event = {
        "action": "document_classified",
        "doc_id": result.doc_id,
        "doc_type": result.doc_type.value,
        "confidence": round(result.confidence, 4),
        "matched_rule_ids": list(result.matched_rule_ids),
        "routed_to": result.routed_to,
    }
    logger.info(json.dumps(event, sort_keys=True))

Validation and audit-trail integration #

Classification does not replace validation; it decides which validation a document receives. A confidently routed Form 1572 still has to clear the Schema Validation & Error Categorization gate for its type, and a document that fails structural validation after routing is a signal worth feeding back into the rule set — either the rule matched a document that merely resembles a Form 1572, or the routing table sent a correctly identified document to the wrong validator. Both are detectable because the audit record for classification and the audit record for validation share the same doc_id, so reconstructing “why did this document end up here, and did it belong here” is a join across two log entries rather than a manual investigation.

The controlled vocabulary each DocumentType maps onto — and the field names that keyword rules search for — should not be invented ad hoc per project. They come from the shared regulatory data dictionary, so that a “Form 1572” from one sponsor’s naming convention and a “Statement of Investigator” from another’s resolve to the same canonical type before a single rule is written. Keeping the rule table itself in a version-controlled file — not inlined and edited freely — means a change to what counts as a Form 1572 is itself a reviewable, auditable diff.

Error categorization and recovery #

Classification failures split cleanly into two classes, and only one of them is retryable.

• Permanent — no rule matched. A document that matches none of the configured signatures is not a transient glitch; it is either a document type the rule set does not yet cover or a malformed submission. Route it to manual classification review, log it as UNKNOWN, and do not retry the same rules against the same bytes expecting a different answer.
• Permanent — corrupt or unreadable PDF structure. When pypdf cannot open a file’s cross-reference table, that is a defect in the file itself. Quarantine the document with the specific parser error attached rather than silently falling back to filename-only classification, which would produce a lower-confidence result without recording why.
• Permanent — conflicting high-weight matches. Two rules for different document types both fire with high weight — for example, a cover letter that quotes another document’s title. Do not silently pick the higher score without flagging it; when the runner-up score is within a configured margin of the winner, treat the result as low-confidence regardless of the raw number, and send it to manual review.
• Transient — signal extraction I/O failure. A read failure fetching file bytes from a network-mounted document store, or a timeout against a metadata cache, is retryable with bounded backoff. The classification rules themselves never change between retries; only the ability to read the input does.

Compliance checklist #

Before a classified document is allowed to reach its validator, confirm each of the following holds:

• Every classification rule is declarative, version-controlled data, not inline conditional logic
• Filename, PDF metadata, AcroForm, and keyword signals are each evaluated independently before scoring
• The confidence threshold is read from configuration, not hardcoded in the scoring function
• A document below threshold is routed to manual review rather than assigned its best-guess type
• No trained or probabilistic model makes the final routing decision
• An audit event records the matched rule IDs, the confidence score, and the routing destination for every document
• The controlled vocabulary for document types matches the shared regulatory data dictionary

FAQ #

Why not just use a machine-learning text classifier for document types? #

A trained classifier cannot name a specific, reviewable reason for its output, and a regulatory routing decision needs to be reconstructable from a fixed rule months later, not from a probability. A rules-first pipeline gives every decision a named, version-controlled cause. A model may still assist a human working the manual-review queue, but it never makes the accept-or-route decision on its own.

What happens to a document that matches no rule at all? #

It is classified as unknown and routed to a manual-classification queue rather than forced into a best guess. That is a permanent condition — retrying the same rules against the same file will not change the outcome — and it is the correct trigger for adding a new rule or fixing a rule that should have matched.

How is the confidence score defined? #

Each rule that matches a document contributes its configured weight to that document type’s running total, and the type with the highest combined weight wins, capped at 1.0. Because the weights are fixed per rule and version-controlled, the same document always produces the same score, which is what makes the threshold decision auditable.

Does classification replace schema validation? #

No. Classification only decides which validator a document is handed to. The document still has to pass the structural and regulatory checks for its type in schema validation; a correctly classified document can still fail validation, and that failure is a separate, later event in the audit trail.

Related #

• Up one level: Automated Document Ingestion & Validation Workflows
• Classifying Site Documents with a Rules-First Pipeline
• Schema Validation & Error Categorization
• OCR & Metadata Extraction Pipelines
• Clinical Metadata Normalization Across Trial Systems