Tesseract vs AWS Textract vs Azure Document Intelligence

Choosing an OCR engine for scanned clinical documents is a compliance decision disguised as a tooling decision. The three realistic options — self-hosted Tesseract, AWS Textract, and Azure Document Intelligence — trade off in ways that matter well beyond raw character-recognition accuracy: where protected health information physically travels, whether a business associate agreement covers the processing, and whether the engine can read a structured enrollment table instead of just a page of prose. This guide compares the three against the demands of scanned clinical PDFs and gives a Python abstraction that keeps the engine swappable, so a residency requirement discovered late does not force a rewrite. It extends the OCR pipeline area, sits alongside the ingestion work in Automated Document Ingestion & Validation Workflows, and its output feeds the same schema validation stage regardless of which engine produced it.

Why naive approaches fail #

Picking an engine by accuracy benchmark alone, then wiring it in directly, causes three recurring failures:

• PHI leaves the trust boundary without a signed agreement. Sending a scanned consent form or an adverse-event narrative to a cloud OCR endpoint is a PHI disclosure. Without a business associate agreement or an equivalent data-processing agreement covering that specific service and region, the call itself is a compliance violation, independent of how accurate the result is.
• Tables silently degrade to unstructured text. A drug-accountability log or a randomization table read by an engine that only does line-level text recognition comes back as words in roughly the right order, with row and column structure gone. Downstream parsing that assumes cell boundaries then extracts data from the wrong field without raising an error.
• The engine is called directly from business logic. When ingestion code calls a vendor SDK’s specific method signature, adding a second engine — or dropping one for cost or residency reasons — means touching every call site instead of swapping one implementation.
• Cost is estimated from a demo, not a monthly page volume. Cloud engines price per page or per API call; a pipeline that scales from a pilot site to a multi-country trial can see OCR spend grow faster than expected if nobody modeled it against the real page-per-site rate.

Architecture overview #

An engine-agnostic pipeline resolves the engine once, from a routing rule, then always talks to the same abstraction regardless of which vendor is behind it.

The decision the routing rule encodes is exactly the trade-off the three engines present:

The honest reading of this table is that there is no universally correct choice: a self-hosted-only compliance posture forces Tesseract regardless of its table limitations, while a document set dominated by structured enrollment forms makes a cloud engine’s native table extraction worth the residency paperwork — provided the agreement is actually in place before the first page is sent.

Setup and configuration #

Install what the deployed engines require; only install the SDKs for engines actually enabled in a given environment.

pip install pytesseract pillow opencv-python-headless
pip install boto3            # only if AWS Textract is enabled
pip install azure-ai-documentintelligence  # only if Azure Document Intelligence is enabled

Engine selection and credentials come from the environment, never from a literal in code, and a missing credential for an enabled engine fails fast at startup rather than on the first document:

"""Environment-driven OCR engine configuration."""
from __future__ import annotations

import os
from dataclasses import dataclass
from enum import Enum


class OcrEngine(str, Enum):
    TESSERACT = "tesseract"
    AWS_TEXTRACT = "aws_textract"
    AZURE_DOCUMENT_INTELLIGENCE = "azure_document_intelligence"


def required(name: str) -> str:
    value = os.environ.get(name)
    if not value:
        raise RuntimeError(f"Missing required environment variable: {name}")
    return value


@dataclass(frozen=True)
class OcrConfig:
    engine: OcrEngine
    aws_region: str | None = None
    azure_endpoint: str | None = None
    azure_api_key: str | None = None

    @classmethod
    def from_env(cls) -> "OcrConfig":
        engine = OcrEngine(required("OCR_ENGINE"))
        if engine is OcrEngine.AWS_TEXTRACT:
            return cls(engine=engine, aws_region=required("AWS_TEXTRACT_REGION"))
        if engine is OcrEngine.AZURE_DOCUMENT_INTELLIGENCE:
            return cls(
                engine=engine,
                azure_endpoint=required("AZURE_DOCINTEL_ENDPOINT"),
                azure_api_key=required("AZURE_DOCINTEL_KEY"),
            )
        return cls(engine=engine)

Full working implementation #

A Protocol fixes the contract every engine must satisfy — normalized text, per-word confidence, and any detected tables — so business logic never imports a vendor SDK directly.

"""Engine-agnostic OCR abstraction for scanned clinical PDFs."""
from __future__ import annotations

import json
import logging
from dataclasses import dataclass, field
from typing import Protocol

logger = logging.getLogger("ocr.engine_comparison")


@dataclass(frozen=True)
class WordConfidence:
    text: str
    confidence: float  # 0.0-100.0


@dataclass(frozen=True)
class ExtractedTable:
    rows: list[list[str]]


@dataclass(frozen=True)
class OcrResult:
    page_text: str
    words: list[WordConfidence]
    tables: list[ExtractedTable] = field(default_factory=list)
    engine: str = ""


class OcrEngineClient(Protocol):
    """Every engine implementation must satisfy this shape."""

    def extract(self, image_bytes: bytes) -> OcrResult: ...


class TesseractEngine:
    """Self-hosted engine; no table structure, full PHI control."""

    def extract(self, image_bytes: bytes) -> OcrResult:
        import pytesseract
        from PIL import Image
        import io

        image = Image.open(io.BytesIO(image_bytes))
        data = pytesseract.image_to_data(image, output_type=pytesseract.Output.DICT)
        words = [
            WordConfidence(text=text, confidence=float(conf))
            for text, conf in zip(data["text"], data["conf"])
            if text.strip() and conf != "-1"
        ]
        page_text = " ".join(w.text for w in words)
        return OcrResult(page_text=page_text, words=words, engine="tesseract")


class TextractEngine:
    """Managed AWS engine with native table and form extraction."""

    def __init__(self, region: str) -> None:
        import boto3

        self._client = boto3.client("textract", region_name=region)

    def extract(self, image_bytes: bytes) -> OcrResult:
        response = self._client.analyze_document(
            Document={"Bytes": image_bytes}, FeatureTypes=["TABLES", "FORMS"]
        )
        words = [
            WordConfidence(text=block["Text"], confidence=block["Confidence"])
            for block in response["Blocks"]
            if block["BlockType"] == "WORD"
        ]
        page_text = " ".join(w.text for w in words)
        tables = _tables_from_textract_blocks(response["Blocks"])
        return OcrResult(page_text=page_text, words=words, tables=tables, engine="aws_textract")


def _tables_from_textract_blocks(blocks: list[dict]) -> list[ExtractedTable]:
    # Simplified: production code walks TABLE -> CELL relationships.
    return [ExtractedTable(rows=[]) for block in blocks if block["BlockType"] == "TABLE"]


def build_engine(config: "OcrConfig") -> OcrEngineClient:
    if config.engine.value == "tesseract":
        return TesseractEngine()
    if config.engine.value == "aws_textract":
        assert config.aws_region is not None
        return TextractEngine(region=config.aws_region)
    raise NotImplementedError(f"engine not wired: {config.engine.value}")


def run_and_record(client: OcrEngineClient, image_bytes: bytes, document_id: str) -> OcrResult:
    result = client.extract(image_bytes)
    event = {
        "action": "ocr_extraction_completed",
        "document_id": document_id,
        "engine": result.engine,
        "word_count": len(result.words),
        "table_count": len(result.tables),
        "min_confidence": min((w.confidence for w in result.words), default=None),
    }
    logger.info(json.dumps(event, sort_keys=True))
    return result

Azure Document Intelligence is deliberately omitted from the concrete client above and left as an exercise with the same shape as TextractEngine — the point of the OcrEngineClient protocol is that a third implementation slots in without any change to run_and_record or anything upstream of build_engine.

Validation and edge-case handling #

• Confidence thresholds differ by engine and must not be compared raw. Tesseract, Textract, and Azure Document Intelligence do not compute confidence identically; route low-confidence words to human review using a threshold calibrated per engine, not one constant shared across all three.
• A page with no machine-readable table still needs a schema. When Tesseract is the configured engine and a document contains a table, tables is always empty; downstream code must know to expect that and fall back to a manual transcription queue rather than treating an empty list as “no table exists.”
• PHI region pinning is a configuration fact, not a runtime check. Confirm the AWS region or Azure endpoint used for OCR is the one covered by the business associate agreement before enabling either cloud engine — the code above trusts AWS_TEXTRACT_REGION and AZURE_DOCINTEL_ENDPOINT completely, so a wrong value there is a compliance failure the pipeline itself cannot detect.
• Switching engines mid-trial changes historical comparability. If a study starts on Tesseract and later adds a cloud engine for new sites, tag every OcrResult with engine — already included above — so an analysis of extraction quality over time can account for the change rather than treating it as noise.

Testing and verification #

Tests exercise the protocol boundary rather than any one vendor, using a fake engine that stands in for all three.

"""Tests for the OCR engine abstraction."""
from __future__ import annotations


class FakeEngine:
    def __init__(self, result: "OcrResult") -> None:
        self._result = result

    def extract(self, image_bytes: bytes) -> "OcrResult":
        return self._result


def test_run_and_record_logs_engine_and_counts(caplog) -> None:
    fake_result = OcrResult(
        page_text="protocol 1234",
        words=[WordConfidence("protocol", 96.0), WordConfidence("1234", 91.5)],
        tables=[ExtractedTable(rows=[["a", "b"]])],
        engine="tesseract",
    )
    engine = FakeEngine(fake_result)
    with caplog.at_level("INFO"):
        result = run_and_record(engine, b"fake-bytes", document_id="doc-1")
    assert result.engine == "tesseract"
    assert "ocr_extraction_completed" in caplog.text
    assert "doc-1" in caplog.text


def test_low_confidence_word_is_detectable() -> None:
    result = OcrResult(
        page_text="site 004",
        words=[WordConfidence("site", 98.0), WordConfidence("004", 61.0)],
        engine="tesseract",
    )
    low_confidence = [w for w in result.words if w.confidence < 80.0]
    assert len(low_confidence) == 1
    assert low_confidence[0].text == "004"

The first test proves the audit event fires with the right engine and document identity regardless of which concrete client produced the result; the second proves a threshold-based review filter can be applied uniformly on top of any engine’s output.

FAQ #

Does using a cloud OCR engine mean giving up control over PHI? #

Not necessarily, but it changes where control is enforced. Self-hosted Tesseract keeps data inside your network by construction; a cloud engine requires an executed business associate agreement or equivalent, a region pinned to your compliance boundary, and a contractual guarantee about data retention and deletion. Both can be compliant — only one keeps the guarantee entirely in your own infrastructure.

Can different sites in the same trial use different OCR engines? #

Yes, provided the routing rule and audit event both record which engine processed which document. A common pattern is Tesseract for a self-hosted regional deployment and a cloud engine for sites where a signed agreement is already in place, unified behind the same protocol shown above.

Which engine is best for a drug-accountability or randomization table? #

A cloud engine with native table extraction reads structured rows and columns directly; Tesseract returns the words in roughly reading order without cell boundaries, which usually requires a layout heuristic or a human transcription step to reconstruct the table safely.

Related #

• Up one level: OCR & Metadata Extraction Pipelines
• Automated Document Ingestion & Validation Workflows
• Extracting Metadata from Scanned Clinical Trial PDFs Using Tesseract
• Security Boundaries for Clinical Data
• Core Architecture & Regulatory Mapping for Clinical Trials