- 1Extract
- 2Normalize
- 3Review
Prescription
Sample documentStructured extraction
Information extracted from the document using OCR and AI.
On this page
AT A GLANCE
A document pipeline that turns medical prescriptions into structured, review-ready data without hiding uncertainty.
My contribution
Pipeline architecture · structured extraction · validation · n8n-to-Python migration · RAG · human-in-the-loop design.
Key facts
Key engineering decisions
Keep uncertainty visible
Use confidence, alternatives, and review paths instead of presenting a structured extraction as a certain answer.
Preserve the visual workflow before migrating
Move the validated n8n flow into FastAPI, LangGraph, and Pydantic without discarding its business rules and branches.
Send the document to the model
Make the image or PDF a real multimodal input so the service can reproduce the intended document-reading step.
Retrieve from controlled catalogs
Use CSV-backed candidates for normalization and classification, keeping source options reviewable and replaceable.
Project evolution
- n8n workflow
- Structured extraction and validation
- Python API migration
- Human-review paths
- CSV catalogs and RAG
Reading the document is not enough
In Argentina, medical prescriptions are often still written by hand. A single slip of paper can mix patient IDs, diagnoses, medications, abbreviations, stamps, and hurried handwriting. Standard OCR can pull text off the page, but it can’t tell you whether a medication name makes clinical sense or whether the reading is reliable enough to process automatically.
This work is commonly resolved through administrative review: a person reads the document, identifies the relevant fields, transcribes them into another system, and decides what to do when something is unclear. The project therefore changed direction from a narrow OCR experiment into an attempt to automate a human document-processing workflow. The goal was not to remove judgment from the process, but to reduce repetitive transcription and direct human attention according to risk.
Ambiguous readings and prescriptions involving high-cost medication were routed to a person. Low-priority, lower-cost prescriptions could continue without human intervention when the OCR confidence was above the required threshold; low-confidence extraction still triggered reprocessing or review. The challenge became designing that path while preserving the original prescription as evidence and transforming its contents into an inspectable, structured contract.
From visual workflow to maintainable service
I worked on the pipeline architecture, data contract, and its three successive implementations. The initial n8n workflow worked correctly and made it possible to design and validate the business logic quickly. Moving to Python was not a correction of that first step: it was the path to the MVP, with greater control over code, contracts, and testing.
The FastAPI and LangGraph version preserved the original workflow decisions—validate, extract, normalize, classify, decide, and consolidate—and made them explicit as nodes and transitions. Swagger served as the technical interface, Pydantic defined verifiable boundaries, and the Gemini integration was isolated behind a client. A third stage added retrieval over CSV catalogs to support normalization and classification with controlled candidates.
Three stages of the system
1. n8n workflow
The first version implemented the full journey as a visual workflow: webhook intake, document validation, OCR, normalization, classification, branching, and structured response. During the first client demo, n8n made it possible to run the process live and follow it node by node, showing how the document changed at each stage before committing the flow to Python. That visual walkthrough turned evolving requirements into something concrete: steps, exceptions, and output fields could be discussed and fine-tuned directly against a working process.
2. Python MVP
The flow moved to FastAPI, LangGraph, and Pydantic to gain modularity, typing, error handling, and testability while preserving the behavior defined in n8n.
3. CSV and RAG
The third implementation added retrieval over controlled catalogs. The system searched for relevant candidates before normalizing medications or assigning the type, subtype, and procedure hierarchy.
A pipeline with control points
Simplified architecture
The document moves through the system as typed state. Each stage produces inspectable information, and validation decides whether the flow can continue.
Intake
Receive an image or PDF with optional patient and membership metadata.
Document validation
Check that the file is processable and matches the expected document type.
Vision and extraction
Send the actual file to the multimodal model and map the reading into a schema.
Normalization
Correct medication candidates using the full prescription, diagnosis, and specialty as context.
Classification and control
Assign categories and evaluate consistency, confidence, and retry requirements.
Risk-based routing
Automatically continue confident, lower-cost cases; route ambiguous, low-confidence, or high-cost prescriptions for human review.
The output contract is part of the product
Extraction was designed around fields such as prescription validity, membership ID, patient, insurer, diagnosis, date, medical specialty, full directions, and a medication list. Missing data was meant to remain null: silently filling a field was worse than acknowledging it was unavailable.
Normalization added another evidence layer for each medication: original text, proposed correction, strength, frequency, alternative candidates, diagnosis match, selected confidence, and suggested action. A consumer could therefore distinguish a direct reading, a normalized hypothesis, and a still-uncertain case.
Context becomes evidence
- Decision
- Calculate confidence from the document as a whole, combining OCR quality with patient metadata and contextual clues from the prescription.
- Rationale
- The surrounding details can help disambiguate a weak reading. Known patient information can be compared with the extracted identity, while the issuing institution, diagnosis, and specialty provide clues about the medication. For example, a prescription headed by an ophthalmology clinic makes an eye-care medication more plausible than an unrelated candidate with a similar handwritten name.
- Tradeoff
- Context is supporting evidence, not proof. A clinic may cover several specialties, metadata may be incomplete, and a plausible medication can still be read incorrectly. These signals can adjust confidence but must not override contradictory document evidence.
- Outcome
- The confidence score reflects agreement across OCR quality, patient data, clinical context, and catalog candidates. Weak or contradictory signals lower confidence and move the prescription toward reprocessing or human review.
The model had to receive the file
- Decision
- Treat the image or PDF as a real multimodal input, not an attachment reduced to its filename and metadata.
- Rationale
- An early integration could read the bytes in FastAPI and then build the prompt using only the filename and text context. The endpoint worked, but it did not reproduce the original OCR because the model never saw the document.
- Tradeoff
- Sending multimodal content requires MIME and size validation, reliable multipart handling, and awareness of provider limits. It also makes it more important to log failures without retaining sensitive medical information.
- Outcome
- The architecture made the binary path to Gemini Vision explicit and separated upload, inference, and parsing failures so they could be tested independently.
Designing for uncertainty
Preserve the original reading
Keep the detected text next to any proposed correction so the OCR evidence is not erased.
Use clinical context
Compare the medication with patient metadata, the full prescription, diagnosis, specialty, and issuing medical institution before selecting a normalization.
Measure consistency
Expose confidence, alternatives, and diagnosis matching instead of returning one opaque answer.
Choose an action
Continue automatically for confident, low-priority or lower-cost prescriptions; rerun OCR when confidence is insufficient; or request human review for ambiguity and high-cost medication.
Third stage: controlled catalog
- Decision
- Add retrieval over curated CSV catalogs to support normalization and classification.
- Rationale
- Relying only on the model's parametric knowledge limited traceability. The third version retrieved medication candidates or a type, subtype, and procedure hierarchy; the model selected and structured its answer from that bounded set.
- Tradeoff
- Retrieval adds catalog versioning, semantic search, and ranking criteria. In return, options can be traced to a managed source and updated without retraining the model.
- Outcome
- Separating OCR, normalization, retrieval, and validation made it possible to add RAG without replacing the API or the complete graph. The catalog became a controlled source of candidates inside the pipeline; this does not imply that the solution reached production validation.
A deliberately small technical surface
The prototype exposed an upload endpoint to run the graph, a health check, and the OpenAPI specification generated by FastAPI. Swagger made it possible to exercise the flow with documents and metadata before building a dedicated interface.
Error handling distinguished model failures, unparseable responses, multipart problems, and internal exceptions. That turned a generic HTTP 500 into actionable integration information without confusing server availability with validation of the entire pipeline.
Prototype outcome
The work produced three connected versions of the system: a functional n8n workflow to define and test the journey, a modular Python MVP with an API and interactive documentation, and a later CSV/RAG stage that retrieved candidates from controlled sources.
The case study does not publish accuracy metrics or present the system as production-ready because reviewed results over a representative set are still unavailable. Its value is in showing how a valid visual automation became a testable service and then incorporated retrieval without rebuilding the pipeline from scratch.
Lessons
In document processing, a structured answer is not the same thing as a correct answer. Schemas improve the contract; confidence, candidates, and human review improve the decision.
The migration also showed that copying nodes one by one is not enough. Real parity depends on preserving the binary input, branches, retries, and the meaning of each field. The natural next stage would be to build an anonymized dataset, evaluate by field and document type, and only then tune thresholds or make operational quality claims.