01
Document-extraction pipeline
Insurance documents in, structured records out: classified, read by vision and language models, checked, and saved.
- Role
- Full Stack Developer · Mavericks Tech Solutions
- When
- Sep 2025 – present
- Relevant for
- AI & AgentsSoftware
Now · each document type matched to the model that measured best · lower cost per document
in
- PDF upload
- blob URI
- Excel
sorted by type → read by
fields out · json
Named Insured
…
Policy Number
…
Effective Date
…
Occupancy: Owner Occupied
…
Building Coverage
…
Producer / Agency
…
01 Intake. A PDF upload, a blob URI or an Excel file. Each lands in Azure Blob Storage under one record ID.
- VLM
- Vision-language model
- OCR
- OCR + LLM
- CHK
- Layout / vision model
My part
I started by testing this pipeline, then moved into prompt engineering to normalize the extracted data, model evaluation, the review screen and deployment.
Problem
Documents arrive by email in dozens of layouts and versions. Reviewers were reading them by hand and retyping the fields that mattered into another system.
Approach
Attachments land in blob storage, and a classifier routes each document by type. Accuracy first reached about 98% with Azure Document Intelligence and OpenAI vision models. We then benchmarked open-source models to cut cost and moved to a hybrid: NuExtract 3 with Docling for checkboxes on dense forms, and PaddleOCR then Kimi for tables and general PDFs.
Outcome
Documents that used to be keyed in by hand arrive as structured records. Every value links back to its bounding box on the page, and low-confidence fields go to a human.
What I did
~98% field-level accuracy
Measured by hand: every extracted field checked against the scanned source, because the models that extract can't reliably grade themselves.
Azure Document IntelligenceAzure OpenAI
PDF, blob URI or Excel in
The API accepts a PDF upload, a blob URI for a file already in storage, or an Excel spreadsheet. Services pass the blob URI along, never the file itself.
Azure Blob StorageREST APIs
5+ models benchmarked
Open-source document models tested on one labelled set, with checkboxes scored as their own benchmark.
NuExtract 3PaddleOCRDoclingKimi
Clean, normalized fields
Prompts and validation turn raw extraction into clean values: currencies to integers, loose dates to ISO-8601, checkboxes to booleans.
Review screen
Extracted fields beside the page, linked by bounding boxes, with flagged fields first.
ReactNext.js
CI/CD on every build
YAML pipelines, Dockerfiles and quality gates on every build.
Azure DevOpsDocker
See also
Stack
- Python
- FastAPI
- PostgreSQL
- Azure Document Intelligence
- Azure OpenAI
- NuExtract 3
- PaddleOCR
- Docling
- Kimi
- Next.js