AI document processing: classify before you extract

Demonstrations of document processing use one document type. Real intake receives invoices, statements, contracts, forms, correspondence and photographs of all of them, arriving together, frequently in the same email, sometimes several in one file.

That variety is the actual problem. Extraction from a known document type is largely solved; knowing which type you are looking at, and what to do with the ones that do not match anything, is where implementations succeed or quietly produce wrong records.

What we'll cover

• Variety is the real problem
• Classification comes first
• The document that is several documents
• Building it with CodeWords
• Quality of the source material
• What to do with the unclassifiable
• Where documents come from
• Retention and what you are now holding
• What the review queue actually needs
• Making it survive
• Limits worth knowing about
• What to build first
• Frequently asked questions

Variety is the real problem

Intake is heterogeneous in a way demonstrations never are.

One supplier's invoice is not another's, and neither resembles the template a vendor demoed.

The same supplier changes format without notice, usually when they change accounting system.

Scans, photographs and native files have completely different characteristics and need different handling.

Several documents arrive as one file routinely, particularly from scanners.

Design for the distribution you have, which means counting what actually arrives before choosing an approach.

Classification comes first

Decide what the document is before deciding what to extract from it.

A small, stable type list works better than a fine-grained one, since precision falls as categories multiply.

Confidence matters here more than in extraction, because a misclassified document is extracted against the wrong schema and produces confident nonsense.

Have an "other" category that a person reviews, and watch its size — growth means a new document type has appeared.

Classify on structure as well as text, since layout carries a lot of the signal.

Measure classification separately from extraction, or a failure in one is attributed to the other.

The document that is several documents

Splitting is a step, and it is the one most often missing.

Scanner batches contain several documents with no separator beyond a page boundary.

Attachments to one email may relate to different things entirely.

A document with its attachments — an invoice with a delivery note and a purchase order — is one logical item and three physical ones.

Get the boundary wrong and every downstream step operates on a mixture.

Review splits with low confidence, since a wrong split is harder to detect later than a wrong extraction.

Building it with CodeWords

CodeWords connects to more than 3,000 integrations, and the connection is made once and reused.

1. Open CodeWords and start a new automation.
2. Describe what should happen in plain language to Cody, the automation builder: where documents arrive, how they are classified, what is extracted, and what needs review.
3. Authorize the connections — the mailbox or storage where documents land, and the system receiving the results.
4. Describe the exceptions: an unrecognised type, a low-confidence classification, a multi-document file, an unreadable scan.
5. Run it over a representative sample of real intake, including the awkward ones, before anything is written.

You describe the outcome; Cody builds it, connects it, and deploys it. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.

Quality of the source material

A photograph of a document is the common case now and is harder than a scan.

Skew, shadow and partial capture are normal in phone photographs and degrade everything downstream.

Poor scans — low resolution, black and white for coloured forms — lose information irrecoverably.

Handwriting is substantially harder than print and should be expected to need review.

Detect unusable input early and ask for a better copy rather than processing it badly.

Improving capture beats improving processing. A clear instruction and an example at the point of upload changes quality more than any model choice.

What to do with the unclassifiable

Have a defined destination, not a silent failure.

A person reviews it, which is the whole point of a queue.

Record why it could not be classified, since the reasons cluster and point at what to add.

Watch the rate, because a rising unclassifiable rate means the intake has changed.

Do not force a category, which produces a wrong record rather than a flagged one and is far more expensive.

Feed reviewed items back as examples, which is how the classification improves over time.

Where documents come from

Email is the dominant channel, with all the mess that implies — forwards, signatures, embedded images, several attachments.

Upload portals produce better data and require the sender to use them, which not everybody will.

Scanners and shared drives are still common and carry no metadata about what a file is.

The source is a signal. A document from a known supplier's address is more classifiable than one from a generic mailbox.

Capture the context, since the covering email frequently says what the attachment is and is discarded by most pipelines.

Acknowledge receipt, because the sender otherwise does not know whether their document arrived.

Retention and what you are now holding

You are now storing documents that may contain personal or financial data at scale.

Decide retention per document type rather than keeping everything indefinitely.

Delete the originals when policy allows, since holding them is a liability with no benefit once the data is extracted.

Restrict access to the original more tightly than to the extracted fields, since the document contains more.

Be careful with the review queue, which accumulates exactly the documents somebody could not classify and is frequently the least controlled store in the system.

Handle deletion requests, which must reach the documents and not only the records derived from them.

What the review queue actually needs

Every real implementation has a queue, so it is worth designing rather than treating as a failure state that will eventually be engineered away.

Show the document beside the extracted fields, since a reviewer comparing two screens is slow and makes mistakes.

Highlight what the system was unsure about, so attention goes to the three fields that need it rather than all twenty.

Make correction faster than retyping, which means editing in place rather than filling a form again.

Let the reviewer say "this is not one of our types", which is information rather than an error.

Record every correction, because corrections are the highest-quality training and evaluation data available and most systems discard them.

Report the queue's size and age, since a queue growing faster than it drains is the signal that the intake has changed or the thresholds are wrong.

Making it survive

Make ingestion idempotent, since the same document arrives twice routinely.

Detect duplicates by content rather than filename, which is how the same invoice submitted twice is caught.

Keep the original alongside everything derived from it, for the retention period.

Handle provider outages by queuing, since documents arriving during an outage must not be lost.

Report the outcome: documents received, classified by type, extracted, queued for review, and unclassifiable.

Limits worth knowing about

Accuracy varies by document type and by source quality, so a single overall figure is misleading.

Handwriting and poor photographs set a floor on what is achievable.

Languages and scripts differ in support, and multilingual documents are the hardest case.

Tables and multi-column layouts are handled inconsistently and are common in exactly the documents people want to process.

No pipeline removes the review queue, and an implementation promising one is describing a demonstration.

What to build first

Classification with an explicit "other" queue, before any extraction. Count what actually arrives for a fortnight, and the distribution will tell you which types are worth extracting from and how large the review queue is going to be — which is the number that determines whether the whole thing is worth building.

Two habits make the difference. Never force an unclassifiable document into a category, since a wrong record is considerably more expensive than a flagged one and is discovered much later. And improve the capture rather than the processing, because a clear instruction at the point of upload changes input quality more than any model choice downstream.

Frequently asked questions

What is the hardest part of document processing?

Variety. Extraction from a known type is largely solved; real intake contains many types from many sources in inconsistent formats, several in one file, and types that change without notice.

Why classify before extracting?

Because a misclassified document is extracted against the wrong schema and produces confident nonsense. Classification confidence matters more than extraction confidence for that reason.

What step is most often missing?

Splitting. Scanner batches and multi-attachment emails contain several documents, and a wrong boundary means every downstream step operates on a mixture — which is harder to detect later than a wrong extraction.

What should happen to documents that cannot be classified?

A defined queue a person reviews, with the reason recorded. Never force a category: a wrong record costs far more than a flagged one, and the unclassifiable rate rising is how you learn the intake has changed.

How do I improve accuracy?

Improve the capture. A clear instruction and an example at the point of upload changes input quality more than any model choice, and photographs with skew, shadow or partial capture degrade everything downstream.

Does the covering email matter?

Yes, and most pipelines discard it. It frequently says what the attachment is, and the sender's address is itself a classification signal.

What are the retention implications?

You are now storing documents containing personal or financial data at scale. Set retention per document type, delete originals when policy allows, restrict access to originals more tightly than to extracted fields, and watch the review queue — it is frequently the least controlled store in the system.

Can the review queue be eliminated?

No. Any implementation promising that is describing a demonstration rather than real intake, where handwriting, poor photographs and novel document types set a floor.

How should a document review queue be designed?

The document beside the extracted fields, uncertain values highlighted so attention goes to the three that need it, correction by editing in place rather than retyping, and an option to say the document is not one of your types.

Why record corrections?

Because they are the highest-quality evaluation data available — real inputs with known correct outputs — and most implementations discard them, which is why their accuracy never improves.

Related reading

• AI data extraction: accuracy and the review threshold
• Automating PDF generation from data
• Workflow automation for finance teams