Document Automation with Bedrock: A Modernization Journey

The legacy system was a three-layer composition that grew organically over eight years. At the base, an on-premises OCR engine (Tesseract with custom post-processing) running on EC2 c5.2xlarge instances with manual autoscaling. In the middle, a Python rule set — over 4,200 lines of regular expressions and conditional logic — attempting to normalize fields like tax IDs, dates in multiple formats, and monetary values with regional separators. At the top, an SQS queue feeding an RDS PostgreSQL database, with a human review layer triggered by a fixed confidence threshold of 70%.

The problem was not that the system did not work. It was that it failed in ways nobody could measure. Extraction accuracy for out-of-pattern documents dropped to 34% without any alarm firing, because the system had no extraction quality observability — only throughput metrics. The operational cost of human review consumed 61% of total pipeline cost, but that number was buried in an 'operations' cost center line, not attributed to the system.

When we ran a full diagnostic, we found three structural failures: no per-document traceability (no way to know which rule version extracted which field on which date), tight coupling between extraction schema and rule code (any new document type required an engineering sprint), and a complete absence of audit trail for regulatory purposes. For an environment under Central Bank supervision, this was classifiable operational risk.

Bedrock Data Automation is not an OCR wrapper with an LLM on top. The important operational distinction is that it operates on an extraction blueprint — a JSON schema defining expected fields, types, validations, and extraction instructions in natural language. This fundamentally changes the maintenance model: instead of debugging Python regex, you iterate on the blueprint and version it in S3.

In practice, we configured separate blueprints per document family, not per exact type. A 'income documents' blueprint covers pay stubs, tax returns, and bank statements with conditional instructions. This reduced the number of blueprints from 23 to 7, with equivalent coverage. Each blueprint is versioned with an immutable ARN — when we update, we create a new version and the pipeline continues using the previous version until the new one passes shadow mode validation.

The most relevant technical attention point is handling multi-page documents with distributed fields. Bedrock Data Automation processes the document as a unit, but fields like 'total value' in a 40-page contract may be on the last page while 'contracting parties' are on the first. We configured page_range_hints in the blueprint for document types where we know the field distribution, which reduced average contract processing latency from 11.2s to 6.8s without accuracy loss — the model does not need to 'search' for the field across the entire document.

For documents with complex tables (financial statements, for example), Bedrock Data Automation's structured output includes bounding box coordinates per cell. We store these coordinates in the audit trail, allowing a human auditor to see exactly where in the physical document each value was extracted from — something impossible in the legacy system.

The choice of Step Functions Express Workflows for orchestration was deliberate and not obvious. Express Workflows have a maximum duration of 5 minutes and do not persist state between executions — this seemed like a problem for documents entering human review, which can take hours. The solution was to split into two workflows: an Express Workflow for the happy path (extraction + validation + delivery, p99 at 12s), and a Standard Workflow for the human review path, which can last up to 24h with native wait state for the A2I callback.

The callback pattern is implemented with sendTaskSuccess / sendTaskFailure via the A2I API: when the reviewer completes the task in the A2I interface, a Lambda is triggered that calls sfn:SendTaskSuccess with the task token stored in DynamoDB. This eliminates polling and keeps Standard Workflow cost low — you pay per state transition, not per wait time.

An idempotency detail that cost a sprint to get right: the initial routing Lambda can be invoked more than once for the same document (S3 event delivery guarantees at-least-once). We implemented deduplication via DynamoDB with a 24h TTL: before invoking Bedrock, the Lambda checks whether document_id + s3_etag already exists in the table. If so, it returns the cached result. The s3_etag is critical here — document_id alone is not sufficient, because the same document can be resubmitted with corrections.

For observability, each Step Functions execution emits events to EventBridge, which feeds a Kinesis Data Firehose → S3 for historical analysis and a Lambda that publishes custom metrics to CloudWatch. X-Ray is enabled on all Lambdas and Step Functions, allowing latency tracing for each step individually — we identified that 34% of total latency was in the routing Lambda cold start, resolved with Provisioned Concurrency of 5 instances during peak hours.

The hardest question in this migration was not technical — it was governance. When an AI model extracts an income value that will feed a credit decision, who is responsible for the error? The regulatory answer requires the institution to demonstrate it has sufficient controls to detect, correct, and trace errors, regardless of origin (human or algorithmic).

We implemented three governance layers that go beyond what most reference architectures suggest. First, versioned model card: for each blueprint version and Bedrock model in use, we maintain a structured document in S3 with: production entry date, measured accuracy rate by document type, known limitations, and risk committee approval. This document is referenced in the audit trail of each extraction.

Second, confidence distribution monitoring: beyond auto-approval rate SLOs, we monitor the statistical distribution of confidence scores by document type week over week. A shift in distribution (even without SLO violation) is an early signal of model drift or a change in the pattern of incoming documents. We implemented this with CloudWatch Metric Math calculating the 25th percentile of the confidence score — if it drops more than 8 percentage points in 7 days, it automatically opens an investigation ticket.

Third, operationalized right to explanation: for each credit decision that used data extracted by the pipeline, the downstream system can query the audit API (Lambda + API Gateway with IAM authorizer) and receive the full audit trail for that document, including excerpts from the original document with bounding boxes highlighting the extracted fields. This is not just compliance — it is the ability to respond to a customer dispute in minutes, not days.

Migrations to managed AI services frequently underestimate real cost because they compare legacy compute cost with the new service's API cost, ignoring adjacent costs. I will be specific about what we measured.

In the legacy system, monthly cost for 180,000 processed documents was: EC2 (c5.2xlarge × 4 instances, 24/7): $1,104; RDS PostgreSQL (db.r5.large Multi-AZ): $420; human review cost (analysts, 61% of time in review): $8,200; rule maintenance (0.3 engineering FTE): $2,100. Total: ~$11,824/month.

In the new system, for the same volume: Bedrock Data Automation (estimate based on public pricing, ~$0.015/page, average 3 pages/document): $8,100; Lambda + Step Functions + A2I: $340; S3 (including audit bucket with Object Lock): $180; human review cost (13% of volume, reduced time): $1,640; blueprint maintenance (0.05 FTE): $350. Total: ~$10,610/month.

The direct cost reduction is modest (~10%). The real gain is in three places that do not appear in the bill: (1) elimination of regulatory risk from absent audit trail — a Central Bank fine for lack of traceability can cost orders of magnitude more; (2) speed of onboarding new document types — from a 3-week sprint to 2 days of blueprint iteration; (3) scalability without fixed cost — the new pipeline has no idle EC2 instances on weekends, representing an additional $280/month savings during low-volume periods.

The attention point: if volume grows to 500,000 documents/month, Bedrock Data Automation cost grows linearly while EC2 cost would grow in steps. Above ~350,000 documents/month, it is worth re-evaluating whether a proprietary fine-tuned model or a hybrid solution (Bedrock for complex cases, lightweight model for simple cases) would be more cost-effective.