OCR in the browser: how Tesseract.js and WebAssembly changed everything

Optical Character Recognition is one of the older problems in computer vision. Tesseract, originally developed at HP in the 1980s and open-sourced in 2005, is still the state of the art for a self-hosted, offline OCR engine. In 2016, Guillermo Webster and contributors compiled Tesseract to JavaScript via Emscripten and shipped it as Tesseract.js. In 2020 they moved to WebAssembly. Today it does everything the C++ version does, at 90–95 percent of the throughput, inside a browser tab.

This is what we run whenever you use nctools's OCR tools. Here is how the pieces fit together, what the trade-offs are, and how we made a 220 MB language pack feel instant.

The pipeline

An OCR request in nctools follows five stages: file decode, image binarisation, layout analysis, LSTM recognition, and result serialisation. Only the middle three are Tesseract. The first is our pdfjs-dist rasteriser (for PDFs) or the browser's Image decoder (for photos). The last is a small serialiser that emits hOCR, plain text, or a searchable PDF.

1. Decoding

A PDF is rasterised page by page at 300 DPI by default — the standard for text recognition. Photos are decoded natively and downsampled to 300 DPI equivalent. We use OffscreenCanvas so the main thread stays responsive during a 200-page batch.

2. Binarisation and cleanup

Tesseract's LSTM recogniser expects black text on a white background. We apply a mild Otsu threshold and a 3x3 median filter to remove scan noise. For phone photographs we additionally correct perspective with a homography from the four detected page corners.

3. Layout analysis and recognition

Tesseract's page segmentation mode (PSM) tells it what to expect. We default to PSM 3 (fully automatic) but expose PSM 6 (single block of text) and PSM 11 (sparse text) for edge cases. Recognition runs in a Web Worker so the UI stays smooth.

import { createWorker } from "tesseract.js";

async function ocr(imageBlob: Blob, language = "eng"): Promise<string> {
  const worker = await createWorker(language, 1, {
    workerPath: "/tesseract/worker.min.js",
    corePath: "/tesseract/tesseract-core.wasm.js",
    langPath: "/tesseract/lang",
    logger: (m) => console.debug(m),
  });
  const { data } = await worker.recognize(imageBlob);
  await worker.terminate();
  return data.text;
}

The language pack problem

Each trained language weighs 10-30 MB. English alone is 22 MB. Sending that on every page load would murder our Lighthouse score and cost users on metered connections a small fortune. We solve it with three techniques.

• Lazy loading — no model is downloaded until the user picks a language.
• Cache API — once fetched, models live in the origin cache indefinitely and are used across sessions.
• Streaming decompression — we serve the .traineddata files gzipped and pipe them through DecompressionStream in the browser, which cuts transfer by 60 percent on average.

Accuracy in the wild

On the standard IIIT 5K benchmark, Tesseract 5 with the LSTM engine hits 95.6 percent character-level accuracy on clean printed text. On our internal corpus of law-firm scans (mixed quality, some handwriting), we see 92 percent. Numbers alone reach 99 percent because we run the digits-only model as a second pass on any block flagged as an amount, date, or invoice number.

What OCR is not

OCR is not text extraction. If a PDF already has a text layer — most modern PDFs do — extracting that layer with pdfjs-dist is faster and perfect. We check for a text layer first, and only fall back to OCR when it is missing or shorter than expected.

OCR is also not translation, not summarisation, and not entity recognition. All of those are downstream tasks that need a language model. We ship a Web LLM integration for on-device summarisation on capable browsers, but that is a topic for another post.

Try it

Drop a scan at /tools/pdf-to-ocr and watch the text stream out. Nothing uploaded, ever.