Definition
You photograph a printed bill, and your phone turns it into editable text in a second. Post offices read handwritten PIN codes automatically, and old books in Hindi and Tamil are being made searchable. All of this is optical character recognition.
Optical character recognition (OCR) converts images of printed or handwritten text into machine-readable characters. A classic OCR pipeline has stages: preprocessing (converting to greyscale, binarising, removing noise and straightening skew), layout analysis (finding text blocks and lines), segmentation (splitting lines into words and characters), recognition (classifying each character) and post-processing (correcting errors with a dictionary or language model). Modern systems such as Tesseract 4 and later use neural networks, often a CNN followed by an LSTM or transformer, to recognise whole lines without explicitly cutting out each character.
In simple words
Reading a messy handwritten note, you first find the lines, then the words, then each letter, and when a letter is unclear you guess from the word around it. OCR follows the same steps: find, split, recognise, then fix mistakes using context.
Overview
- Optical character recognition (OCR) turns images of text into machine-readable characters.
- A pipeline preprocesses the image, finds text, recognises it and corrects errors.
- Tesseract, started at HP in the 1980s and later developed by Google, uses an LSTM recogniser since version 4.
- Indian scripts such as Devanagari need line-level recognition because letters join under the headline.
- Quality is measured with the character error rate and word error rate.
- Critical fields such as amounts and IDs should be validated.
Why it matters
Huge amounts of information still live on paper: land records, invoices, forms, exam sheets, books and signboards. OCR makes them searchable and usable by software, powers document-scanning apps and number-plate readers, and helps blind and low-vision people through screen readers.
Key terms
- Binarisation
- Turning a greyscale image into black ink and white background, for example with Otsu's threshold.
- Deskewing
- Rotating a scanned page so that text lines are horizontal.
- Segmentation
- Splitting a page into lines, words and characters.
- Projection profile
- Counting ink pixels per row or column to find gaps between lines or characters.
- Character error rate (CER)
- Insertions, deletions and substitutions divided by the number of true characters.
- Intelligent character recognition
- OCR for handwriting, which varies far more than print.
Types
OCR systems come in several types, depending on the kind of text and document.
1. Printed-text OCR
Reading machine-printed text in books, forms and screens.
- How it works: Regular fonts make recognition accurate after binarisation and deskewing; character error rates can be below 1% on clean scans.
- Example: Digitising old government gazettes so they can be searched.
- When to use it: Use it for large-scale document digitisation and search. Validate key numbers with format rules.
2. Handwriting recognition (ICR)
Reading handwritten characters and words.
- How it works: Neural line recognisers trained with CTC loss handle varied styles; isolated digits are easiest, cursive text hardest.
- Example: Reading handwritten PIN codes on letters; the course example read 110001 correctly.
- When to use it: Use it for forms, exam sheets and postal sorting, with human checks for uncertain cases.
3. Scene-text recognition
Detecting and reading text in natural photos.
- How it works: A text detector finds words at any angle or perspective, then a recogniser reads each cropped word.
- Example: Reading shop signs and vehicle number plates in street photos.
- When to use it: Use it for navigation apps, number-plate recognition and translation of signs.
4. Document AI (layout and key-value extraction)
Understanding document structure: tables, fields and reading order.
- How it works: Layout models combine text, position and image features to extract fields such as invoice number, date and total.
- Example: Pulling GSTIN, date and amount from thousands of supplier invoices.
- When to use it: Use it to automate data entry from forms, receipts and ID cards.
5. Multilingual and Indic OCR
OCR for many scripts, including the 22 scheduled Indian languages.
- How it works: Models must handle conjuncts, vowel signs and mixed-script text; national projects such as Bhashini support Indian-language OCR.
- Example: Digitising Marathi and Hindi land records for district offices.
- When to use it: Use script-specific models and verify numbers and names carefully.
Explained in detail
The OCR pipeline
Preprocessing matters as much as recognition. Photos from phones have uneven lighting, shadows and perspective, so the image is converted to greyscale, the contrast is evened out, and a threshold separates ink from paper. Small specks are removed and the page is deskewed. Layout analysis then finds columns, paragraphs, tables and images. Within a text block, a horizontal projection profile, the count of ink pixels in each row, shows lines as peaks separated by empty gaps; a vertical profile within a line shows gaps between characters. Each character image is resized to a fixed size and passed to a classifier, which might be a k-NN, an SVM or a CNN. Finally a dictionary or language model fixes likely errors, such as reading "0" in the middle of a word as "o".
Classic versus modern OCR
| Classic pipeline | Modern neural OCR | |
|---|---|---|
| Unit recognised | One segmented character at a time | A whole line or word image |
| Segmentation | Explicit, by gaps | Learned implicitly, often with CTC loss |
| Joined scripts | Hard: Devanagari's shirorekha joins letters | Handled much better |
| Handwriting | Poor | Good with enough training data |
| Examples | Early Tesseract, template matching | Tesseract 4+, cloud OCR services, TrOCR |
OCR for Indian scripts and measuring quality
Indian scripts bring special challenges. In Devanagari, Bengali and Gurmukhi, the headline (shirorekha) connects letters, so gaps between characters do not exist; conjuncts combine consonants into new shapes, and vowel signs sit above, below or beside a consonant. Tamil, Telugu and Kannada have many rounded, similar shapes. Neural line-level recognisers handle these far better than character segmentation, and national efforts such as Bhashini support OCR for Indian languages. Quality is measured by the character error rate, the edit distance between the output and the true text divided by the true length, and the word error rate. A CER of 2 per cent sounds small but means about one error in every 50 characters, roughly one per line, which may be unacceptable for legal records.
The code below runs a mini pipeline: it joins six unseen handwritten digits into one line image spelling the PIN code 110001, finds the six characters as runs of columns containing ink, centres each in an 8 × 8 box, and reads all six correctly with an SVM.
How it works
- Clean the image: greyscale, even out lighting, binarise and deskew.
- Find text regions and split them into lines.
- Recognise each line, with a character classifier or a line-level neural model.
- Correct errors with a dictionary, format rules or a language model.
- Measure the character error rate on a labelled sample and review critical fields by hand.
The maths
- Substituted, deleted and inserted characters
- Number of characters in the true text
The true text is "MUMBAI 400001" (13 characters including the space). The OCR output is "MUM8AI 40001": the B was read as 8 (one substitution) and one 0 was missed (one deletion). CER = (1 + 0 + 1) ÷ 13 ≈ 0.15, or 15 per cent. A post-processing rule that PIN codes have six digits flags "40001" as invalid, and a list of city names corrects "MUM8AI" to "MUMBAI". After these fixes, only the missing zero remains for a person to check.
In code
import numpy as np
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
digits = load_digits()
img_tr, img_te, y_tr, y_te = train_test_split(digits.images, digits.target, random_state=0)
reader = SVC(gamma=0.001, C=10).fit(img_tr.reshape(len(img_tr), -1), y_tr)
# Build an image of the PIN code 110001 from unseen test digits, with blank gaps between them
text = "110001"
picks = [np.where(y_te == int(ch))[0][i] for i, ch in enumerate(text)]
line = np.hstack([np.pad(img_te[i], ((0, 0), (1, 1))) for i in picks])
for row in line[::2]: # a rough picture of the line
print("".join(" .:#"[min(int(v) // 5, 3)] for v in row))
# Segment: characters are runs of columns that contain ink
ink = line.sum(axis=0) > 0
starts = [c for c in range(len(ink)) if ink[c] and (c == 0 or not ink[c - 1])]
ends = [c for c in range(len(ink)) if ink[c] and (c == len(ink) - 1 or not ink[c + 1])]
chars = []
for s, e in zip(starts, ends):
glyph = line[:, s:e + 1]
canvas = np.zeros((8, 8))
left = (8 - glyph.shape[1]) // 2
canvas[:, left:left + glyph.shape[1]] = glyph[:, :8]
chars.append(canvas.reshape(-1))
print("Characters found:", len(chars))
print("Read:", "".join(str(d) for d in reader.predict(np.array(chars))), "| truth:", text)Output
:: :: .:. :. .#: .##.
## :##: : :. # :. # # :##:
.##. . .#. .. .. .. . : # :#.
:##: #. .: : : .# :.:# :##
Characters found: 6
Read: 110001 | truth: 110001Try it yourself
An OCR pipeline
Put the stages in order.
- 1Recognise each character or line
- 2Split blocks into lines and characters
- 3Preprocess: greyscale, binarise, deskew
- 4Analyse the layout to find text blocks
- 5Correct errors with rules and a dictionary
Match the problem to the fix
Which step solves each OCR problem?
- Text lines slope across the page
- Grey shadow over part of the photo
- "B" read as "8" inside a word
- Devanagari letters joined by the headline
Digitising village land records
A district office scans 50,000 old land-record pages written in Hindi and Marathi.
Common mistakes
- Skipping preprocessing and blaming the recogniser for errors.
- Using character segmentation by gaps for joined scripts such as Devanagari.
- Trusting OCR output for critical numbers without validation.
Check yourself
Summary
- OCR turns images of text into characters through preprocessing, layout analysis, segmentation, recognition and correction.
- Modern neural OCR reads whole lines and handles joined scripts and handwriting better.
- Measure quality with the character error rate and verify critical fields.
Finished reading?
Sources: Tesseract OCR documentation · Bhashini, India's national language technology mission