Optical Character Recognition (OCR)

CSS Optional — Paper-II, Section-B · P-II.V.XIII · A heading in the standard CSS textbook's DIP chapter (not named separately in the FPSC syllabus). OCR is the best end-to-end application of DIP: it chains thresholding, morphology, segmentation, geometric correction and recognition — use it as the example in any long DIP answer.

1. Definition

Optical Character Recognition converts images of printed, typed or handwritten text (scans, photos, screenshots) into machine-encoded, editable and searchable text.

Offline vs online: offline OCR works on a finished image; online handwriting recognition also uses pen stroke order and timing (tablets, stylus input) and is generally more accurate.

2. The OCR Pipeline

graph LR
    A["Image acquisition"] --> B["Pre-processing"]
    B --> C["Layout analysis and segmentation"]
    C --> D["Feature extraction"]
    D --> E["Classification / recognition"]
    E --> F["Post-processing"]
    F --> G["Editable text"]

1. Acquisition — scanner (typically 300 dpi) or camera. Too low a resolution breaks small characters (a sampling problem).

2. Pre-processing (each step is a DIP topic):

• Grayscale conversion → Y = 0.299R + 0.587G + 0.114B
• Binarization → Otsu or adaptive thresholding (handles shadows and uneven lighting)
• Noise removal → median filter; morphological opening to remove specks, closing to repair broken strokes
• Skew correction (de-skewing) → estimate the text angle with the Hough transform or projection profiles, then rotate
• Normalization → scale characters to a fixed size; thinning to 1-pixel strokes for stroke-based features

3. Layout analysis and segmentation:

• Separate text blocks from images, tables, columns
• Lines → horizontal projection profile (rows of zero black pixels = gaps between lines)
• Words / characters → vertical projection profile or connected-component labelling
• Hard cases: touching characters (need splitting), broken characters (need merging)

4. Feature extraction — describe each character with numbers:

• Zoning: divide the character into a grid, count black pixels per zone
• Projection histograms, chain codes of the outline
• Structural features: loops ("B" has 2, "A" has 1), end points, junctions, stroke directions
• Modern systems learn features automatically with CNNs

5. Classification / recognition:

• Template (matrix) matching — compare with stored glyphs by correlation; simple but font- and size-sensitive
• Statistical classifiers — k-nearest neighbour, SVM on the feature vector
• Neural networks — CNNs per character; CNN + LSTM (CRNN) with CTC loss reads a whole line without splitting it into characters; Transformer models (e.g. TrOCR) for handwriting

6. Post-processing — dictionary lookup and language models fix errors ("C0mputer" → "Computer"); format reconstruction (paragraphs, tables).

3. Challenges

• Handwriting variability; cursive and joined scripts
• Low resolution, blur, noise, poor lighting in phone photos
• Skewed, curved or warped pages (book spines)
• Complex layouts: multi-column pages, tables, mixed text and images
• Similar-looking characters: O / 0, l / 1 / I, S / 5, rn / m
• Multilingual and right-to-left text

Urdu Nastaliq — a Pakistan-relevant challenge:

• Cursive: characters join, and each letter's shape changes with its position (initial / medial / final / isolated)
• Diagonal, sloping baseline — words descend right-to-left, so line and character segmentation is very hard
• Dots (nuqtas) and diacritics sit above or below and are easily lost as "noise"
• Heavy ligatures — so Urdu OCR usually recognizes whole ligatures (segmentation-free) instead of single characters, increasingly with deep learning (CNN-LSTM)

4. Measuring Accuracy

• Character Error Rate CER = (S + D + I) / N — substitutions, deletions and insertions needed to turn the output into the correct text, divided by the number of reference characters
• Word Error Rate (WER) — the same at word level
• Accuracy = 1 − CER. Clean printed English typically reaches 98–99%+ character accuracy; handwriting and degraded scans are far lower.

5. Applications

Open-source engine to name in answers: Tesseract (originally HP, later Google; LSTM-based since version 4).

---

📐 Worked Numericals

N1 — Character Error Rate: reference PAKISTAN (N = 8); OCR output PAK1STN.

• Substitution: I → 1 (S = 1)
• Deletion: the second A is missing (D = 1); insertions I = 0
• CER = (1 + 1 + 0) / 8 = 25% → accuracy 75%

N2 — Line segmentation by projection profile: black-pixel counts per row of a binarized page: [0, 0, 12, 15, 14, 0, 0, 10, 13, 11, 0].

• Non-zero runs: rows 2–4 and rows 7–9, separated by zero rows 5–6
• → 2 text lines found

N3 — Zoning features: a 4 × 4 binary "Z":

1 1 1 1
0 0 1 0
0 1 0 0
1 1 1 1

Split into four 2 × 2 zones and count 1s: top-left = 2, top-right = 3, bottom-left = 3, bottom-right = 2 → feature vector [2, 3, 3, 2], which a classifier compares against stored vectors.

N4 — Template matching: template for "I" = 010 / 010 / 010.

• Candidate X = 010 / 010 / 011 → 8 of 9 pixels match → 88.9%
• Candidate Y = 111 / 010 / 010 (a "T") → 7 of 9 match → 77.8%
• X is recognized as "I" — but the small gap between scores shows why pure template matching is fragile with noise and font changes.

N5 — Skew angle: a text baseline runs from (0, 100) to (1000, 135) pixels.

• tan θ = 35 / 1000 = 0.035 → θ ≈ 2.0° → rotate the page by −2° before segmentation (a geometric transformation).

N6 — Why 300 dpi: 10-point text has a cap height of about 0.1 inch. At 300 dpi → ≈ 30 pixels tall (reliable); at 72 dpi → ≈ 7 pixels (too few to separate similar letters).

---

🎯 CSS Exam Angle

• "What is OCR? Explain its steps": definition → pipeline diagram → one line per stage, naming the DIP technique used (Otsu, median, opening, Hough, projection profile) → challenges → applications. Linking each step to a syllabus technique is what lifts the answer.
• Add the Urdu Nastaliq paragraph for a Pakistan-specific edge, and ANPR / CNIC / OMR as local applications.
• A CER calculation (N1) is a quick, safe numerical.

📌 Past CSS Paper Frequency

⚠️ MCQ Traps

• OMR detects marks, not characters; MICR is magnetic, not optical.
• ICR is for handwriting; plain OCR is for printed text.
• Binarization is a thresholding step; skew correction is a geometric (rotation) step.
• CTC-based line recognizers do not need character segmentation.
• CER counts substitutions + deletions + insertions, divided by the reference length.
• Tesseract is open-source.

Self-Check

1. Name the pre-processing steps of OCR and the DIP technique behind each.
2. Why is Urdu Nastaliq harder to recognize than printed English?
3. Differentiate OCR, ICR and OMR with one example each.

📜 Past-Paper Questions & MCQs (CSS 2016–2026)

MCQs

2024 P-II · Q17. Optical character recognition:

• (A) Cannot be done without non-deterministic algorithms
• (B) Can be done without non-deterministic algorithms ✅
• (C) Can be done more efficiently and robustly with deterministic algorithms
• (D) None of these
• Answer: (B) (best reading).
• Why: OCR can be done with deterministic methods such as template matching, feature rules and decision trees, so (A) is false. (Modern OCR uses neural networks, but they are not required in principle.)
• Note: (C) is also arguable, because the claim that deterministic methods are more robust is doubtful.