Header

UZH-Logo

Maintenance Infos

Supervised OCR Error Detection and Correction Using Statistical and Neural Machine Translation Methods


Amrhein, Chantal; Clematide, Simon (2018). Supervised OCR Error Detection and Correction Using Statistical and Neural Machine Translation Methods. Journal for Language Technology and Computational Linguistics (JLCL), 33(1):49-76.

Abstract

For indexing the content of digitized historical texts, optical character recognition (OCR) errors are a hampering problem. To explore the effectivity of new strategies for OCR post-correction, this article focuses on methods of character-based machine translation, specifically neural machine translation and statistical machine translation. Using the ICDAR 2017 data set on OCR post-correction for English and French, we experiment with different strategies for error detection and error correction. We analyze how OCR post-correction with NMT can profit from using additional information and show that SMT and NMT can benefit from each other for these tasks. An ensemble of our models reached best performance in ICDAR’s 2017 error correction subtask and performed competitively in error detection. However, our experimental results also suggest that tuning supervised learning for OCR post-correction of texts from different sources, text types (periodicals and monographs), time periods and languages is a difficult task: the data on which the MT systems are trained have a large influence on which methods and features work best. Conclusive and generally applicable insights are hard to achieve.

Abstract

For indexing the content of digitized historical texts, optical character recognition (OCR) errors are a hampering problem. To explore the effectivity of new strategies for OCR post-correction, this article focuses on methods of character-based machine translation, specifically neural machine translation and statistical machine translation. Using the ICDAR 2017 data set on OCR post-correction for English and French, we experiment with different strategies for error detection and error correction. We analyze how OCR post-correction with NMT can profit from using additional information and show that SMT and NMT can benefit from each other for these tasks. An ensemble of our models reached best performance in ICDAR’s 2017 error correction subtask and performed competitively in error detection. However, our experimental results also suggest that tuning supervised learning for OCR post-correction of texts from different sources, text types (periodicals and monographs), time periods and languages is a difficult task: the data on which the MT systems are trained have a large influence on which methods and features work best. Conclusive and generally applicable insights are hard to achieve.

Statistics

Downloads

59 downloads since deposited on 01 Feb 2019
59 downloads since 12 months
Detailed statistics

Additional indexing

Item Type:Journal Article, refereed, original work
Communities & Collections:06 Faculty of Arts > Institute of Computational Linguistics
Dewey Decimal Classification:000 Computer science, knowledge & systems
410 Linguistics
Uncontrolled Keywords:OCR post-correction Machine Learning Neural Machine Translation Statistical Machine Translation
Language:English
Date:2018
Deposited On:01 Feb 2019 15:44
Last Modified:25 Sep 2019 00:07
Publisher:Gesellschaft für Sprachtechnologie und Computerlinguistik (GSCL)
ISSN:0175-1336
OA Status:Green
Official URL:https://jlcl.org/content/2-allissues/1-heft1-2018/jlcl_2018-1_3.pdf
Project Information:

Download

Green Open Access

Download PDF  'Supervised OCR Error Detection and Correction Using Statistical and Neural Machine Translation Methods'.
Preview
Content: Published Version
Language: English
Filetype: PDF
Size: 1MB
Licence: Creative Commons: Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)