Old OCR text cripples language model training, and FineBooks wants to fix that at scale
Hugging Face and EleutherAI's FineBooks project evaluated 14 open-source OCR models on historical texts. The leading model, dots.mocr, achieved 97.6 percent character accuracy at a cost below two dollars per thousand pages.
The FineBooks initiative represents a collaborative effort between Hugging Face and EleutherAI to address data quality issues in language model training. By focusing on historical documents, the project aims to convert scanned archives into clean text suitable for machine learning pipelines.
High-quality optical character recognition is critical for large language models, as errors in source text can introduce noise that degrades performance. This research highlights the specific challenges posed by older printed materials, which often contain degraded fonts or layout complexities that standard tools struggle to parse.
Testing involved comparing fourteen open-source OCR systems across more than two thousand book pages. The evaluation identified dots.mocr as the top performer, balancing high precision with significant cost efficiency for large-scale data processing.
These findings suggest that affordable, accurate OCR tools can unlock vast amounts of historical knowledge for AI development. As training data requirements grow, optimizing extraction methods becomes essential for building robust and knowledgeable models.
This page provides an editorial summary based on publicly available information. It is not a republished article. Use the source link below for the original report.