Senior Python Software Engineer/Open-Source Contributor

97.6% on 300-year-old books. Then came the blank pages. August 11, 2026 · 00:12 UTC · News Tl;dr. On August 10, Hugging Face and EleutherAI published FineBooks, starting with a leaderboard that scores 14 open-weight OCR models on all 2,165 pages of an expert-transcribed corpus of 18th and 19th century natural history books. The top models read old print at roughly 97.6% character accuracy, and the whole run costs between $0.34 and $1.94 per 1,000 pages on rented GPUs. The interesting failure is not archaic type. It is that several models, handed a plate or a blank leaf, write paragraphs that were never there. The benchmark nobody had. Measuring OCR needs pages whose correct transcription is already known, character for character. Almost nobody has that for historical books. FineBooks found a set: between 2011 and 2012, the EU's IMPACT digitisation programme and BHL-Europe hand-corrected six volumes from the Biodiversity Heritage Library, 2,165 pages in English, French, German and Latin, down to roughly one error in two thousand characters. Both projects are long dead and their websites are gone. The transcriptions survived because they were released under CC-BY and preserved in a GitHub repository by IMPACT's technical manager, Clemens Neudecker. Somebody in 2012 picked an open license, which turns out to be the least glamorous form of time travel available to Ai Bacon: none of the models on this board existed then, and most of their architectures had not been invented. The rebuilt ground truth ships as finebooks/bhl-impact-gt, with every scan matched to its transcription, and the scorer ships as bhl-ocr-eval. One methodology note deserves to be copied by every other leaderboard: they ran every model themselves under a pinned container image, pinned model revision, pinned script commit and a recorded job id. Scores from hosted inference routers are refused outright, because a router will not tell you which provider served the request, at what quantization, or under what serving config, and character error rate moves with all three. The board. The headline metric is character error rate on the reading lane, which folds case, the long s and ligatures into their modern equivalents. dots.ocr and dots.mocr land at 2.35% and 2.37%, which the project's own results file calls statistically identical. OvisOCR2 takes third at 3.05%, PaddleOCR-VL-1.6 fourth at 3.92%. Now read that chart again from the bottom. Tesseract 5, a non-neural engine whose lineage predates the transformer by decades, comes in at 6.42% and lands within a point and a half of Qwen3.5-9B, the largest model on the board. It also never once fell into a repetition loop, which is more than any neural row can say. Cost splits the top differently than accuracy does. dots.mocr costs $1.94 per 1,000 pages on Hugging Face Jobs; PaddleOCR-VL-1.6 costs $0.34 for 96.1% accuracy. That is 5.7x the price for 1.5 points of characters, which is a trade you should make deliberately rather than by grabbing the top row. Params are measured, not quoted. The board sums every weight in each repo's safetensors index, vision towers included, instead of trusting the name. GLM-OCR, usually listed at 0.9B, is 1.33B. olmOCR-2-7B is 8.29B. Winning a size class is easier when you get to pick the number on the label. The blank-page problem. 428 of the 2,165 pages are sparse: engraved plates, blanks, near-empty leaves whose entire correct transcription might be one line of credits. The board scores them as their own stratum, and this is where the models stop resembling each other. Character error rate is a ratio: mistakes divided by the number of characters that should be on the page. When the denominator is tiny and the model invents text, the ratio blows past 1.0 and keeps going. A sparse CER of 26.93 means the model produced roughly 27 wrong characters for every real one. It is the OCR equivalent of an eyewitness giving a detailed description of a suspect at a crime scene where nobody was standing. That contrast is the finding. GLM-OCR reads ordinary content pages at 2.36% CER, better than everything except the dots pair, and then produces 26.93 on sparse pages. LightOnOCR-2 posts 15.79, olmOCR-2 posts 13.64, Qwen3.5-9B posts 19.98. Meanwhile dots.ocr sits at 1.79 and DeepSeek-OCR-2 at 2.09. If you are building a document pipeline, this is the column to weight, because real collections are full of plates, blanks, dividers and ads. An aggregate CER that averages over them will quietly hide a model that hallucinates whenever the page runs out of words. Why this matters if you never touch a library. Public domain books are one of the largest pools of long-form text that anyone can legally train on, and the text quality is currently set by whatever OCR tool a library happened to run once, at scan time, years ago. FineBooks cites the Talkie project's measurement of what that costs: a language model trained on OCR-derived text learned at 30% of the efficiency of the same model trained on human transcriptions of the same books. That reframes OCR from a digitisation chore into a training-data multiplier. When EleutherAI and collaborators shipped Common Pile, the largest openly licensed corpus to date, it carried around 300,000 public domain books whose text came from those earlier pipelines. Re-running them is, in the project's words, one of the highest-leverage improvements available to open training data. The next phase is already stated. The Biodiversity Heritage Library holds more than 300,000 digitized items and more than 64 million pages; roughly 200,000 of those items carry public domain rights statements, and FineBooks intends to re-run OCR across them with a leading model and publish the result as its first dataset. Do the arithmetic on their own cost figures and 64 million pages runs about $22,000 of GPU time at $0.34 per thousand, or about $124,000 at $1.94. The public domain slice is smaller than that, which puts a full re-processing somewhere between a grant line item and a rounding error in a frontier training budget. What the board does not tell you. The authors are unusually direct about scope, so take the caveats at full strength: * Four languages, one typeface family. English, French, German and Latin, all antiqua. The board says nothing about German Fraktur, non-Latin scripts, non-European languages or handwriting. * Books only. Single-column running text with consistent layout. Newspapers, magazines and archival material have different failure modes that are not measured here. * The confidence intervals are wide on purpose. The bootstrap resamples the six volumes rather than individual pages, so it is honest about book-to-book variation. Where intervals overlap, the models are tied, and the top two overlap. * Loop rate is a selection effect. Pages where a model repeats itself into its token cap are excluded from that row's other numbers, so heavy loopers look better than they are. PaddleOCR-VL-1.6 loops on 6.47% of pages; the top row is scored on 96.1% of the corpus. The verdict splits by use case. Good enough for LLM training corpora: yes, comfortably better than the legacy text sitting in open corpora today. Good enough for libraries replacing legacy OCR: it depends, because legacy pipelines emit ALTO XML with word-level coordinates and these models emit Markdown or plain text, sometimes with region boxes but never word positions. Good enough for scholarly transcription: not quite, since the models silently modernise the long s and ligatures that a diplomatic transcription exists to preserve. SmolDocling at 19.65% CER is a reminder that the size floor is real, too. Key takeaways. * FineBooks, a Hugging Face and EleutherAI collaboration, scored 14 open-weight OCR models on all 2,165 pages of expert-transcribed 18th and 19th century books, with the ground truth, harness and per-run provenance published alongside. * dots.ocr (2.35% CER) and dots.mocr (2.37%) lead and are statistically tied; the largest model on the board, Qwen3.5-9B at 9.65B parameters, finishes eighth at 5.07%. * On the 428 sparse or blank pages, several models invent text wholesale: GLM-OCR scores 2.36% on content pages and 26.93 on sparse ones, with Qwen3.5-9B at 19.98 and olmOCR-2 at 13.64. * Cost runs $0.34 to $1.94 per 1,000 pages on rented GPUs, which makes re-OCR of an entire collection a budgeting question rather than a research question. * The motivation is training data: a model trained on OCR-derived text learned at 30% of the efficiency of one trained on human transcriptions, and Common Pile carries around 300,000 books produced by older pipelines. * Scope is narrow by design: antiqua typefaces, four languages, single-column books, no Fraktur, no handwriting, no newspapers. AI OCR Open Weights Hugging Face EleutherAI Benchmarks Datasets Training Data AI Bacon Weekly Liked this? Get smarter about AI, weekly. The model launches that matter, the local-LLM tricks worth stealing, and the tools actually shipping. Hand-curated into one email a week, cut down to signal. No hype, ever. Frontier models Local LLMs Builder tools Zero hype Free and curated. One tap to unsubscribe, always. Error: Domain verification failed (missing Origin header).

Hugging Face breach signals dangerous AI cyber era. According to CNBC, a Hugging Face breach exposes model supply chain risks and prompts firms to tighten keys, secrets, and MLOps security controls. Analysis. The reported incident involving Hugging Face highlights emerging cybersecurity risks tied to open source artificial intelligence model repositories and how organizations across sectors may remain unprepared for supply chain attacks targeting AI assets. * AI model repositories now represent high value targets for attackers seeking to inject malicious code that can compromise downstream applications in finance healthcare and autonomous systems. * Many enterprises lack visibility into the provenance and integrity of models downloaded from public hubs creating blind spots that amplify potential damage from compromised weights or training scripts. * Proactive measures such as model scanning runtime monitoring and signed model pipelines offer practical defenses that reduce exposure while supporting continued innovation in machine learning deployments. Deep dive into AI supply chain vulnerabilities. Artificial intelligence development increasingly depends on shared model hubs where developers upload and download pretrained networks. When such platforms experience security breaches the ripple effects extend far beyond the initial repository because thousands of companies integrate these models into production systems without additional verification steps. Technical attack vectors. Attackers can alter model files to include backdoors that activate only under specific conditions or embed data exfiltration routines that operate during inference. These techniques differ from traditional malware because they exploit the statistical nature of neural networks making detection through conventional antivirus tools ineffective. Implementation challenges arise from the sheer volume of models available and the computational cost of thorough inspection. Solutions include automated static analysis tools that inspect model architectures for anomalous layers combined with behavioral testing in isolated sandboxes before deployment. Business impact and monetization opportunities. Companies that develop robust AI security platforms can capture significant market share as demand grows for trusted model marketplaces and verification services. Subscription based scanning services and enterprise grade model registries represent recurring revenue streams while consulting firms can assist organizations in establishing secure AI development lifecycles. Competitive differentiation will favor vendors that integrate regulatory compliance features such as audit trails for model lineage helping clients meet emerging standards around AI accountability. Early movers gain advantages by building network effects through verified model ecosystems that attract both developers and enterprise buyers. Future outlook and industry shifts. The landscape will likely see increased collaboration between AI platform providers and cybersecurity firms to embed security by design principles into model distribution pipelines. Regulatory bodies may introduce mandatory disclosure requirements for model sources pushing organizations toward private verified repositories over public open access options. Ethical considerations remain central as compromised models could lead to biased or harmful outputs affecting end users. Best practices emphasize transparency in model sourcing regular third party audits and continuous monitoring to maintain trust in deployed artificial intelligence systems. Frequently asked questions. What makes AI model repositories attractive targets for cyber attacks? Repositories host widely reused components so a single compromise can affect numerous downstream applications across industries creating high impact with relatively low effort for attackers. How can businesses protect against compromised AI models? Organizations should adopt model provenance tracking automated scanning for anomalies and deployment only from verified signed sources while maintaining isolated testing environments. What regulatory trends are emerging around AI security? Authorities are exploring requirements for supply chain documentation and incident reporting specific to AI assets to ensure accountability and reduce systemic risks in critical sectors. Are there business opportunities in AI cybersecurity? Yes demand is rising for specialized tools services and platforms that provide model verification secure hosting and compliance support creating new revenue streams for technology providers. CNBC. CNBC delivers real-time financial market coverage and business news updates. The channel provides expert analysis of Wall Street trends, corporate developments, and economic indicators. It features insights from top executives and industry specialists, keeping investors and business professionals informed about money-moving events. The coverage spans global markets, personal finance, and technology sector movements.