← Fix Subtitles

Research · Technical note v1.0 · Published 2026-10-27

Subtitle Repair Benchmark: 11,597 injected defects, scored

A reproducible benchmark of the subtitle checker and fixer behind VideoText's Fix Subtitles tool. It measures whether the fixer finds subtitle errors, whether the file it returns meets the rules, and whether it tells you the truth about what it could not fix. The corpus, code and every per-file result are public.

DOI: 10.5281/zenodo.23701131Read the paper (PDF)Zenodo recordHow to cite

Direct answer

How well does the VideoText subtitle fixer work?

Evaluated on 2,000 synthetic subtitle files containing 11,597 injected defects. The VideoText subtitle engine detected all injected defects and produced compliant output for 99.2% of files containing repairable defects.

Where a defect had no safe fix in place, the fixer left it and reported it rather than hiding it. It never changed the wording of a cue. These are results on synthetic English files checked against explicit rules, published by VideoText; they are not an independent peer review.

  • 2,000 SRT files, 118,803 cues, built from public-domain books
  • Rules: 42 characters per line, 2 lines, 20 characters per second, 1.5 s minimum for cues over 20 characters, no overlaps
  • Compared with two simple baseline fixers on the same files

Key results

injected defects detected
11,597 / 11,597
false positives on unmodified cues
0 / 107,206
of files with repairable defects made compliant
99.2%
claimed fixes confirmed in the exported file
10,322 / 10,322

Median processing time was 1.13 ms per 60-cue file. All 2,412 violations the fixer could not repair were reported on the cue that has them, and no file it marked as passed had a violation.

Files compliant after repair

200 files per condition. Baseline A clamps overlaps and wraps lines. Baseline B also extends cues that are too fast where the next cue allows it. Rows marked * are stress conditions with no repair in place.

ConditionVideoText engineBaseline ABaseline B
Overlap200/200200/200200/200
Long line200/200200/200200/200
3+ lines191/200200/200200/200
Fast reading200/2000/200200/200
Invalid timing200/2000/200200/200
Mixed199/2000/200200/200
Overflow text*200/2000/2000/200
Dense overlap*0/2000/2000/200
Dense fast*0/2000/2000/200
  • On defects with room to repair, Baseline B matched or slightly beat the engine (1,200 vs 1,190 files). The engine keeps an author's line breaks when it splits a three-line cue, which occasionally leaves a piece under 1.5 s. It reports each one.
  • When text needs more than two lines, only the engine produced compliant files: it split the cue. Both baselines left three or more lines.
  • In the dense conditions no fixer succeeded. The baselines turned 1,200 overlaps into cues too fast to read, without saying so. The engine left them and reported every one.

Does the fixer report the truth?

Fixes marked "fixed" and confirmed in the export10,322 / 10,322
Open findings confirmed by the oracle2,412 / 2,413
Violations left in the file without a finding or warning0 / 2,412
Files marked "passed" that are not compliant0 / 1,565
Files with the same words, in the same order2,000 / 2,000
Cues away from any defect whose timing moved0 / 85,289
Files unchanged when fixed a second time2,000 / 2,000

Methodology

  • Corpus: 2,000 SRT files of 60 cues each, cut from five public-domain Project Gutenberg books by a seeded generator. Every clean file meets all the rules.
  • Defects: 11,597 defects injected into chosen cues, never next to each other, so each finding has one cause.
  • Oracle: an independent rule checker written for the study that does not call VideoText code. It checks 42 characters per line, 2 lines, 20 characters per second, a 1.5 s minimum for cues over 20 characters, no overlaps, and end after start.
  • System tested: the VideoText subtitle engine with Fix timing and Line breaks on, and grammar, filler removal and re-sync off.
  • Reproducible: two runs give byte-identical files and identical results.
ConditionInjected defectDefects
OverlapA cue ends 0.3–1.5 s after the next one starts1,200
Long lineA two-line cue joined into one line over 48 characters1,200
3+ linesA cue broken into three lines1,200
Fast readingDuration cut to 25–40 characters per second1,200
Invalid timingEnd time at or before the start time1,200
MixedTwo of each defect above in one file2,000
Overflow text*Two cues merged into one line over 88 characters1,197
Dense overlap*Next cue starts 0.3–0.8 s after a cue over 30 characters1,200
Dense fast*A cue at 25–40 CPS with the next cue 0.04 s after it1,200

Limitations

  • Published by VideoText, not peer reviewed. The author develops VideoText. The data and code are released so anyone can check the numbers.
  • Synthetic files. Real files have correlated errors, markup, speakers, VTT positioning and encoding problems that were not tested.
  • Same rules. The oracle is separate code but uses the same thresholds as the engine, so it tests the implementation, not whether those rules suit every audience.
  • No audio. Timing was not checked against speech, and the corpus is English only.
  • Simple baselines. The two baselines were written for the study. They are not any other company's product, and no claim is made about other tools.
  • Two checkers. These results are for the engine behind Fix Subtitles, its API and the fix_subtitles tool. The free check_subtitles tool for AI assistants is a separate checker. Before a fix made for this study it detected 9,997 of 11,597 defects (86.2%); with the fix, all of them (paper, section 5.5).

Research files

Licensed CC BY 4.0. To reproduce the run, follow the README in benchmark-code-and-results.zip.

How to cite

Guntupalli, S. (2026). Honest Automated Subtitle Repair: A Reproducible Defect-Injection Benchmark of the VideoText.io Subtitle Quality Engine (Version 1.0) [Technical note]. Zenodo. https://doi.org/10.5281/zenodo.23701131

@techreport{guntupalli2026subtitlerepair,
  author      = {Guntupalli, Santhosh},
  title       = {Honest Automated Subtitle Repair: A Reproducible Defect-Injection Benchmark of the VideoText.io Subtitle Quality Engine},
  institution = {VideoText},
  type        = {Technical note},
  year        = {2026},
  version     = {1.0},
  publisher   = {Zenodo},
  doi         = {10.5281/zenodo.23701131},
  url         = {https://doi.org/10.5281/zenodo.23701131}
}

DOI 10.5281/zenodo.23701131 · Title on Zenodo: Honest Automated Subtitle Repair: A Reproducible Defect-Injection Benchmark of the VideoText.io Subtitle Quality Engine

Frequently asked questions

How accurate is the VideoText subtitle fixer?

In a reproducible benchmark of 2,000 synthetic SRT files with 11,597 injected defects, the engine detected every injected defect, flagged none of 107,206 unmodified cues, and produced compliant output for 99.2% of files with repairable defects. Results are for synthetic English files checked against explicit rules, not for every real-world file.

Is this benchmark peer reviewed?

No. It is a technical note published by VideoText on Zenodo with a DOI. The author develops VideoText. To make it checkable, the corpus, ground truth, scoring code and per-file results are all published, including the conditions where VideoText does not win.

What does the fixer not repair?

When the next cue starts too soon for a fix in place, the fixer leaves the cue as it is and reports it instead of creating a cue too short to read. In the two dense stress conditions, no fixer tested produced a compliant file. The fixer also does not judge wording or translation.

Can I test another subtitle tool with this benchmark?

Yes. Download corpus.zip from Zenodo, run your tool on the 2,000 SRT files, and score the output with the oracle in benchmark-code-and-results.zip. The ground truth file lists which cue received which defect.

Do these results apply to the free check_subtitles tool for AI assistants?

Not directly. That tool uses a separate, lighter checker. Before a fix made for this study it missed every three-line cue (86.2% of defects detected). With the fix it detects all 11,597 defects; see section 5.5 of the paper.