Deepfake detection accuracy falls by up to 49% in a year of AI video progress

Updated on September 17, 2026

Until recently a synthetic video of someone speaking left giveaways behind: a mouth drifting out of sync, an artifact at the seam where one face had been pasted over another. Those seams are what automated detectors were built to find, and for years they found them reliably. The video models released between late 2025 and early 2026 leave no seam at all, because they replace nothing: they generate the entire frame.

AI video detection accuracy is now close to a coin toss. On the DF26 benchmark, published on 7 September 2026, the AUROC of nine state-of-the-art detectors falls from a 75.6 to 94.3 range on the older CelebDF++ benchmark to a 44.0 to 69.7 range, one of them losing 46.1 points, and people asked to judge the clips identify the fakes 52.6% of the time. The lesson is not that better detectors are on the way. It is that authenticity has to be fixed while a video is being made, because reconstructing it afterwards no longer works. That single drop is 49% of the detector’s accuracy gone in a year.

What the DF26 benchmark measures, and why its numbers matter

The DF26 study is an evaluation benchmark published on 7 September 2026 (arXiv:2609.07369) by researchers at the Ukrainian Catholic University, the Czech Technical University in Prague, Hover Inc. and the Institute of Software Systems of the National Academy of Sciences of Ukraine. It holds 2,691 videos: 271 real and 2,420 generated by seven models released between July 2025 and March 2026, across three public-speaking scenarios (pieces to camera, official footage, studio interviews). For each real video, drawn from OpenVid-1M, TalkingCelebs or MAVOS-DD, the authors derive a semantic prompt from its own frames and generate matching synthetic clips, producing real and fake pairs that share the same visual context. No generated video carries a visible watermark, logo or export marker, and since the pairs are matched on purpose, no detector can score well here simply by noticing that the fakes were filmed differently.

Nine state-of-the-art detectors, none of them reliable

Every detector tested loses most of its advantage over chance. All nine were trained on FaceForensics++ and evaluated without retraining, the realistic condition, since no forensic team can retrain a classifier on a generator that has not shipped yet. The strongest result on the DF26 videos is 69.7 AUROC, from a detector that reaches 89.9 on the older CelebDF++ benchmark, and one detector ends below the 50 that represents pure guessing.

Bad as they are, the averages flatter the field. Broken down by generator the failures turn extreme: on Grok Imagine 1.0, Effort scores 13.6 AUROC and DFD-FCG 20.5, while on Veo 3.1 Effort scores 21.3 and PwTF-DVD 26.7. The authors say plainly that commercial generators are the hardest case and that aggregate scores conceal severe failure modes, since one system can look acceptable on a given model and sink far below chance on the next. A single published accuracy figure is therefore an average hiding the cases that matter, which is also who owns the error margin when a wrong answer lands in a claims file.

Not a September anomaly: the curve has been falling for two years

The decline did not start with this study. Deepfake-Eval-2024 (arXiv:2503.02857, presented at CVPRW 2026) collected content that actually circulated online during 2024: 45 hours of video, 56.5 hours of audio and 1,975 images, from 88 sites in 52 languages. Against earlier academic benchmarks, the AUC of open detectors fell by 50% on video, 48% on audio and 45% on images. Two independent teams point the same way: detectors lose half their performance on real-world material and the rest when it comes from a generator they have never seen. Work on model-specific artifacts, including the Edinburgh study on AI fingerprints, arrives at the same limit by another route.

Why they are losing ground: the problem changed shape

The detectors are not underperforming so much as answering a question nobody asks any more. All nine were trained on FaceForensics++ (arXiv:1901.08971), built from manipulations of real faces: face swapping, facial reenactment, attribute editing. What such a model learns to find is the discontinuity between an authentic region and an altered one, and the videos in the DF26 set contain no manipulation at all, so that discontinuity is not there to find. The authors call this a distribution shift across generative models, and their Table 3 rules out the obvious alternative, since the collapse tracks the generator rather than the filming scenario.

Retraining works, and the way it works is the real problem. Fine-tuning GenD-PE on the HunyuanVideo image-to-video subset lifts its macro average from 76.0 to 82.3, with Grok Imagine going from 66.5 to 96.8 and Veo 3.1 from 76.6 to 95.6. The detector learns one generator at a time, and only once that generator has shipped and been sampled, so between a release and a retrained classifier sits a window in which the evidence is undecidable, reopened by every new model. A more accurate detector does not close that gap, which is why detection does not hold as a strategy and why data authenticity platforms such as TrueScreen move the certainty upstream instead.

The human eye is not the backup

People do no better than the software, and their failure cuts both ways. The DF26 human study collected 232 labeling sessions, mostly from students at technical universities, each participant judging 30 videos, 15 of them fake, in five-second clips replayable up to ten times, with three options: AI generated, AI manipulated, real.

On genuine footage people perform much the same everywhere, while on the DF26 fakes they find nothing to go on, and the generator fooling them most often is LTX 2.3 distilled I2V. Nor is this local: a meta-analysis of 56 papers covering 86,155 participants and 137 effect sizes puts average human accuracy at 55.54%, with a 95% confidence interval from 48.87% to 62.10% (doi:10.1016/j.chbr.2024.100538). Because that interval crosses 50, human performance cannot be distinguished from chance, and training with feedback, automated support and caricaturing raises it only to 65.14%.

The second edge is the one organizations underestimate: 24.0% of the real videos in the DF26 set were called fake, so the exposure is not only that a fabricated clip gets believed, but that a genuine one stops being believed. A video of an event that happened, filed in a dispute that is real, now invites a challenge costing nothing to raise.

If doubt cannot be settled afterwards, make it unnecessary beforehand

Platforms such as TrueScreen, the Data Authenticity Platform, work the problem from the opposite end: instead of analyzing a video after it has circulated, they fix its authenticity at the moment it is captured. That reverses the question. Once a file arrives carrying the record of how it came into existence, whether a classifier can separate a generated video from a filmed one stops being the decisive test.

What certifying a video at source actually means

Authenticity established at source means the origin of a file, its integrity and the moment it was made are all fixed while the content is being produced, rather than inferred from the pixels afterwards. Organizations that need a video to hold up in an insurance or judicial setting use TrueScreen to capture it in a forensic environment, with integrity verification, an RFC 3161 timestamp and the acquisition coordinates, following the ISO 27037 approach to digital evidence, and certification is completed through the seal of qualified third-party providers integrated via API. The contrast is arithmetical: the best of the nine detectors in the DF26 study reaches 69.7 AUROC and people identify fakes 52.6% of the time, while a file captured under a documented procedure asks no one to trust either number.

Declared markings help where they exist. TrueScreen reads C2PA Content Credentials, the IPTC digitalSourceType property and generator tags written into EXIF and XMP, and treats a file without Content Credentials as unmarked rather than suspicious, since most cameras and apps still write nothing at all. Digital provenance is a chain that holds only if the first link is placed deliberately. Synthetic content analysis covers photo, video and audio as part of capture and returns a confidence level rather than a verdict, which is why it supports a decision instead of making one.

Take a loss adjuster documenting hail damage to a warehouse roof. When the video is certified as it is recorded, with its integrity, time of capture and coordinates fixed through forensic acquisition from the app, the other side cannot open a debate about whether it was generated, because the file is not an exhibit to be analyzed but a record to be verified. The burden shifts. The insurer no longer has to prove the video is real, because the video brings that proof with it, which is what the right to authenticity as positive proof of the true amounts to in a claims file.

The authors are careful about the reach of their own result. The benchmark is small at 2,691 videos and visual only, leaving audio, speech quality, lip sync and audio-video consistency unevaluated, and it measures no commercial detection tool, says nothing about still images, and does not claim detection is worthless in principle. Its finding is narrower and harder to argue with: detectors trained on FaceForensics++ and placed in front of generators they have never seen do not hold.