Malmö University Publications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
The Invisible Watermark: A Pilot Study on the Verifiability Gap in AI Text Watermarking
Department of Computer Technologies, Technical Sciences Vocational School, https: //ror.org/00jb0e673Kırklareli University, Kırklareli, 39100, Türkiye.ORCID iD: 0000-0002-6559-7704
Department of Computer Technologies, Gonen Vocational School, https: //ror.org/02mtr7g38Bandırma Onyedi Eylül University, Bandırma, 10200, Türkiye; Department of Computer Technologies, Gönen Vocational School, https: //ror.org/02mtr7g38Bandırma Onyedi Eylül University, Bandırma, 10200, Türkiye.ORCID iD: 0000-0002-7530-7715
Department of Computer Technologies, Gonen Vocational School, https: //ror.org/02mtr7g38Bandırma Onyedi Eylül University, Bandırma, 10200, Türkiye; Department of Computer Technologies, Gönen Vocational School, Bandırma Onyedi Eylül University, Bandırma, 10200, Türkiye.
Department of Computer Engineering, Bitlis Eren University, Bitlis, Türkiye; Department of Computer Engineering, https: //ror.org/00mm4ys28Bitlis Eren University, Bitlis, 13100, Türkiye; Computer Engineering Department, https: //ror.org/00mm4ys28Bitlis Eren University, Bitlis, 13100, Türkiye.ORCID iD: 0000-0003-0271-216X
Show others and affiliations
2026 (English)In: Applied Sciences, E-ISSN 2076-3417, Vol. 16, no 20, article id 10335Article in journal (Refereed) Published
Abstract [en]

Article 50 of the EU AI Act requires generative AI providers to mark synthetic content in a detectable format. This obligation is carried out through a mechanism invisible to end users. The resulting gap leaves accountability around disinformation, copyright infringement, and public trust largely unaddressed. This paper reports a pilot study that probes this transparency-verifiability gap through a comparative document analysis of five major providers (Anthropic, OpenAI, Google, Meta, xAI) combined with a small-scale empirical robustness pilot of the open-source SynthID-Text method on GPT-2. The document analysis shows that regulatory disclosure does not equate to an independently verifiable detection mechanism for third parties. In this pilot experiment, watermarked texts scored significantly higher than unwatermarked texts (t(38) = 15.94, p < 0.001, Cohen's d = 5.04, ROC-AUC = 1.000; Youden's J-optimal threshold = 0.533, yielding 100% sensitivity and 100% specificity on this intact (uncorrupted) sample; the design was adequately powered (80%) only for effects of d ≥ 0.91 or larger, A calibration/test-separated validation on independent held-out data (n = 320) confirmed this discrimination within the pilot setup (TPR at FPR ≤ 1% = 98.75%), without implying generalization beyond it. This signal, measured with an untrained detector, degraded substantially under a deliberately engineered high-severity stress test (50% word-level corruption), a decline confirmed by Holm-corrected pairwise comparisons across nine of the ten corruption-level pairs (F(4, 56) = 98.39, p < 0.001, generalized η2 = 0.744, r = -0.824). These are preliminary, pilot-scale findings; they indicate that Article 50 guidance should require standardized error-rate reporting and independently auditable detection access, not disclosure alone. They offer an exploratory contribution to the AI ethics literature on transparency and accountability. Given the small open-source backbones (GPT-2, with a replication on Pythia-410M), the primarily untrained detector (a small-scale trained Bayesian detector did not outperform it on intact text), the single watermarking scheme, and the largely synthetic corruption type (only one model-based paraphrase condition was tested), these results remain preliminary, pilot-scale evidence whose generalization to commercial systems, large-scale trained detectors, other watermarking families, and a broader range of realistic paraphrase and translation attacks requires future research.

Place, publisher, year, edition, pages
Multidisciplinary Digital Publishing Institute (MDPI) , 2026. Vol. 16, no 20, article id 10335
Keywords [en]
AI text watermarking, EU AI Act Article 50, pilot study, SynthID, transparency, verifiability
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:mau:diva-88849DOI: 10.3390/app162010335ISI: 001914228000001Scopus ID: 2-s2.0-105054835022OAI: oai:DiVA.org:mau-88849DiVA, id: diva2:2115242
Available from: 2026-11-04 Created: 2026-11-04 Last updated: 2026-11-09Bibliographically approved

Open Access in DiVA

fulltext(3543 kB)57 downloads
File information
File name FULLTEXT01.pdfFile size 3543 kBChecksum SHA-512
1873349f9f8622143d559582c456f9b72aa8314ed87627637827e9f4f3154fbc25a59dbebf4dfcea9302544c6c77b49a7e372ce383ecf5bb8a97085ae7dd0fd0
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Authority records

Akin, Erdal

Search in DiVA

By author/editor
Bulut, SelmaKorkmaz, AdemTümen, VedatAkin, Erdal
By organisation
Department of Computer Science and Media Technology (DVMT)Sustainable Digitalisation Research Centre (SDRC)
In the same journal
Applied Sciences
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 57 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 252 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf