Redaction ToolsPDF benchmark v0.1.1 / Blinded
Blinded on extraction-conditions-1
Web app · published by Jeff Nixon (Tool owner)
Leak rate
7.4%
95% interval 2.9–17.6% · n = 54
Scored by usLeaked 4 of 54 sensitive values (7.4%).
- Over-redaction
- —
- Text kept (worst case)
- 8.7%
- Cases covered
- 1 of 1 cases
- Document damaged
- 1 cases
Is the document still usable?
- text_retained failed in 1 of 1
Where the leaks are
| Layer | Leaked here | Only here | If it survives |
|---|---|---|---|
| ocr high | 5.6%1.9–15.1% 3 of 54 | 0.0%0.0–6.6% 0 of 54 | catches translucent or hairline covers |
| metadata high | 0.0%0.0–6.6% 0 of 54 | 0.0%0.0–6.6% 0 of 54 | invisible, survives forwarding |
| annotation high | 0.0%0.0–6.6% 0 of 54 | 0.0%0.0–6.6% 0 of 54 | shown by many viewers |
| attachment high | 0.0%0.0–6.6% 0 of 54 | 0.0%0.0–6.6% 0 of 54 | one toggle away |
| font_subset low | 0.0%0.0–6.6% 0 of 54 | 0.0%0.0–6.6% 0 of 54 | narrows the value; partial disclosure |
| image_xobject critical | 0.0%0.0–6.6% 0 of 54 | 0.0%0.0–6.6% 0 of 54 | a crop is a view, not a deletion |
| content_stream critical | 1.9%0.3–9.8% 1 of 54 | 0.0%0.0–6.6% 0 of 54 | copy-paste, pdftotext, any parser |
| prior_revision critical | 0.0%0.0–6.6% 0 of 54 | 0.0%0.0–6.6% 0 of 54 | 'save' instead of export; trivially recovered |
| rendered_pixels critical | 7.4%2.9–17.6% 4 of 54 | 1.9%0.3–9.8% 1 of 54 | visible to anyone who opens the file |
| optional_content high | 0.0%0.0–6.6% 0 of 54 | 0.0%0.0–6.6% 0 of 54 | one toggle away |
By category
| category | Leak rate | Over-redaction |
|---|---|---|
| PERSON | 7.4%2.9–17.6% 4 of 54 | — not measured |
By severity
| severity | Leak rate | Over-redaction |
|---|---|---|
| high | 7.4%2.9–17.6% 4 of 54 | — not measured |
By trap
| trap | Leak rate | Over-redaction |
|---|---|---|
| nan | 7.4%2.9–17.6% 4 of 54 | — not measured |
Removed, by where the value lives
- image_text
- 85.2%67.5–94.1%
23 of 27
- text_layer
- 100.0%87.5–100.0%
27 of 27
What was not measured
Every layer was read in every run.
Provenance
- Scored with
- pdfredeval 0.1.1
- Engines
- ocr tesseract · pypdf 6.19.0 · pdfium 153.0.7999.0@sourcebuild-native · renderer pypdfium2 · pypdfium2 5.13.0 · tesseract tesseract 5.5.2
- Alignment
- fiducial, residual 0.18 px
- Run id
- 20260930T155353-blinded-web-extraction-conditions-1-a1-821c00
- the run declares no capabilities, so every probe was scored in scope; a tool is not credited here for anything it never claimed to do