Passed Your Accessibility Check? Don’t Be So Sure.
Why does my PDF file pass Adobe Acrobat’s Accessibility Checker but fails using CommonLook’s PDF Validator or PAC3?
We get asked this question a lot: Why does CommonLook (or PAC3) tell me I’m failing when the PDF passes in Acrobat? To make a long story a little bit shorter (and you can read on for more details), it’s because Acrobat doesn’t actually test against WCAG 2.0, PDF/UA, or any accessibility standard. Sure, it checks “stuff” but not to the level required for standards conformance. In fact, in older versions of Acrobat, when you’d choose to run a “508 check,” after it was done testing, a disclaimer would come up telling you that Acrobat wasn’t actually certifying that the document passes at that level. In newer versions they’ve removed the disclaimer, but they’ve also removed the option to run a “508 check.” Now it’s just a “full check.”
So, what’s the difference? While Acrobat doesn’t test for conformance against any established accessibility standards, PAC3, the CommonLook Validator, and CommonLook PDF do. Specifically, PAC3 tests against PDF/UA and the CommonLook tools will test against not only PDF/UA but also WCAG 2.0 (A and AA), the HHS 2018 standard, the “original” Section 508, and the “original” HHS standard as well (both Section 508 and the HHS standard have been updated within the past couple of years).
Of course, no matter what tool you’re using to test a PDF, there are still manual checks that need to be done. For example, a machine can’t automatically determine that the reading order is correct in a PDF. That requires someone actually checking the order of the tags in comparison to the physical view of the document. Similarly, verifying that the correct tags are being used for the content, as well as testing for color use and contrast, are manual checks as well. So where do some of the real failures come into play? Here are some examples…
- Acrobat (and all checkers) will fail a Figure tag that’s lacking Alternative text but if a Figure tag does have Alt text it’ll automatically pass the Acrobat check. In CommonLook, however, this becomes a “user verification” item because CommonLook wants you to make sure that the alternative text provided for the image is accurate for that image.
- Acrobat’s implementation of mutually exclusive radio buttons is inherently inaccessible because, for mutual exclusivity to be achieved, the radio buttons need to be in the same group and, therefore, they all have the same name and the same tooltip. While some screen readers will compensate for this issue, it goes against accessibility standards.
- Speaking of tooltips, similar to Alternative text for Figures, Acrobat doesn’t require the tester to verify tooltip accuracy. So, again, there could be complete gibberish in the tooltip for a form annotation and it’ll pass in Acrobat.
- Acrobat doesn’t require Alternative text for Links. Without getting too technical, WCAG 2.0, Guideline 1.1, requires Alternative text for “non-text content.” Link annotations are actually not text, so Link tags need to have Alternative text (and/or the link annotation itself needs to have the Contents attribute).
- Acrobat won’t ask you to verify the ListNumbering Attribute (which is important for ordered lists).
There are other things, too, that Acrobat won’t test against such as validating the Tables of Contents have been tagged and structured correctly, along with not testing against accessibility standards, Acrobat also doesn’t test against ISO 32000-1 (or ISO 32000-2) – the standard on “how to make a PDF,” and it also doesn’t prompt the tester to check the accuracy of the metadata including the title, language and any language changes that may be in the PDF.
In conclusion, while we know that many organizations require their PDFs pass the Acrobat Full Check before they are posted online or otherwise made available to people, it’s important that remediators and testers know that the Acrobat check won’t guarantee standards conformance. That being the case, even if your PDF passes the Acrobat check, you could still be at risk for providing inaccessible content to your end users. Testers beware!