In today’s Artificial Intelligence (AI) era, automation has become integral to every process within the Information and Technology industry. Digital Accessibility is also on the rise, with many people discussing how automation and AI can be used in various processes like design, development, testing, creating audit reports and producing ACRs or VPATs for their digital products.
However, while automated accessibility testing tools are helpful, they have their limitations. These tools typically detect only 25 to 40% of accessibility problems on a website, though various tools might report different percentages. The key fact remains, Accessibility is for People. Many issues experienced by people with disabilities, especially when using assistive technologies like screen readers, cannot be fully detected by automated tools.
I'm not here to debate how much coverage automated accessibility testing tools have or compare which tool finds more issues than another. I’m here to talk about the top 5 accessibility issues that can be identified by a native screen reader user which automated tools can’t identify.
-
Inaccurate Alt Text for Images
Alt text provides descriptions for images so that people who cannot see them can understand their meaning. The web is filled with images, and it won’t be wrong to say one can barely find a page that doesn’t have an image. Imagine if alt texts are not specified for images, then how would a blind person access the information conveyed through images?
Does every image need alt text description? Not really, because each image does not convey information. That’s where human judgement comes into play to decide whether the image is informative or decorative.
Automated tools can check whether alt text exists, but they cannot judge if the description is accurate, useful, or contextually relevant. A native screen reader user can easily identify when the alt text is insufficient, confusing, or overly verbose. They can suggest concise, accurate alternatives that improve the overall experience.
Alt Text Detection and Evaluation Aspect Automated Tool Capability Screen Reader User Capability Detection Can Identify Missing Alt Text Can Identify Missing Alt Text Accuracy Assessment Cannot Judge if Description Makes Sense Can Evaluate Contextual Accuracy Conciseness Evaluation Cannot Determine Optimal Length Can Suggest Concised Descriptions Decorative vs Informative Cannot make Judgment Calls Can Distinguish Based on Context -
Illogical Reading Order
Screen readers follow the DOM order (left-to-right, top-to-bottom). Basically, it will read in a sequence in which the content is included in the code. Sounds pretty straight forward, doesn’t it? But in reality, the reading order often doesn’t make sense when screen reader users access the information which might visually read perfectly fine.
- In a column layout, a screen reader reads the first line of column 1, then column 2, leading to confusion.
- On product catalog pages, products are placed horizontally on a retail site where product image descriptions are read first and then product names are read out. This makes it confusing and difficult for the user to understand the context.
Only a native screen reader user can detect such issues and suggest reordering for meaningful flow. Automated tools can’t decide on their own if the reading order makes sense. Human judgement is required to decide if the information is read out in a meaningful sequence.
-
Invisible Content read by Screen Reader Users
Content is not visible but is still read out by screen readers. This is an issue very commonly found on rich and dynamic sites. Web pages have various widgets, like menus, accordions, carousels, tabs and the list can go on. A question that comes to mind is how does having widgets lead to invisible content being read out?
These widgets reveal additional content on user request. But this is not true in the case of screen reader users. Even if users haven’t activated the widget, all the content is read out for them. For example, in a carousel, the content of all slides is read out instead of only the active slide. Yes, this does happen and it happens too often. For a screen reader user, content of each slide should be read out only when the user has requested it, i.e. on activating Next, Previous or slide picker controls.
Automated tools cannot catch this. A native screen reader user can detect when hidden content is read unnecessarily and flag it for correction.
The fact is that the screen reader experiences such behavior with menus, tabs, accordions and many such widgets. All the content is read out even if the user hasn’t activated a menu, expanded an accordion or selected a tab. Again, visually everything is working as expected but what a screen reader user experiences is totally different.
This happens due to content not hidden correctly but can automated tool catch that? No, No, No only a native screen reader user will help you catch if invisible content is read out for them.
-
Form Instructions not Associated with Form Fields
Instructions are tiny little helpers added to assist users fill up the form successfully. Date format, password requirements, file type and size allowed, mandatory field markers etc. are examples of form instructions found on the web. These instructions are provided to help users, but do they really help screen reader users? Well not always.
Instructions are placed beside the form field to make sure users can read them while filling up the details. However, these instructions are not always associated with the form field. Result? Screen reader users won’t be able to access them when they navigate between fields using the Tab key.
Form Instruction Accessibility Comparison Instruction Category Sighted User Experience Screen Reader User Experience Required Field Indicators Asterisk (*) clearly visible before/after input field No announcement when field receives focus Input Format Guidelines Format instructions (e.g., "MM/DD/YYYY") displayed below input field No announcement when field receives focus Validation Requirements Password rules, character limits shown as nearby text Validation criteria not connected to field Contextual Help Information Help text positioned adjacent to field labels Help content missed during keyboard navigation Screen reader users generally navigate between form fields using the Tab key and as they navigate, form field label (if provided) is read out for them. Yes, only the associated label is read out for them. What about instructions? They aren’t read out because instructions are not generally associated with the field.
In another example, an asterisk (*) is placed before or after the input field but it isn’t associated with the field, resulting in users not knowing that it is a mandatory field. Small things, right? But it has a big impact!
As a user if I don’t know whether a form field is mandatory or not, I might not bother filling it. This leads to form submission error and I have to go back and rectify it. This is very frustrating to say the least.
Automated tools can check if label is associated with the field but can’t decide on its own if instruction exists and whether it is associated or not. Thanks to native screen reader user testing, such issues can be easily identified!
-
Dynamic Alerts Not Announced for Screen Reader Users
Rich and dynamic interactions are built to enhance the user experience. Information is updated dynamically as the user interacts with the page but often screen reader users are left out. Yes, the information gets updated dynamically but is that announced by the screen reader?
Screen readers are often not informed through code about dynamically updating information. Result? Users are unaware of what is happening. They either believe nothing happened or something is not working for them.
For example, in an online banking application, when a screen reader user fills up a customized account statement request form, feedback gets dynamically added on the page. However, the screen reader user’s focus remains on the Submit button and nothing is announced either. But when the user reads through the entire page, they can find the updated information within a section of the page. For sighted users this is quite intuitive but for screen reader users it is a pretty helpless situation.
Automated tools won’t be able to catch this because the updated information gets added dynamically. One will have to run the test again after the dynamic update gets added but even then, the tool can’t say if screen reader will read it or not. A native screen reader user can easily catch this issue to help address the accessibility barrier and contribute towards building an inclusive experience for all users.
Comparison: Automated vs Manual Testing
| Testing Aspect | Automated Tools | Native Screen Reader Users |
|---|---|---|
| Code-level Issues | Excellent Detection | Knowledge Based Technical Analysis |
| Workflow Validations | Cannot Test Actual User Workflows | Validates Complete User Workflows |
| Context Understanding | Rule-Based only | Semantic Comprehension |
| Accessibility Implementation | Checks Code-Level Implementation | Validates if it Works for Real Users |
| Testing Speed | Rapid Automated Scanning | Time-Intensive Process |
| Consistency in Reports | Consistent Reproducible Results | Consistency in Reports may vary between Users |
Conclusion
Native screen reader user testing is essential to catch many critical accessibility issues that automated tools cannot detect. The five issues mentioned above are just a handful of examples. My intent is not to write off automated testing tools, but they are helpful for code-level issues. Whereas manual testing by people with disabilities is irreplaceable when it comes to real-world experience.
Native screen reader user testing is just one part of user testing. We should not restrict our accessibility testing to just screen reader users but include different user groups using various assistive technologies.
Accessibility is about making solutions accessible for people with disabilities and they shouldn’t be left out of the testing process. Access barriers that native users help identify can’t be captured using automation and if we want to make our digital solutions accessible then nothing can be as effective as native user testing. Remember the slogan “Nothing About Us without Us”.
