Accessibility audits are a great way to identify the accessibility conformance of a digital product. Based on the comprehensive audit that is performed by our accessibility experts, BarrierBreak provides an Accessibility Score.
Let’s learn more about the BarrierBreak scoring method.
BarrierBreak (BB) provides two types of scores:
- BB Defect Score
- BB Conformance Score
Note:- Currently there is no standard scoring method defined for WCAG 2.0 or WCAG 2.1 or WCAG 2.2. However, WCAG 3 is working on a measured approach that could be implemented in the future.
What is the BB Defect Score?
The Defect Score helps you to understand how accessible your product is based on the severity level of the accessibility issues found during the audit.
How is the BB Defect Score calculated?
An issue is assigned a severity level based on the impact it has on people with disabilities. There are four severity levels.
- Blocker
- Critical
- Major
- Minor
We start with calculating the issue count in each severity level.
Following which, the BB Defect score for each severity level is calculated using the formula below:
The next step is to calculate the total BB Defect Score. By adding the BB Defect score for each severity level, we get the total BB Defect Score of your product.
For example, if the BB Defect Score found for each severity is 22%, 37.5%, 4%, 5% respectively; the total Defect Score will be 68.5% i.e. the product is at a Very High Risk.
How does BB Defect Score help?
The defect score helps to prioritize the order of issues to be fixed based on the severity levels. Higher the BB Defect Score means the product is less likely to be accessible. In other words, the accessibility of the product and BB Defect Score are inversely proportional. The goal is to reduce the BB Defect Score and bring it to ZERO or closest to zero.
| Risk | Range |
|---|---|
| Low | 0 to 15% |
| Medium | 15 to 40% |
| High | 40 to 60% |
| Very High | 60% and above |
Simple as that! The lower the BB Defect Score, the more accessible your product!
What is the BB Conformance Score?
The Conformance Score helps determine how conformant the product is with WCAG. The BB Conformance Score and the accessibility of the product are directly proportional. A higher conformance score means, the product is conforming more to the WCAG standards.
How is the BB Conformance score calculated?
Each Success Criteria is assigned one of the following three criticality levels.
- High
- Medium
- Low
Under each Success Criteria, the issues relating to it are assigned a severity level with a weightage, similar to the BB Defect Score.
The conformance score is calculated based on the combination of the severity of the issues and the criticality of the corresponding Success Criteria.
- If the Success Criteria Passes – a predefined score is assigned as per the criticality of the success criteria.
Note: If any Success Criteria is Not Applicable, it is be considered Passed/“Supports” based on the approach that is usually taken in VPATs.
- If the Success Criteria Fails – a score is calculated based on the assigned weightage for the severity level of the issues and the corresponding criticality level of Success Criteria.
For example, if the product is failing WCAG SC 2.1.1 due to an issue with Blocker Severity; the Conformance Score for this Success Criteria will be 0.
The maximum score that can be obtained is the score if every Success Criteria is passed.
Once the score for each Success Criteria is calculated, the BB Conformance Score is derived using the below formula:
For example, if the total score obtained by adding the score for each Success Criteria is 73, and the maximum score that can be obtained is 100, the conformance score will be 73%.
How does BB Conformance Score help?
The conformance score helps to understand where the product stands from an Accessibility perspective – the higher the % means the product is more likely to be accessible.
| Rating | Range |
|---|---|
| Poor | 0 to 20% |
| Average | 20 to 70% |
| Good | 70 to 90% |
| Excellent | 90% and above |
Determining the appropriate weightage for each Success Criteria and severity is a subjective process. These weightages are based on a well-informed understanding of accessibility standards by our accessibility experts. Additionally, achieving a high Conformance Score doesn’t necessarily mean a product is fully accessible to all users. The conformance score could be high, but the product might still have issues that are Blockers or Critical in nature.
Note: Testing with individuals with disabilities remains crucial in assessing the actual effectiveness of accessibility measures on a website.
If you have any further questions or would like more information on the accessibility scoring reach out to us at info@barrierbreak.com. Also, if you want to know how our accessibility experts test your digital products and make them accessible, get in touch with our team at sales@barrierbreak.com.