Medium-level review of a VPAT

• Overview
• Compare a VPAT and an ACR
• Overview of an Accessibility Compliance Report (ACR)

Overview

This document will help you perform a rapid review of vendor provided accessibility documentation, to see if a deeper review is necessary or desired. This is not a thorough review. It is intended to:

• Establish the reliability of the vendor's accessibility documentation
• Give a general sense of the overall accessibility of the product

Estimated time to perform task: Under 10 minutes.

Compare a VPAT and an ACR

Before starting, you should understand the difference between a VPAT (Voluntary Product Accessibility Template) and the accompanying ACR (Accessibility Compliance Report).

VPAT

The VPAT is the entire template with all the pages of instructions and resources to help gain a comprehensive understanding of the document.

ACR

The ACR is the report portion of the VPAT after the instructions and resources are removed. The ACR contains only the introduction form and the details of the criteria tables.

Overview of an Accessibility Compliance Report (ACR)

The vendor may send an entire VPAT or just the ACR portion of it. The ACR contains the important parts to review. At minimum, you need to examine the introduction form and the criteria tables.

Introduction form

The introduction form contains general information about the product and the evaluation. Verify the product and version. Four specific fields in this introduction help establish the reliability of the form as a whole.

1. Evaluation methods used: The evaluation methods are perhaps the most critical aspect to the introduction. Look for:
   • Assistive technology used during the testing process
   • Multiple methods, including both manual and automated testing
   • Third party evaluators that can be vetted to certify their value
   • Specific methodology references

   Answers that should concern you:

   • Similar to another evaluated product
   • Use of a vendor-proprietary testing method, unless the method can be vetted appropriately
   • Automated testing with no manual testing
   • General product knowledge
   • Other similar responses
2. Applicable standards and guidelines: What standards were used to create this report? This field requires at least one standard. However, it is better if multiple standards were used and listed. Look for:
   • WCAG 2.0 or 2.1, A and AA conformance

   Additional standards that show more detailed knowledge and commitment by the vendor:

   • Revised Section 508
   • EN 301 549
   Note:

   WCAG 2.0 is most likely to be displayed, since WCAG 2.1 was just released in December 2018. The level of conformance is also important, which exists as Level A, AA, and AAA. This document focuses on level AA, which is a common standard and should be the minimum goal.
3. Contact information: The name of the tester, their position within the organization, and specific contact information should be provided.

   The tester's position is usually more reliable if they are:

   • On the vendor's accessibility team
   • From a third-party accessibility firm

   Be more cautious if the tester's position is from the vendor's:

   • Marketing team
   • Sales team

   Marketing and sales people may have little knowledge of the product and thus any details of any testing that may have been done. They are also likely to have a biased purpose.

4. Date: This field should contain the date the VPAT was issued. Look for a date within the last six to nine months.

   Be cautious of anything over one year. You may need to request an updated report.

   Note:

   This is highly dependent on the type of application it is and how frequently the vendor releases regular updates. Therefore, this is a general guideline, not a gold standard.

Criteria tables

The tables may represent information about three standards:

• WCAG
• Revised Section 508
• EN 301 549

For the purposes of this document, we are only looking at the WCAG standards and accompanying tables. The WCAG standard contains three conformance levels, A, AA, and AAA, which each have an individual table. There may be references to the other standards within these tables as there are matching criteria from all three standards.

The tables include three columns:

• Criteria
• Conformance Level
• Remarks and Explanations

Criteria will list the specific standard targeted by the evaluation.

Conformance Level gives you the opportunity to gauge the possible accuracy of the submission. The conformance level column can have one of four different values:

• Supports
• Partially Supports
• Does Not Support
• Not Applicable (N/A)

Remarks and Explanations should ideally be as detailed as possible about the specific problems encountered. Look for:

• Specific examples of how the product supports, partially supports, or doesn't support the criterion
• Remarks and explanations for all criteria

Responses that should concern you:

• Explanations that duplicate the wording of the criteria
• All criteria marked as "Supports"
• Language like "this product was designed to...", which doesn't indicate if it actually does

The critical guidelines

Here are the most critical WCAG guidelines to concentrate on in your initial review.

• 1.1 - Text Alternatives
  • 1.1.1 - Non-Text Content (Level A)

    Summary: This is mostly about alternative text for images, charts, tables, and so on. Most products should support this guideline. The use of CAPTCHA elevates this guideline to critical. CAPTCHAs are an element of controversy in the accessibility community because of their potential barrier to access. If CAPTCHAs are included within a vendor's submission, there must be more than one option (for example, visual and auditory) as a minimum.

    Priority: Critical

• 1.2 - Time Based Media
  • 1.2.1 - Audio Only & Video Only (Pre-recorded Level A)

    Summary: There must be text alternatives to pre-recorded audio or video content, usually in the form of text-based transcripts. This is critical for accessibility as these text transcripts can then easily be translated into different modes for different user needs.

    Priority: Critical

  • 1.2.2 - Captions (Pre-recorded Level A)

    Summary: There must be captions provided for any pre-recorded audio/video content. This is critical for a hearing-impaired user. "Does not support" or "Partially supports" indicate the product should be flagged for further testing. View a 1-minute video overview about captions.

    Priority: Critical

  • 1.2.3 - Audio Description or Media Alternative (Pre-recorded level A)

    Summary: Video content is either given additional audio descriptions within the confines of the audio track or there is an entirely separate audio track that contains the entirety of the video content described in audio format. Exceptions here could be expected as full conformance seems a rarity in this area. However, it should still be viewed as a critical component.

    Priority: Critical

• 1.3 - Adaptable
  • 1.3.1, 1.3.2, 1.3.3 - Info and Relationships, Meaningful Sequence, Sensory Characteristics (Level A)

    Summary: These three guidelines are related to meaning and how the structures of applications can be read, and thus interpreted, with assistive technology. For this reason, they should be viewed more as a whole than as individual sub-sections. This area is as critical to accessibility as any on this list. However, because these are design guidelines, it is also the one with the largest room for interpretation. The good news is most modern web-based applications generally conform to these concepts during their initial design phase. Answers here should generally list "Supports". Possible exceptions should be very detailed if they exist.

    Priority: Critical

• 1.4 - Distinguishable

  1.4.1, 1.4.2 - Use of Color, Audio Control (Level A)

  Summary: These deal with the use of color and the ability to control audio playback within an application.

  • If color is used in a defining way (for example, click the red button), there must be alternative ways to identify that element or color identification.
  • If audio content is to be automatically played, there must be a way for the content to be stopped or paused, and the volume must be operated independently from the system volume.

  These guidelines are critical as both have the potential for significant barriers to access.

  Priority: Critical

• 1.4.13 - Content on Hover or Focus (Level AA WCAG 2.1 specific)

  Summary: This guideline deals with hovering over or focusing on an item to activate extra content. Hover action elements are notorious for accessibility problems.

  Priority: Critical

• 2.1 - Keyboard Accessible
  • 2.1.1, 2.1.2 - Keyboard, No Keyboard Trap (Level A)

    Summary: These guidelines should be considered the starting point in the evaluation process and the most critical within the standard.

    • Guideline 2.1.1 deals solely with the ability to navigate the application with keyboard controls, which is crucial to the use of assistive technologies.
    • Guideline 2.1.2 deals with the process of navigating to a specific element and then becoming "trapped" on the element and not being able to navigate away from it.

    This doesn't preclude an application that has "partially supports" from being considered. However, they should be flagged and then sent for more in-depth testing before purchase. View a 1-minute video on testing keyboard support.

    Priority: Critical

  • 2.1.3 - Keyboard (No Exceptions) (Level AAA)

    Summary: The guideline requires that an entire application be keyboard navigable. Even though AAA conformance guidelines are not included in the rest of this document, it is included here to highlight the importance of keyboard navigation. If this guideline is supported within a submission, it should be given top priority.

    Priority: n/a

• 2.3 - Seizures and Physical Interactions
  • 2.3.1 - Three Flashes or Below Threshold (Level A)

    Summary: This guideline restricts the flashing of content to no more than three flashes per second within an application. Considering its impact, its critical importance is clear.

    Priority: Critical

• 2.4 - Navigable
  • 2.4.1, 2.4.2 - Bypass Blocks, Page Titled (Level A)

    Summary: These guidelines are very basic functions that are included on most web-based applications today.

    • The first guideline deals with "skip to" links at the top of web pages which allow quicker navigation to those critical areas. View a short video on checking skip links.
    • The second simply requires an appropriate and descriptive title for each web page. View a short video on checking page titles.

    If they are unsupported, this should be a red flag.

    Priority: Critical

  • 2.4.7 - Focus Visible (Level AA)

    Summary: There must be an indicator for keyboard focus within a web-based application. This is critical for multiple reasons. This focus indicator is the primary way testers evaluate the keyboard access and tabbing order of an application. So, this can not only act as evidence of that testing process, it will also be critical for future testing of the application during update cycles.

    Priority: Critical

• 3.1 - Readable
  • 3.1.1 - Language of Page (Level A)

    Summary: This guideline ensures that a web page's language is programmatically defined for assistive technologies to access. Exceptions should be rare.

    Priority: Critical

• 3.2 - Predictable
  • 3.2.1 - On Focus (Level A)

    Summary: This guideline essentially prohibits the context change of a web element when it receives system focus through standard navigation. An example could be the automatic submission of a web form when the button receives focus, or the automatic launch of a new browser tab when a link gets focus.

    Priority: Critical

  • 3.2.2 - On Input (Level A)

    Summary: This guideline addresses changes in context when a user changes the state of a component, and the change will persist after interaction stops. An example of a state change would be checking a checkbox or entering text in a text box. These are elements where a user must be alerted to a context change to validate that element's status.

    Priority: Critical

• 3.3 - Input Assistance
  • 3.3.1 - Error Identification (Level A)

    Summary: This ensures that input entry errors are identified in a clear and descriptive manner to all users. While slightly ambiguous, the intent here is that any error message must be immediately identifiable and accessible to any user using any type of assistive technology. The error should also be described in a way that the user can figure out what is wrong.

    Priority: Critical

  • 3.3.2 - Labels or Instructions (Level A)

    Summary: There must be proper labeling and instructions for any application input fields. Examples include search fields or a web form. This is a barrier to access, so exceptions should be viewed cautiously.

    Priority: Critical

• 4.1 - Compatible
  • 4.1.1 - Parsing (Level A)

    Summary: This requires a web-based application's markup language (generally HTML) to be appropriately structured. These can include properly opening and closing tags, appropriate specifications for nested list elements, and input fields with associated attributes. Essentially, this requirement assures the proper valid syntax (format) of the markup language.

    Priority: Critical

  • 4.1.2 - Name, Role, Value (Level A)

    Summary: This guideline targets custom user interface elements created with scripting components instead of the standard markup language elements. It requires that these custom components' name, role, attributes, and any change of status be identifiable by assistive technologies. An example would be a custom checkbox being identified as "checkbox checked" after it has been marked by a user.

    Priority: Critical