W3C SVG

Audit Scope

Pages

The audit has been completed in June-July 2020 using the master branch running locally. The following pages have been audited:

  • Front page
  • Course archive
  • Log in and you have logged out
  • User profile

Additionally, using the course-template, the following pages have been audited:

  • Course index
  • Course content
  • Exercise results
  • Submit an exercise both for exercises that require typed input and for file uploads
  • View a submission

Technology

At this stage, the amount of technology used for testing is quite narrow.

Tested using Chrome

with Voiceover

 Firefox

with Orca

 Safari iOS

Hopefully this gives a good enough indication of the general state of the interfaces. In the future, it would be desirable to test more deeply with a range of common assistive technologies.

Many tests do not require assistive technology, these have been completed using Chrome on macOS.

Target level

So far, these tests relate to all guidelines of WCAG-A and WCAG-AA.

Things to Fix

WCAG-A Summary

Perceivable

Text Alternatives

  • 1.1.1 Non-text Content (level A):

    All non-text content that is presented to the user has a text alternative that serves the equivalent purpose, except for the situations listed below.

    • Controls, Input: If non-text content is a control or accepts user input, then it has a name that describes its purpose. (Refer to Guideline 4.1 for additional requirements for controls and content that accepts user input.)
    • Time-based Media: If non-text content is time-based media, then text alternatives at least provide descriptive identification of the non-text content. (Refer to Guideline 1.2 for additional requirements for media.)
    • Test: If non-text content is a test or exercise that would be invalid if presented in text, then text alternatives at least provide descriptive identification of the non-text content.
    • Sensory: If non-text content is primarily intended to create a specific sensory experience, then text alternatives at least provide descriptive identification of the non-text content.
    • CAPTCHA: If the purpose of non-text content is to confirm that content is being accessed by a person rather than a computer, then text alternatives that identify and describe the purpose of the non-text content are provided, and alternative forms of CAPTCHA using output modes for different types of sensory perception are provided to accommodate different disabilities.
    • Decoration, Formatting, Invisible: If non-text content is pure decoration, is used only for visual formatting, or is not presented to users, then it is implemented in a way that it can be ignored by assistive technology.

Some inappropriately described images are present, for example on the front page and profile page.

Some form fields are not described, including at log in and submitting exercises.

Time-based Media

  • 1.2.1 Audio-only and Video-only (Prerecorded) (level A):

    For prerecorded audio-only and prerecorded video-only media, the following are true, except when the audio or video is a media alternative for text and is clearly labeled as such.

    • Prerecorded Audio-only: An alternative for time-based media is provided that presents equivalent information for prerecorded audio-only content.
    • Prerecorded Video-only: Either an alternative for time-based media or an audio track is provided that presents equivalent information for prerecorded video-only content.
  • 1.2.2 Captions (Prerecorded) (level A):

    Captions are provided for all prerecorded audio content in synchronized media, except when the media is a media alternative for text and is clearly labeled as such.

  • 1.2.3 Audio Description or Media Alternative (Prerecorded) (level A):

    An alternative for time-based media or audio description of the prerecorded video content is provided for synchronized media, except when the media is a media alternative for text and is clearly labeled as such.

No time-based media features in the sample checked (nor anywhere in the platform outside of course-specific content).

Adaptable

  • 1.3.1 Info and Relationships (level A):

    Information, structure, and relationships conveyed through presentation can be programmatically determined or are available in text.

There are many cases of mislabelled forms, as covered on Form Controls, and missed or misordered heading levels, as mentioned in Semantic Markup and Sequences

  • 1.3.2 Meaningful Sequence (level A):

    When the sequence in which content is presented affects its meaning, a correct reading sequence can be programmatically determined.

The DOM order does not match the reading order in the profile page, and may be broken in some exercises.

  • 1.3.3 Sensory Characteristics (level A):

    Instructions provided for understanding and operating content do not rely solely on sensory characteristics of components such as shape, size, visual location, orientation, or sound.

No problems with sensory characteristics were found.

Distinguishable

  • 1.4.1 Use of Color (level A):

    Color is not used as the only visual means of conveying information, indicating an action, prompting a response, or distinguishing a visual element.

No information on the pages checked was conveyed using only colour.

  • 1.4.2 Audio Control (level A):

    If any audio on a Web page plays automatically for more than 3 seconds, either a mechanism is available to pause or stop the audio, or a mechanism is available to control audio volume independently from the overall system volume level.

No audio is used in the platform, and this audit did not consider specific courses.

Operable

Keyboard Accessible

  • 2.1.1 Keyboard (level A):

    All functionality of the content is operable through a keyboard interface without requiring specific timings for individual keystrokes, except where the underlying function requires input that depends on the path of the user's movement and not just the endpoints.

The modals displayed when submitting an assignment are not easily keyboard or screen reader accessible.

  • 2.1.2 No Keyboard Trap (level A):

    If keyboard focus can be moved to a component of the page using a keyboard interface, then focus can be moved away from that component using only a keyboard interface, and, if it requires more than unmodified arrow or tab keys or other standard exit methods, the user is advised of the method for moving focus away.

There was no keyboard trap found on any of the pages.

  • 2.1.4 Character Key Shortcuts (level A):

    If a keyboard shortcut is implemented in content using only letter (including upper- and lower-case letters), punctuation, number, or symbol characters, then at least one of the following is true:

    • Turn off: A mechanism is available to turn the shortcut off;
    • Remap: A mechanism is available to remap the shortcut to use one or more non-printable keyboard characters (e.g. Ctrl, Alt, etc);
    • Active only on focus: The keyboard shortcut for a user interface component is only active when that component has focus.

No character key shortcuts are used.

Enough Time

  • 2.2.1 Timing Adjustable (level A):

    For each time limit that is set by the content, at least one of the following is true:

    • Turn off: The user is allowed to turn off the time limit before encountering it; or
    • Adjust: The user is allowed to adjust the time limit before encountering it over a wide range that is at least ten times the length of the default setting; or
    • Extend: The user is warned before time expires and given at least 20 seconds to extend the time limit with a simple action (for example, 'press the space bar'), and the user is allowed to extend the time limit at least ten times; or
    • Real-time Exception: The time limit is a required part of a real-time event (for example, an auction), and no alternative to the time limit is possible; or
    • Essential Exception: The time limit is essential and extending it would invalidate the activity; or
    • 20 Hour Exception: The time limit is longer than 20 hours.
  • 2.2.2 Pause, Stop, Hide (level A):

    For moving, blinking, scrolling, or auto-updating information, all of the following are true:

    • Moving, blinking, scrolling: For any moving, blinking or scrolling information that (1) starts automatically, (2) lasts more than five seconds, and (3) is presented in parallel with other content, there is a mechanism for the user to pause, stop, or hide it unless the movement, blinking, or scrolling is part of an activity where it is essential; and
    • Auto-updating: For any auto-updating information that (1) starts automatically and (2) is presented in parallel with other content, there is a mechanism for the user to pause, stop, or hide it or to control the frequency of the update unless the auto-updating is part of an activity where it is essential.

The only time limit is the session duration, after which the user is logged out. From inspection of the code, this appears to be around two weeks. This passes the requirements, as it is above the 20 hour exception.

Seizures and Physical Reactions

  • 2.3.1 Three Flashes or Below Threshold (level A):

    Web pages do not contain anything that flashes more than three times in any one second period, or the flash is below the general flash and red flash thresholds.

Nothing flashing is present on any of the audited pages.

  • 2.4.1 Bypass Blocks (level A):

    A mechanism is available to bypass blocks of content that are repeated on multiple Web pages.

Skip links are not present, but currently a work in progress.

Page titles are present and descriptive. Page Titles includes notes on possible improvements.

  • 2.4.3 Focus Order (level A):

    If a Web page can be navigated sequentially and the navigation sequences affect meaning or operation, focusable components receive focus in an order that preserves meaning and operability.

The focus order is not always logical.

  • 2.4.4 Link Purpose (In Context) (level A):

    The purpose of each link can be determined from the link text alone or from the link text together with its programmatically determined link context, except where the purpose of the link would be ambiguous to users in general.

Many pages have inditinguishable link text or non-specific text. For example, the course points page includes the link text “1/5”, referring to a submission count, many times.

Understandable

Readable

  • 3.1.1 Language of Page (level A):

    The default human language of each Web page can be programmatically determined.

The language of the page is stated correctly.

Predictable

  • 3.2.1 On Focus (level A):

    When any component receives focus, it does not initiate a change of context.

A focus state is implemented on master, but is not yet in production.

Input Assistance

  • 3.3.1 Error Identification (level A):

    If an input error is automatically detected, the item that is in error is identified and the error is described to the user in text.

Labels are not always present or visible. It is not possible to always ascertain the label visually or programatically.

Robust

Compatible

  • 4.1.1 Parsing (level A):

    In content implemented using markup languages, elements have complete start and end tags, elements are nested according to their specifications, elements do not contain duplicate attributes, and any IDs are unique, except where the specifications allow these features.

  • 4.1.2 Name, Role, Value (level A):

    For all user interface components (including but not limited to: form elements, links and components generated by scripts), the name and role can be programmatically determined; states, properties, and values that can be set by the user can be programmatically set; and notification of changes to these items is available to user agents, including assistive technologies.

HTML is generally valid, although some views include repeated IDs.

Roles of button and link appear unpredictably and inconsistently. Links and Buttons explains more.

WCAG-AA Summary

Perceivable

Time-based Media

The interfaces examined don’t use any live media or prerecorded video, so there is nothing to require captions or audio description.

Adaptable

  • 1.3.4 Orientation (level AA):

    Content does not restrict its view and operation to a single display orientation, such as portrait or landscape, unless a specific display orientation is essential.

All pages work equally well in either mobile orientation, except the “Results” page, which is inaccessible on certain mobile screen sizes (tested using iPhone Simulator) in portrait (notably, the smartphones available for testing were all bigger than this size).

  • 1.3.5 Identify Input Purpose (level AA):

    The purpose of each input field collecting information about the user can be programmatically determined when:

    • The input field serves a purpose identified in the Input Purposes for User Interface Components section; and:
    • The content is implemented using technologies with support for identifying the expected meaning for form input data.:

No forms have autocorrect attributes. This complies with the requirements except for;

  • Login - local user login should explicitly identify the username and password to enable assistive technology to autofill the fields.
  • Profile - the language field may benefit from identification.

Further to this, it may be worth considering adding autocomplete="off" to questionnaire forms, where accidental submission consumes available submissions from a limited pool.

Distinguishable

  • 1.4.3 Contrast (Minimum) (level AA):

    The visual presentation of text and images of text has a contrast ratio of at least 4.5:1, except for the following:

    • Large Text: Large-scale text and images of large-scale text have a contrast ratio of at least 3:1;
    • Incidental: Text or images of text that are part of an inactive user interface component, that are pure decoration, that are not visible to anyone, or that are part of a picture that contains significant other visual content, have no contrast requirement.
    • Logotypes: Text that is part of a logo or brand name has no minimum contrast requirement.

There are many elements with too low colour contrast. The footer, which is visible on every page is one of these, as well as many of the badges, buttons and alerts.

Most of these are unstyled Bootstrap components, so it may make sense to re-assess this once there is a new Bootstrap theme in place, to find which need addressing once the theme colours are eliminated.

  • 1.4.4 Resize text (level AA):

    Except for captions and images of text, text can be resized without assistive technology up to 200 percent without loss of content or functionality.

Adjusting the text size independently of the page zoom does not work.

  • 1.4.5 Images of Text (level AA):

    If the technologies being used can achieve the visual presentation, text is used to convey information rather than images of text except for the following:

    • Customizable: The image of text can be visually customized to the user's requirements;
    • Essential: A particular presentation of text is essential to the information being conveyed.

No images of text could be found.

  • 1.4.10 Reflow (level AA):

    Content can be presented without loss of information or functionality, and without requiring scrolling in two dimensions for:

    • Vertical scrolling content at a width equivalent to 320 CSS pixels;:
    • Horizontal scrolling content at a height equivalent to 256 CSS pixels.:
    • Except for parts of the content which require two-dimensional layout for usage or meaning.:

The responsive design means that reflow works as intended up to 400% (also tested at 500%). The only exception is the table on the course results page, which overflows horizontally.

  • 1.4.11 Non-text Contrast (level AA):

    The visual presentation of the following have a contrast ratio of at least 3:1 against adjacent color(s):

    • User Interface Components: Visual information required to identify user interface components and states, except for inactive components or where the appearance of the component is determined by the user agent and not modified by the author;
    • Graphical Objects: Parts of graphics required to understand the content, except when a particular presentation of graphics is essential to the information being conveyed.

Most pages met the “non-text contrast” requirement. The elements that did not were;

  • input text fields, which have too low a border colour (this can likely be addressed with the Bootstrap theme)
  • Classes .badge and .label do not always have sufficient contrast against their background.
  • 1.4.12 Text Spacing (level AA):

    In content implemented using markup languages that support the following text style properties, no loss of content or functionality occurs by setting all of the following and by changing no other style property:

    • Line height (line spacing) to at least 1.5 times the font size;:
    • Spacing following paragraphs to at least 2 times the font size;:
    • Letter spacing (tracking) to at least 0.12 times the font size;:
    • Word spacing to at least 0.16 times the font size.:
    • Exception: Human languages and scripts that do not make use of one or more of these text style properties in written text can conform using only the properties that exist for that combination of language and script.:

Adjusting text spacing works corretly (this was tested using the text spacing bookmarklet). The only exception was the <dl> at the top of course content pages (tested using course-template), which caused some horizontal overflow.

This was also tested with the front page from the current master branch - note that the previous implementation failed.

  • 1.4.13 Content on Hover or Focus (level AA):

    Where receiving and then removing pointer hover or keyboard focus triggers additional content to become visible and then hidden, the following are true:

    • Dismissable: A mechanism is available to dismiss the additional content without moving pointer hover or keyboard focus, unless the additional content communicates an input error or does not obscure or replace other content;
    • Hoverable: If pointer hover can trigger the additional content, then the pointer can be moved over the additional content without the additional content disappearing;
    • Persistent: The additional content remains visible until the hover or focus trigger is removed, the user dismisses it, or its information is no longer valid.
    • Exception: The visual presentation of the additional content is controlled by the user agent and is not modified by the author.:

Very few cases of content (like tooltips) appearing on hover were found in the pages sampled. Those that were found always had an element of reduncancy, although they did not meet these requirements (usually as they were not hoverable).

Operable

  • 2.4.5 Multiple Ways (level AA):

    More than one way is available to locate a Web page within a set of Web pages except where the Web Page is the result of, or a step in, a process.

Generally, if we consider courses to be “subsites”, these criteria are basically met. It’s possible to navigate course pages either;

  • From a “contents” list of chapters and pages
  • From a list of point-bearing exercises, grouped by page
  • Using “back” and “next” links within the pages.

It’s always possible to reach the front page and other key pages, and the menu of the user’s own courses provides quick links to the pages the user is most likely to want to access.

Navigation within courses, and jumping from one course to another is provided in at least two ways.

Throughout all pages, except “log out” and the front page, there are mismatched heading levels. These can be examined most easily in WAVE, and should create a logical, nexted tree structure.

Understandable

Readable

  • 3.1.2 Language of Parts (level AA):

    The human language of each passage or phrase in the content can be programmatically determined except for proper names, technical terms, words of indeterminate language, and words or phrases that have become part of the vernacular of the immediately surrounding text.

The version of the front page on master is the only part where the language of parts is correctly identified. The same technique should be applied to the course archive page. Additionally, the language names are present in the profile page.

No other pages use multiple languages which are visible simeltaneously (some courses do, but specific courses were out of scope for this review).

Predictable

  • 3.2.3 Consistent Navigation (level AA):

    Navigational mechanisms that are repeated on multiple Web pages within a set of Web pages occur in the same relative order each time they are repeated, unless a change is initiated by the user.

The navigation is present in the same location on every page.

The only element which is unpredictable is the menu which allows users to choose a group to submit with, as it is related to the course, but appears in the main navigation area, and may appear differently in different courses.

Most elements can be identified consistently. Issues have been identified in the previous audit about the use of Links and Buttons .

An important note here is that teachers are free to override the CSS within courses. This allows individual courses to use inconsistent identification for elements, and is outside of the scope of this audit. It may be worth producing documentation for teachers covering the importance of not overriding some of the default styles, or to consider limiting the effect of teacher styles to only the content area.

Input Assistance

  • 3.3.3 Error Suggestion (level AA):

    If an input error is automatically detected and suggestions for correction are known, then the suggestions are provided to the user, unless it would jeopardize the security or purpose of the content.

Forms don’t always show error messages when fields are missed or in the incorrect format. Those that do may have unclear error messages.

  • 3.3.4 Error Prevention (Legal, Financial, Data) (level AA):

    For Web pages that cause legal commitments or financial transactions for the user to occur, that modify or delete user-controllable data in data storage systems, or that submit user test responses, at least one of the following is true:

    • Reversible: Submissions are reversible.
    • Checked: Data entered by the user is checked for input errors and the user is provided an opportunity to correct them.
    • Confirmed: A mechanism is available for reviewing, confirming, and correcting information before finalizing the submission.

No pages cause legal or financial consequences, however since submissions are finite and contribute to university grades, they may meet the definition of “testing”.

Submissions are not reversible. No mechanism to check values before submission are provided. No confirmation step is used. Any of these would meet the criteria. This has already been discussed earlier in GitHub issue #508 and MOOC-grader issue #56.

Robust

Compatible

  • 4.1.3 Status Messages (level AA):

    In content implemented using markup languages, status messages can be programmatically determined through role or properties such that they can be presented to the user by assistive technologies without receiving focus.

The majority of sampled pages do not contain “live” status messages.

The status messages shown, when submitting an exercise, are not marked up appropriately or highlighted to assistive technology.

What's New?

20 June 2020

  • Update all WCAG links to point to WCAG 2.1.
  • Publish summary of WCAG-AA results.

25 June 2020

  • Initial publication following an audit at WCAG-A level.

Form Controls

As a web application, A+ depends on forms to take user input for a range of purposes. The following forms were examined in the tests carried out:

  • Log in (local users)
  • Edit profile
  • Create a group
  • Submit an exercise requiring textual input
  • Submit an exercise requiring a file upload

Requirements

The following relevant requirements from WCAG were examined:

In short, any controls present must:

  • be labelled in a programatically determinable way
  • make use of semantic markup, or, where this isn’t possible, have their name and role explicitly stated
  • have a visible label
  • make any detected errors in the user’s input clear in text

Issues

The following are examples of the issues found.

(Fixed in A+ v1.8) Local user login

No visual indication is given if login fails. A clear error message and styling to match the error states of other text fields in the application would make this more understandable.

GitHub issue (591)

(Fixed in A+ v1.8) Submit a text-based exercise

The form control used to type the input to an exercise is associated with a label, although the label only contains the question number, and no clear indication of what should be entered into the field.

GitHub issue (592)

Profile

There is no indication if updating the user’s language suceeded or failed. Introducing an invalid value intentionally doesn’t appear to cause an error condition.

GitHub issue (591)

Submit a file upload exercise

This works correctly, however the label points to a field with a “non-unique id”, so the correct label cannot be programatically determined.

GitHub issue (593)

Link Purpose

Requirements

Sometimes, assistive technology users will look at a list of links or headings from a web page. VoiceOver allows users to do this from the “rotor” while browsing the web.

By ensuring the text is descriptive and unique, we enable users to quickly find what they are looking for. If every link were to, for example, consist of the words “click here”, it would be difficult to identify which one points to which destination.

This is especially apparent on the front page, where some of the links have the instance name (often a year number) as the label.

Partial screenshot showing the rotor listing front page links

Examples

Course Results

The course results page contains similar text for every exercise, for example:

1/10

in context, it’s visible that this is in the “Submissions” column, but when viewing the list of links, these are somewhat meaningless.

As covered in the page “Links and Buttons”, these don’t perform the action of a “link”, (they open a drop-down menu), so they needn’t be visible on the list of links at all.

There is a GitHub issue (597)

Links and Buttons

Requirements

One key requirement is that names, roles and values can be determined programmatically. The roles are most often deduced from HTML elements (so, for example, a <button> tag can safely be assumed to be a “button”).

In several places, A+ used buttons and links in a different way to the expectation.

Users typically expect that clicking a link takes the user to another page, and clicking a button performs an action.

Examples

Front Page

Each course on the front page contains a block which looks like this:

Partial screenshot showing the links to one course

This contains a link (the title text) and a button. Both behave link links. Since they have different roles, VoiceOver announces one as a link and one as a button.

Additionally, if a user wants to navigate by headings or form elements, these links will appear in both lists.

Course Index

On the course index page, when the recent results are shown, there is something that appears to be a text link, labelled as:

Show older assignments.

This is marked up as a link, but changes the content of the current page. Users may expect to be taken to a different page, instead the content of this pages with no clear indication.

If this were labelled as a button, it would give a much clear indication that “clicking this does something”. There could also be benefits to users not using assistive technology to acheive a highler level of consistency in how interface elements are used.

Modal Dialogs

The interface makes heavy use of modal dialogs, for example when a user submits an exercise, the grader results appear in a modal window and when a user views a previous submission, it opens in one. Sometimes, links from modals trigger other modals to open.

Presently, these are not navigable to keyboard users and break the logical focus order. This causes failures in:

Mouse-users’ experience

Presently, mouse users can click a link or button (these should likely be buttons, as addressed in #TODO), and the modal fills the page. The rest of the page is “dimmed”. Clicking the background closes the modal, and a close button is present. It’s not possible to scroll the background page.

Keyboard-users’ experience

It’s possible to follow the links that open the modal with the keyboard, however focus does not move to inside the modal. Continuing to navigate around the page using the tab button is possible, and the background page moves underneath the overlay.

The user can enter the modal by tabbing all the way to the end of the page.

Upon exiting the modal, either using ESC or the close button, focus is typically at the top of the page, rather than the point where the user might be expecting it, at the point they left.

Suggestions

  1. Double-check all of these functions are best served by modal dialogs, rather than opening a new page. GitHub issue #589
  2. Consider the action that opens the modal. A link may not be the most appropriate option here, as the user isn’t being taken to a new page.
  3. Record the item the user activates when they open the modal, return focus there when they close it.
  4. Ensure keyboard users can only focus on the elements inside the visible modal, and can’t interact with the background page while it is open. Take a look at the proposed inert attribute.

Page Titles

Requirements

The requirements relating to the page titles are being met. All tested pages have descriptive, discernable names, with the most specific information first and the website name last.

Possible Improvements

Through testing, it became clear that the use of the | (pipe) character as a divider is annoying when using VoiceOver. Page titles are spoken as:

Results vertical line programming 2 vertical line a plus

The words “vertical line” take approximately as much speech time than the relevant content.

Replacing this with a - (hyphen) results in a short pause and no description, which may be preferable. That said, this is “low” priority, as many websites use the | character and screen reader users are generally used to this convention.

Semantic Markup and Sequences

Semantic markup

When users of screen readers navigate pages, some may rely on the structure of the document to move around. This may mean taking an “outline”, made up of the nested headings, or using landmarks, such as nav and main to jump around the web page.

Luckily, these often don’t need defining explicitly, as many native HTML5 elements have an implied role. In some places, these could be easily swapped into the layout. For example, <div class="section"> could be easily substituted for <section>. to be made by using HTML5 elements in place of some classes.

This would improve compliance with the following success criteria:

Sequences

Since users may replace some of the styles offered by the page, or may move through the page using the keyboard or a screen reader, it’s iportant that the page makes sense if some style information is missing.

Ensuring that the items appear in the DOM in a logical order will help accomplish this, as defined in:

Examples and solutions

This list is likely not exhaustive:

(Fixed in A+ v1.8) Course archive

The course archive uses h3 tags to represent courses. These are wrapped in lis.

First, there is no h2, so they’re nested directly under the page’s h1.

Also, they don’t behave like headings, it doesn’t make sense to encourage users to jump to them within the page as they would subheadings in a document, since there is no further content to explore, only the link to follow.

The formatting could be maintained without using a heading tag, and since the courses are sorted chronologically, subheadings could be used to, for example, group the courses by year.

There is a GitHub issue (596)

(Fixed in A+ v1.8) Profile

The sequence of elements in the profile page does not match reading order. Looking at the page visually, the user might follow from the avatar, to the blue “info” block on the right which explains how to change it.

Following the DOM order, as a screen reader would, the user first passes the avatar, before continuing through the next fields, and then returns to the block explaining how to change their avatar.

There is a GitHub issue (590)