Audio & Video
[Author: Bill Fischer]
Cognitive Inclusion
Motion Design for a Neurodiverse Population
Guidelines
Live-action videos, animation, and special effects should be paced in way that does not cause negative reactions or stress to persons with cognitive challenges such as autism. Integration of the following design guidelines can help to accomplish this:
Overall pacing that is easy to follow and has generous pauses between action sequences and scenes that allow for caption or audio description. Consider these scenarios:
Do not use large rapidly animating images that can cause epileptic seizures (no more than 3 blinks per second... or between 2 and 25 hz). Also, avoid overlaid complementary colors or stripes that may scintillate. Guidelines from Webaim (external link).
When users are consuming video, they should periodically be supplied with orienting devices that reduce cognitive load. It can include:
Title cards.
Establishing scene shots.
A linear story structure.
A clear, narrative-driven visual hierarchy within each scene.
Examples
(external links)(created as part of The Epic Project)
This animation neurodiversity, created as part of The Epic Project, with integrated captions contains a segment that is paced well and allows ample time to read the (integrated) captions
This animation about anxiety for Mental Health America contains a segment that requires reading at a faster rate and exerts a higher cognitive load.
The same animation about anxiety is an example of audio descriptions of on-screen action squeezed into the existing narration gaps.
This animated Motion Graphic called Spectre contains A few seconds that may be overwhelming for some and trigger seizures in others.
Audio Storytelling For The Sight impaired
This approach aims to create an experience that sight, hearing, and cognitively impaired persons can experience together. One that is optimized to provide a quality experience for all, at the same time, in the same place, on the same channel.
Writing visual narratives and integrating story-supporting sound design can often negate the need for separate audio descriptions.
Examples:
This script for this animation about the Harlem Hellfighters, created for the Jim Crow Museum as part of The Epic Project, contains an opening scene that integrates additional text and sound design that paints a picture for sight-impaired persons.
“They never lost a man through capture, lost a trench, or a foot of ground to the enemy.” wrote historian, Arthur W. Little.”
audio notes:
environment: battle sounds (artillery, gunfire)
foley: a single shot from a gun, reloading, then another
The Prairie Home Companion live radio broadcasts excel at storytelling with sound design
Theme Song: 00:05 - 00:10 (setting up the cowboy genre)
Foley: 03:07 - 3:15 (traveling by horse)
Scoring: 03:23 - 3:50 (setting sense of mood)
Environment 04:50 - (piano &laughing in a saloon)
Audio Descriptions for The Sight Impaired
When To Use Them
If audio-visual storytelling has not been applied to a video or animation, audio descriptions may be necessary.
Examples
Here are some examples from the CYC animated poems, created as part of The Epic Project. (external links)
Audio Description Guidelines
If Video should have audio description accompanying video (for the sight impaired) ... defined as narration added to the soundtrack to describe important visual detail that cannot be understood from the main soundtrack alone. The four cornerstones of audio description are “what, where, who and how”. If there is not time between dialogue to fit all of that in... choose the description that most benefits the story-understanding. Think about how much needs to be described in detail versus how much can be left to the audiences imagination... and still tell the story.
Audio and video can be paused and stopped. Audio/video should not autoplay.
Standard audio descriptions fit the voice over within the existing timing of the video. This can sometimes require difficult choices resulting in the exclusion of some information.
Extended audio descriptions pause, or allow the use to pause, the video to allow the inclusion of more detailed information.
Audio Transcripts for the hearing Impaired
Transcripts Overview
Audio only media should have an audio text transcription, for the hearing impaired, which includes a written description of the dialogue, narration and pertinent sound effects.
Transcript Writing Guide
Decide if you will be using verbatim (every "um", etc.) or clean (without the "um"s, etc.).
Speakers should be identified by name when it is important to to the narrative. Be consistent when identifying speakers.
Note important non-voice audio such as sound effects, foley, environment and music.
Use adjectives to characterize an action as needed (such as: laughing sarcastically vs. laughing knowingly vs. laughing to themselves).
Aim for 99% accuracy in spelling
Add punctuation
Integrated Captioning for the hearing Impaired
Captions can be integrated into the animation as part of the design. This is a more universal approach. Many of these, however, don't really leave the text on the screen long enough to read, let alone watch the animated visuals.
Example animations:
(all external links)
Percival Prattis (created for The Jim Crow Museum as part of The Epic Project).
MHA: Depression (created for Mental Health America as part of The Epic Project).
Insight Into Neurodiversity (created for the Neuromedia website as part of The Epic Project).
Traditional Captioning for the hearing Impaired
Captioning is an add-on to videos that have not been optimized for sight-impaired persons. It is a 'fix'. In most cases, the viewer is tasked to continuously decide whether to watch the action or read the captions because the video timing has not been set to allow for both. Alternatively, the viewer may stop the video to have time to read the captions without missing the action. But, that does not facilitate watching along with hearing persons and breaks the flow of the video as intended by the producer. It is a separate and unequal experience
Caption Writing Guidelines; Overview
Video should have manually written captions (for the hearing impaired). They can be closed (off by default) or open (on by default). When using YouTube, the captions can be edited and re-timed then downloaded for off-line use if desired.
Captioning Guide
Decide if you will be using verbatim (every "um", etc.) or clean (without the "um"s, etc.).
Speakers should be identified by name when it is important to to the narrative. Be consistent when identifying speakers.
Sync the captions to the video.
Ensure the captions are visible long enough to read at a moderate pace.
Note important non-voice audio such as sound effects, foley, environment and music. These can disappear after a few seconds
Use adjectives to characterize an action as needed (such as: laughing sarcastically vs. laughing knowingly vs. laughing to themselves).
Use a san-serif font
Aim for 99% accuracy in spelling. punctuation, and grammar. No paraphrasing is allowed.
Limit one to three lines per caption frame, with no more than 32 characters per line.
Place captions on the screen without blocking important content. See 'Design For Captioning' below.
Layout For Captioning
The image below defines the area in a video that should be free of important information because it may be covered by the captions. The rule is to keep important content above 280 pixels from the bottom.
Video Tutorial for Captioning in YouTube
YouTube has a built-in tool for captioning that allows editing for the creator and user controls for the the viewer (such as Font size, and color) as required by ADA regulations.
This YouTube Captioning Video Tutorial (external link) by vidIQ. Published on April 25, 2020 will walk you through the process of using YouTube's tools for captioning.