Audio & Video
[Author: Bill Fischer]
Cognitive Inclusion
Motion Design for a Neurodiverse Population
Guidelines
Live-action videos, animation, and special effects should be paced in way that does not cause negative reactions or stress to persons with cognitive challenges such as autism. Integration of the following design guidelines can help to accomplish this:
Overall pacing that is easy to follow and has generous pauses between action sequences and scenes that allow for caption or audio description. Consider these scenarios:
Do not use large rapidly animating images that can cause epileptic seizures (no more than 3 blinks per second... or between 2 and 25 hz). Also, avoid overlaid complementary colors or stripes that may scintillate. Guidelines from Webaim (external link).
When users are consuming video, they should periodically be supplied with orienting devices that reduce cognitive load. It can include:
Title cards.
Establishing scene shots.
A linear story structure.
A clear, narrative-driven visual hierarchy within each scene.
Examples
(external links)(created as part of The Epic Project)
This animation neurodiversity, created as part of The Epic Project, with integrated captions contains a segment that is paced well and allows ample time to read the (integrated) captions
This animation about anxiety for Mental Health America contains a segment that requires reading at a faster rate and exerts a higher cognitive load.
The same animation about anxiety is an example of audio descriptions of on-screen action squeezed into the existing narration gaps.
This animated Motion Graphic called Spectre contains A few seconds that may be overwhelming for some and trigger seizures in others.
Integrated Audio Descriptions & Sound Design For The Sight impaired
This approach aims to create an experience that sight, hearing, and cognitively impaired persons can experience together. One that is optimized to provide a quality experience for all, at the same time, in the same place, on the same channel.
Writing visual narratives and integrating story-supporting sound design can often negate the need for separate audio descriptions.
Examples:
This script for this animation about the Harlem Hellfighters, created for the Jim Crow Museum as part of The Epic Project, contains an opening scene that integrates additional text and sound design which paints a picture for sight-impaired persons.
The blue text, below, hilites the added text that serves as audio description.
“They never lost a man through capture, lost a trench, or a foot of ground to the enemy.” wrote historian, Arthur W. Little.”
audio notes:
environment: battle sounds (artillery, gunfire)
foley: a single shot from a gun, reloading, then another
The Prairie Home Companion live radio broadcasts excel at storytelling with sound design
Theme Song: 00:05 - 00:10 (setting up the cowboy genre)
Foley: 03:07 - 3:15 (traveling by horse)
Scoring: 03:23 - 3:50 (setting sense of mood)
Environment 04:50 - (piano &laughing in a saloon)
Traditional Audio Descriptions for The Sight Impaired
When To Use Them
If audio-visual storytelling has not been applied to a video or animation, audio descriptions may be necessary.
Examples
Here are some examples from the CYC animated poems, created as part of The Epic Project. (external links)
I will Rise (descriptions are all at the beginning)
Rise Up (video is paused for Descriptions)
I am Music (descriptions are at the beginning and end)
Anxiety (descriptions are quickly fitted intermittently)
Google Slides Storyboard Template
Use this Storyboard Template. It has form fields to help you map out the closed caption timing.
Open the file, then choose File > Make a Copy.
Rename your copy.
Choose your thesis project folder for the destination.
Open the copied slideshow, insert your storyboard images.
Add narration/dialogue, timing, and sound design notes
Note: Create boards at full HD size: 1920x1080
Audio Description Guidelines
Audio descriptions are narration added to the soundtrack to describe important visual detail that cannot be understood from the main soundtrack alone. These are for the benefit of sight-impaired persons. The four cornerstones of audio description are “what, where, who, and how”. If there isn't time between the dialogue to fit all of that in... choose the description that best supports the understanding of the story being told. Think about how much needs to be described in detail versus how much can be left to the audience's imagination... and still tell the story.
Audio and video must have controls that allow the user to pause and restart. It should not autoplay.
Standard audio descriptions fit the voice-over within the existing timing of the video. This can sometimes require difficult choices resulting in the exclusion of some information.
Extended audio descriptions pause, or allow the user to pause, the video to allow the inclusion of more detailed information.
Audio Transcripts for the hearing Impaired
Transcripts Overview
Audio only media should have an audio text transcription, for the hearing impaired, which includes a written description of the dialogue, narration and pertinent sound effects.
Transcript Writing Guide
Decide if you will be using verbatim (every "um", etc.) or clean (without the "um"s, etc.).
Speakers should be identified by name when it is important to to the narrative. Be consistent when identifying speakers.
Note important non-voice audio such as sound effects, foley, environment and music.
Use adjectives to characterize an action as needed (such as: laughing sarcastically vs. laughing knowingly vs. laughing to themselves).
Aim for 99% accuracy in spelling
Add punctuation
Integrated Captioning for the hearing-Impaired
Overview and Examples
Captions can be integrated into the animation as part of the design. This is a more universal approach. Many of these, however, don't really leave the text on the screen long enough to read, let alone watch the animated visuals.
Example animations:
(all external links)
Percival Prattis (created for The Jim Crow Museum as part of The Epic Project).
MHA: Depression (created for Mental Health America as part of The Epic Project).
Anxiety (created for the Neuromedia website as part of The Epic Project).
Google Slides Storyboard Template
Use this Storyboard Template. It has form fields to help you map out the integrated audio descriptions.
Open the file, then choose File > Make a Copy.
Rename your copy.
Choose your thesis project folder for the destination.
Open the copied slideshow, insert your storyboard images.
Add narration/dialogue, timing, and sound design notes
Note: Create boards at full HD size: 1920x1080
Traditional Captioning for the hearing-Impaired
Captioning is an add-on to videos that have not been optimized for sight-impaired persons. It is a 'fix'. In most cases, the viewer is tasked to continuously decide whether to watch the action or read the captions because the video timing has not been set to allow for both. Alternatively, the viewer may stop the video to have time to read the captions without missing the action. But, that does not facilitate watching along with hearing persons and breaks the flow of the video as intended by the producer. It is a separate and unequal experience.
The FCC requires 99% accuracy for all television programs. This standard has become the unofficial, defacto target for videos delivered on all platforms
Optimizing video Timing for Captioning
It's often difficult or impossible to read captions and watch the action in a video simultaneously. When this is the case, we are forced to choose one or the other. Or, frequently pause the video. This can be avoided by alternating the screen time between important actions and narration or dialogue. In other words, don't overlap important actions with spoken words that will be captioned in the timing of the video.
Layout For Captioning
The image below defines the area in a video that should be free of important information because it may be covered by the captions. The rule is to keep important content above 280 pixels from the bottom.
Video Tutorial for Automated Captioning in YouTube
YouTube will auto-generate captions for uploaded videos. If your dialogue or narration is crisp and well-annunciated, YouTube's automated captions can achieve 99% accuracy.
To improve accuracy, YouTube has a built-in tool for captioning that allows editing for the creator and user controls for the viewer (such as Font size, and color) as required by ADA regulations.
This YouTube Captioning Video Tutorial (external link) by Filmora MVP. Published on July 2, 2021, will walk you through the process of using YouTube's and Facebook's tools for setting up and editing automated captioning (as well as several other online video editors). Editing automatic captions is important to achieve a high degree of accuracy.
Manually Written Caption Writing Guidelines Overview
Videos should have manually written captions (for the hearing impaired). They can be closed (off by default) or open (on by default). When using YouTube, the captions can be edited and re-timed then downloaded for off-line use if desired.
Caption Writing Guide
Decide if you will be using verbatim (every "um", etc.) or clean (without the "um"s, etc.).
Speakers should be identified by name when it is important to to the narrative. Be consistent when identifying speakers.
Sync the captions to the video.
Ensure the captions are visible long enough to read at a moderate pace.
Note important non-voice audio such as sound effects, foley, environment and music. These can disappear after a few seconds
Use adjectives to characterize an action as needed (such as: laughing sarcastically vs. laughing knowingly vs. laughing to themselves).
Use a san-serif font
Aim for 99% accuracy in spelling. punctuation, and grammar. No paraphrasing is allowed.
Limit one to three lines per caption frame, with no more than 32 characters per line.
Place captions on the screen without blocking important content. See 'Design For Captioning' below.