Audio & Video
[Author: Bill Fischer]
Video Captioning
Caption Writing Guidelines; Overview
Video should have manually written captions (for the hearing impaired). They can be closed (off by default) or open (on by default). When using YouTube, the captions can be edited and re-timed then downloaded for off-line use if desired. These Jean Hanks Lecture Videos are examples of open captioning.
Captioning Guide
Decide if you will be using verbatim (every "um", etc.) or clean (without the "um"s, etc.).
Speakers should be identified by name when it is important to to the narrative. Be consistent when identifying speakers.
Sync the captions to the video.
Ensure the captions are visible long enough to read at a moderate pace.
Note important non-voice audio such as sound effects, foley, environment and music. These can disappear after a few seconds
Use adjectives to characterize an action as needed (such as: laughing sarcastically vs. laughing knowingly vs. laughing to themselves).
Use a san-serif font
Aim for 99% accuracy in spelling. punctuation, and grammar. No paraphrasing is allowed.
Limit one to three lines per caption frame, with no more than 32 characters per line.
Place captions on the screen without blocking important content. See 'Design For Captioning' below.
DESIGN FOR CAPTIONING
The image below defines the area in a video that should be free of important information because it may be covered by the captions.
Video Tutorial for Captioning in YouTube
YouTube has a built-in tool for captioning that allows editing for the creator and user controls for the the viewer (such as Font size, and color) as required by ADA regulations.
This YouTube Captioning Video Tutorial (external link) by vidIQ. Published on April 25, 2020 will walk you through the process of using YouTube's tools for captioning.
Integrated Captioning
Captions can also be integrated into the animation as part of the design. This is a more universal approach. Most of these, however, don't really leave the text on the screen long enough to read, let alone watch the animated visuals.
Examples:
(all external links)
Audio-Visual Storytelling
Writing Visual Narratives
Writing visual narratives can often negate the need for separate audio descriptions.
Video Writing Example 1 (Standard Form)
(2 versions needed - one with audio description and one without):
Narrator says: Sometimes I wonder what my dog is thinking
Image shows: Person on their knees looking directly into a dog's eyes from 6 " away
Video Writing Example 2 (Visual Narrative Form)
(1 version needed - because the narrator describes the visual action):
Narrator says: Sometimes I stare directly into my dog's eyes, wondering what she is thinking
Image shows: Person on their knees looking directly into a dog's eyes from 6 " away
Visual Narrative Examples
(external links)
Affordance
When users are consuming video, they should periodically be supplied orienting devices that reduce cognitive load. This can include:
Title cards.
Establishing scene shots.
A linear story structure.
A clear, narrative-driven visual hierarchy within each scene.
Sound Design
This is the practice, used in radio for many years, of adding audio to recordings that reinforce, and sometimes, outright tell a visual story.
Scoring can add emotion
Environment and Foley can tell "visual" stories
Here are some podcasts that excel at audio-visual storytelling (external links):
Prairie Home Companion live radio broadcast video
Theme Song: 00:05 - 00:10 (setting up the cowboy genre)
Foley: 03:07 - 3:15 (traveling by horse)
Scoring: 03:23 - 3:50 (setting sense of mood)
Environment 04:50 - (piano &laughing in a saloon)
Audio Descriptions for Video
When To Use Them
If audio-visual storytelling has not been applied to a video or animation, audio descriptions may be necessary.
Examples:
Audio Descriptions from the CYC project (external link) (includes several different approaches)
Most feature films and original content on Netflix
Audio Description Guidelines
If Video should have audio description accompanying video (for the sight impaired) ... defined as narration added to the soundtrack to describe important visual detail that cannot be understood from the main soundtrack alone. The four cornerstones of audio description are “what, where, who and how”. If there is not time between dialogue to fit all of that in... choose the description that most benefits the story-understanding. Think about how much needs to be described in detail versus how much can be left to the audiences imagination... and still tell the story.
Audio and video can be paused and stopped. Audio/video should not autoplay.
Standard audio descriptions fit the voice over within the existing timing of the video. This can sometimes require difficult choices resulting in the exclusion of some information.
Extended audio descriptions pause, or allow the use to pause, the video to allow the inclusion of more detailed information.
Audio Only Transcripts
Transcripts Overview
Audio only media should have an audio text transcription, for the hearing impaired, which includes a written description of the dialogue, narration and pertinent sound effects.
Writing in visual narrative form will automatically include dialogue, narration, and sound effects seamlessly.
Transcript Writing Guide
Decide if you will be using verbatim (every "um", etc.) or clean (without the "um"s, etc.).
Speakers should be identified by name when it is important to to the narrative. Be consistent when identifying speakers.
Note important non-voice audio such as sound effects, foley, environment and music.
Use adjectives to characterize an action as needed (such as: laughing sarcastically vs. laughing knowingly vs. laughing to themselves).
Aim for 99% accuracy in spelling
Add punctuation
Motion Design for a Neurodiverse Population
Guidelines
Live action video, animation and special effects should be paced in way that does not cause negative reactions or stress to persons with cognitive challenges such as autism. Integration of the following design guidelines can help to accomplish this:
Images should be high contrast to ensure sight impaired persons can understand them. Reference the imaging guidelines in this site.
Overall pacing that is easy to follow and has generous pauses between action sequences and scenes that allow for caption or audio description. Consider these scenarios:
A hearing impaired person may be reading captions and watching the video simultaneously. Try doing this yourself to get a feeling for it.
A sight impaired person may be listening to audio-descriptions of the action between dialogue. If you write in visual narrative form this experience will be optimized. If there is a separate audio-description version, then room for that should be built-in to the pacing.
Do not use large rapidly animating images that can cause epileptic seizures (no more than 3 blinks per second... or between 2 and 25 hz). Also, avoid overlaid complimentary colors or stripes that may scintillate. Guidelines from Webaim (external link).
Examples
(external links)
WMU Autism Procedures: Matching simple Pictures: This is an app containing animations designed specifically for young autistic children.
Animation for Mental Health America This is paced well for both reading the (integrated) captions and audio descriptions.
Animated Motion Graphics in the Spectre project: This few seconds may be overwhelming for some and trigger seizures in others.