Audio & Video

[Author: Bill Fischer]



Video Captioning

Caption Writing Guidelines; Overview

Video should have manually written captions (for the hearing impaired). They can be closed (off by default) or open (on by default). When using YouTube, the captions can be edited and re-timed then downloaded for off-line use if desired. These Jean Hanks Lecture Videos are examples of open captioning.

Captioning Guide

  • Decide if you will be using verbatim (every "um", etc.) or clean (without the "um"s, etc.).

  • Speakers should be identified by name when it is important to to the narrative. Be consistent when identifying speakers.

  • Sync the captions to the video.

  • Ensure the captions are visible long enough to read at a moderate pace.

  • Note important non-voice audio such as sound effects, foley, environment and music. These can disappear after a few seconds

  • Use adjectives to characterize an action as needed (such as: laughing sarcastically vs. laughing knowingly vs. laughing to themselves).

  • Use a san-serif font

  • Aim for 99% accuracy in spelling. punctuation, and grammar. No paraphrasing is allowed.

  • Limit one to three lines per caption frame, with no more than 32 characters per line.

  • Place captions on the screen without blocking important content. See 'Design For Captioning' below.

DESIGN FOR CAPTIONING

The image below defines the area in a video that should be free of important information because it may be covered by the captions.

diagram showing caption-safe area for video

Video Tutorial for Captioning in YouTube

YouTube has a built-in tool for captioning that allows editing for the creator and user controls for the the viewer (such as Font size, and color) as required by ADA regulations.

This YouTube Captioning Video Tutorial (external link) by vidIQ. Published on April 25, 2020 will walk you through the process of using YouTube's tools for captioning.

Integrated Captioning

Captions can also be integrated into the animation as part of the design. This is a more universal approach. Most of these, however, don't really leave the text on the screen long enough to read, let alone watch the animated visuals.

Examples:

(all external links)



Audio-Visual Storytelling

Writing Visual Narratives

Writing visual narratives can often negate the need for separate audio descriptions.

Video Writing Example 1 (Standard Form)

(2 versions needed - one with audio description and one without):

    • Narrator says: Sometimes I wonder what my dog is thinking

    • Image shows: Person on their knees looking directly into a dog's eyes from 6 " away

Video Writing Example 2 (Visual Narrative Form)

(1 version needed - because the narrator describes the visual action):

    • Narrator says: Sometimes I stare directly into my dog's eyes, wondering what she is thinking

    • Image shows: Person on their knees looking directly into a dog's eyes from 6 " away

Visual Narrative Examples

(external links)

Affordance

When users are consuming video, they should periodically be supplied orienting devices that reduce cognitive load. This can include:

  • Title cards.

  • Establishing scene shots.

  • A linear story structure.

  • A clear, narrative-driven visual hierarchy within each scene.

Sound Design

This is the practice, used in radio for many years, of adding audio to recordings that reinforce, and sometimes, outright tell a visual story.

  • Scoring can add emotion

  • Environment and Foley can tell "visual" stories

Here are some podcasts that excel at audio-visual storytelling (external links):



Audio Descriptions for Video

When To Use Them

If audio-visual storytelling has not been applied to a video or animation, audio descriptions may be necessary.

Examples:

Audio Description Guidelines

  • If Video should have audio description accompanying video (for the sight impaired) ... defined as narration added to the soundtrack to describe important visual detail that cannot be understood from the main soundtrack alone. The four cornerstones of audio description are “what, where, who and how”. If there is not time between dialogue to fit all of that in... choose the description that most benefits the story-understanding. Think about how much needs to be described in detail versus how much can be left to the audiences imagination... and still tell the story.

  • Audio and video can be paused and stopped. Audio/video should not autoplay.

  • Standard audio descriptions fit the voice over within the existing timing of the video. This can sometimes require difficult choices resulting in the exclusion of some information.

  • Extended audio descriptions pause, or allow the use to pause, the video to allow the inclusion of more detailed information.



Audio Only Transcripts

Transcripts Overview

Audio only media should have an audio text transcription, for the hearing impaired, which includes a written description of the dialogue, narration and pertinent sound effects.

Writing in visual narrative form will automatically include dialogue, narration, and sound effects seamlessly.

Transcript Writing Guide

  • Decide if you will be using verbatim (every "um", etc.) or clean (without the "um"s, etc.).

  • Speakers should be identified by name when it is important to to the narrative. Be consistent when identifying speakers.

  • Note important non-voice audio such as sound effects, foley, environment and music.

  • Use adjectives to characterize an action as needed (such as: laughing sarcastically vs. laughing knowingly vs. laughing to themselves).

  • Aim for 99% accuracy in spelling

  • Add punctuation



Motion Design for a Neurodiverse Population

Guidelines

Live action video, animation and special effects should be paced in way that does not cause negative reactions or stress to persons with cognitive challenges such as autism. Integration of the following design guidelines can help to accomplish this:

  • Images should be high contrast to ensure sight impaired persons can understand them. Reference the imaging guidelines in this site.

  • Overall pacing that is easy to follow and has generous pauses between action sequences and scenes that allow for caption or audio description. Consider these scenarios:

    • A hearing impaired person may be reading captions and watching the video simultaneously. Try doing this yourself to get a feeling for it.

    • A sight impaired person may be listening to audio-descriptions of the action between dialogue. If you write in visual narrative form this experience will be optimized. If there is a separate audio-description version, then room for that should be built-in to the pacing.

  • Do not use large rapidly animating images that can cause epileptic seizures (no more than 3 blinks per second... or between 2 and 25 hz). Also, avoid overlaid complimentary colors or stripes that may scintillate. Guidelines from Webaim (external link).

Examples

(external links)