Part 1 – Scripting

2 September 2019

RNIB, UK

 

Over the past year, we have experimented extensively with audio description on 360 content in the Immersive Accessibility Project (ImAc). The aim is finding solutions for enhancing the viewing experience of someone with sight loss watching this content and we have broadly used the following criteria to gather feedback on the quality of experience:

• Understanding
• Enjoyment
• Immersion

As part of our testing, we used and considered different types of content and scenarios: animation – live action, fiction – documentary and on-site delivery for example a museum walkthrough and it’s fair to say that the treatment of audio description needs to be adapted specifically to each format. So, like any other creative production, there is no one right answer. Here are some of the treatments we tried.

Presentation style of audio description on 360 content

We explored the presentation styles of two elements:

• Script and delivery of audio description
• Sound design

Script and delivery of audio description

A 360 display supports multiple assets such as characters, props, space and design etc. that benefit from additional description. Nonetheless the delivery of content is still linear and the time available for adding audio description is limited. So, the question is how to make the audio description effective enough that the viewer is immersed in the content which is one of the aims of the 360 format.

Possibilities:

Script in first person

This is where the audio description is scripted in first person and the main character of the story becomes the describer. The style of writing and delivery is also similar to how the character sounds and speaks.

Example:

“I reversed away from him and he watched me go. I learned to drive with my knees while I played my guitar. There were four of us in the car now, Nat on guitar, Chuck on melodica, Big Sam on dashboard drums.”

Script in second person

This is where the describer is sitting next to the viewer or standing over their shoulder describing the scene the scene. Style of writing and delivery is casual, informal and friendly.

Example:

“She reverses the car. He watches her go. Later she steers with her knees, playing a guitar. There are three friends with her – a girl on guitar, a boy on melodica, while a bulky lad drums on the dashboard.”

Standard audio description

This is the standard audio description style where describer objectively sets out what is happening in the scene, the characters etc.

Example:

“She puts the car into reverse and pulls away from him. He watches her go. Later she steers the car with her knees as she plays her guitar. Three friends are with her, playing guitar, melodica and one drumming on the dashboard.”

Primary audio description with an additional track

One of the trickiest aspects of describing 360 content is predicting what the viewer will do next – look left where one can see the snow-capped peaks of the Swiss Alps or right, towards a meadow filled with daisies.

So, what happens in these situations where the viewer has complete control over what they see and when they see it?

We considered further controls which allowed people to activate descriptions when they changed direction.

In this mode, there are three audio tracks:

• First: original soundtrack of the content
• Second: audio description track
• Third: additional description track which is the extended description

The original soundtrack remains untouched.

The audio description track describes the five key elements of the scene – who, what, why, where and when in order of priority for understanding the storyline as in standard audio description.

The additional audio track or the extended description is used to set the scene and describe elements such as props, characters and their costumes, lighting etc. or event background knowledge which would help people with sight loss visualise the scene. The rationale here being greater familiarity with the scene is linked to greater immersion.

This track can be triggered by the viewer when they hear a ‘beep’ that is used to signal the presence of this additional track.

Audio introduction

In addition to the above, an audio introduction could be useful. Viewer listens to the audio introduction before watching the content which sets the scene. It refers to details that the describer cannot include in the content because of the lack time. These are quite common in the theatre world and are used to introduce to the viewers to the characters, their physical description, their costumes, the set design etc.

The absence of one right answer

During our testing, we’ve discussed various factors that contribute to an immersive experience including how many voices would be considered too many in an immersive environment, use of language, intonation, delivery style and the importance of directionality and placement of the audio description. However, it does seem that the 360 content opens up a whole host of alternatives on how audio description can be presented to the viewer and often a customised approach in which a combination of the scripting styles and sound design would be needed to achieve the ideal experience.

Also, something to consider is the use of HMDs for people with significant sight loss who are unlikely to engage with content visually. Are they necessary? Perhaps, another device for head tracking may be more appropriate but further work is needed to investigate the viability of these alternatives.

Further reading

Read more about how sound affects the experience of 360 for viewers with sight loss.

Link to this article

https://www.imacproject.eu/2019/01/14/audio-description-in-3d-audio/

Tagged as: accessibility, AD