Work Package 1

Deliverable 1.1. Consortium operating procedures

Authors: Jose Miguel Sanjuan (I2CAT)

Summary: ImAc is a project that comes together different actors from the European research ecosystem: universities, private companies, non-profit organizations, and research centres. This deliverable aims to provide a comprehensive document that addresses the internal operation procedures, the roles and responsibilities of each partner, internal protocols for reporting and the contractual obligations as collected in the CA and GA. The main topics included in D1.1 are:

  • Description of the responsibilities of the different roles and bodies as defined in the Consortium Agreement and Grant Agreement.
    · Project reporting procedures and the editorial criterions for addressing the project outputs.
    · Communication procedures, including meeting protocols and voting rules.
    · Contractual obligations in the dissemination actions.

Read Deliverable 1.1. Consortium operating procedures

Deliverable 1.2. Ethical Considerations and Data Management

Author: Pilar Orero (UAB)

Summary: One crucial aspect of research is to ensure that ethical procedures are followed and that the main research outputs and data are available to fellow researchers and society in general. The process to achieve ethical clearance and the documents put in place for ImAc project are detailed in this deliverable. All the information regarding permissions and forms approved by UAB Ethical Committee to be taken into account when performing any user interaction are presented. The document also provides a description of the data management plan, which refers to the provisions taken in the project for data sharing, protection, and exploitation.

Read Deliverable 1.2. Ethical Considerations and Data Management

Work Package 2

Deliverable 2.1. User Centered Design

Author: Pilar Orero, Anna Matamala (UAB)

Summary: This document explains the user-centered approach taken in ImAc. From the beginning, it was decided that users would be involved in the creation of the immersive access services and the tools that would be needed to create and display the services. To that end, different workflows, types of users, and technological components were identified. Based on this different user cases were defined and shared with participants so that they could contribute through focus groups and pre-pilot actions to the definition of access services and tools. This deliverable describes the user-centered process and results. It also presents a description of the focus groups and pre-pilot 1 actions. User needs have been the basis of user requirements on which the project has been built.

Read Deliverable 2.1. User Centered Design

Deliverable 2.2. User Requirements

Author: Sven Glasser (RBB)

Summary: The ImAc project follows a user centered design approach and aims at a continuous involvement of users, whose needs and requirements must be carefully transformed into technical solutions and developments. User scenarios and user needs were initially identified in D2.1. Moreover, the results of pilot activities yielded newly identified user needs or refinements of already known ones, which all had to be formulated as technical requirements in order to provide the basis for the technical specification of the ImAc platform. These ImAc user requirements are presented in this deliverable D2.2.

All requirements, carefully acquired and documented here, demonstrate very clearly, that home users are new to the omnidirectional, 360° video world. They expect to find well known functionalities especially for playback and accessibility services controls. At the same time, user feedback led to requirements pointing to completely new features like functionalities to guide them to the speaker. Professional users asked for new technical possibilities to produce accessible 360° content. Working on omnidirectional media adds a new level of complexity: The addition of access services is no longer purely time-based, but must now include a spatial dimension, which adds a lot more efforts while navigating, authoring and reviewing accessible 360° media. In ImAc we developed a methodology to formalize the process of deriving requirements from these sources and defined syntax to document the requirements including a prioritisation system. Additionally, we defined rules to handle changes arising from the iterative development process used in ImAc.

Read Deliverable 2.2. User Requirements

Deliverable 2.3. Platform Specification

Author: Francesc Mas (CCMA)

Summary: ImAc follows a user centred design, where project developments are driven by real user needs. This is achieved by involving these users in each step of the design and implementation of the ImAc project. A number of professional users and end consumers were consulted, in order to obtain and define the accessibility requirements needed as an integral and intuitive part of any immersive experience. The resulted list of requirements was supplemented by partner’s expertise in the media industry and together led to this document (D2.3) which outlines the key characteristics of the ImAc platform: end-to end settings, familiar environment, lightweight asset management, web-based technologies and ready to use software.

This document  also describes the architecture of the ImAc platform along the content flow from production to the consumer devices based on professional user and end users’ requirements.

Read Deliverable 2.3. Platform Specification

Work Package 3

Deliverable 3.1. Architecture Design

Authors: Chris Hughes (USAL), Peter tho Pesch (IRT), Mario Montagud (i2CAT), Marc Brelot (MSE), Romain Bouqueau (MSE) and Enric Torres (ANG)

Summary: This document is presented in two iterations and describes the technical architecture of the ImAc project. As the first iteration it is a working document providing a blueprint for the technical implementation of the project, and has been updated after the initial prototype implementation and testing phase to reflect changes needed to improve the platform. The technical architecture is directly based on the user requirements established in WP2, specifically the platform specification (D2.3) that identifies the essential system requirements needed to ensure a successful implementation. In turn it feeds directly into the Integration and Testing Report (D3.6), which is designed to evaluate the success of each phase of implementation. Chapter 1 provides an overview of this document, describes the objectives and scope of the technical architecture, and details how it fits into the larger ImAc project. Chapter 2 describes the Goals and Constraints of the system architecture. These are the key requirements, which have a significant bearing on the architecture and they link directly to the end user requirements defined in the requirements table in D2.2. Satisfying these needs is essential to developing a successful framework for the ImAc project. Chapter 3 provides an architectural representation of the project and establishes the architecture in terms of usage scenarios, logical and process views and a deployment structure. Chapter 4 describes the existing resources and libraries, which will be used within the implementation of the project, where they are used within the architecture, and how they will be integrated with. Chapter 5 discusses the size and performance requirements of the implementation. This details the key sizing and timing requirements for the platform to be used successfully, as stipulated in the Platform Specification, which will provide the basis for the testing strategy. Chapter 6 concludes the document with a summary of the technical architecture.

Read Deliverable 3.1. Architecture Design

Deliverable 3.2. Accessibility Content Manager

Authors: Enric Torres Feixas, Kimiasadat Mirehbar (Anglatècnic)

Summary: This deliverable is meant to be a document accompanying ACM (Accessibility Content Manager) software tool and acts as a guide to it submitted altogether.

This software is dedicated to managing the workflow for production and cataloguing of access service contents for immersive media in a secured and controlled manner. Basically the management includes the assignation of access services to producers, transmission of Low Quality 360º videos to producers, reception of completed access service files from producers and their verification, and finally cataloguing the verified files and delivering the associated contents when required for broadcasting or publishing. To carry out the complete workflow, the software also performs background tasks such as communication with other systems and automatic processes which are also described in this delivery.

The document mainly guides the professional reader to gain a knowledge on the issue by addressing the progress made in development of ACM, objectives and stakeholders, architecture, user manuals and HOW TOs and background processes.

Read Deliverable 3.2 Accessibility Content Manager

Deliverable 3.3. Content Packaging and Distribution

Authors: Marc Brelot, Romain Bouqueau, Rodolphe Fouquet (Motion Spell)

Summary: The ImAc project Task T3.3 “Content packaging and distribution” defines how the media contents generated by the Production tools, described in D4.1[3], D4.2[4] and D4.3[5], are prepared and managed through the Accessible Content Manager (ACM), described in D3.2, and then packaged and delivered to the player, described in D3.5 [2]. This deliverable focuses on the software developed in that task, by describing the global architecture, the different modules and the workflow. This document aims at exposing first what has been currently developed for the first phase of the project (mostly linked with the pilot 1) and then presents the planed evolutions of architecture and modules that would fulfil the final needs of pilot 2. In parallel, the specification and implementation of the different modules strive to rely as much as possible on standards, and to provide generic APIs allowing to setup a full production chain open to scalability and extension.

Read Deliverable 3.3 Content Packaging and Distribution

Deliverable 3.4. Accessibility Interface

Authors: Ronald Mies (IRT), Peter tho Pesch (IRT), Stefan Pöschel (IRT), Doreen Ritter (RBB), Zora Schärer (RBB), Mario Montagud (i2CAT), Jordi Mata Ferrate (CCMA), Francesc Mas Peinado (CCMA)

Summary: The user interface (UI) of a media player / app plays a key role in providing guidance to the end user, and in giving orientation to control the media content being consumed. It should ease access to the media service and also be accessible itself. The UI specifically also should allow easy access to the access services (AS), e.g. by making it easy to activate AS and to change appropriate settings. For the user to be aware of AS being offered and any applicable options, the UI should be fed with appropriate signalling attributes / metadata describing the AS, which is generated at the media provision side.

This document addresses the topic “access to access services” by looking at the UI itself as well as the signalling of AS. It contains the experience from the ImAc consortium regarding UI / user guidance that has been collected so far and describes best practices for the UI as well as for the signalling of AS in the content stream and regarding handling redundant, potentially inconsistent signalling information from different abstraction levels in the UI.

Through a structured analysis of the UIs of State-of-the-Art media players, general insights and some recommendations have been provided to support users accessing AS. The current implementation of the UI giving access to the AS in the ImAc portal and player (implemented in T3.5) reflects the status after feedback from T3.4 as well as from user tests in pilot phase 1 (see D5.4 “Pilot evaluation report”). This will be used for further user tests (specifically pilot phase 2), see section 3.

User guidance on access to AS could be eased by harmonising the UI design. This is not an isolated issue for immersive media but rather a general issue and clearly goes beyond ImAc: this requires cooperation between media (portal) providers and implementors.

Key findings:

  • Hardly any media player currently provides access to the full set of AS (subtitles, audio description, sign language); VR players do not focus on AS at all (section 2.10, page 99).
  • The documented media players that do support access to AS, allow a quick access to AS and their settings (one “step” of action via the UI). Having said that, the UIs have very diverse designs, and the way guidance is given to control the AS varies widely.
  • There is no harmonised convention for icons / representations used for AS. ImAc decided to use a set of icons proposed by Danish Radio to have a universal set to represent AS for all ImAc users in all countries (section 2.10, page 100).
  • A multitude of specifications and standards describing signalling and metadata for subtitles is available in the market, both for broadcast as well as online streaming media distribution. For application in 360o video / XR environment, additional metadata is required. An overview of the required ImAc extensions, comparing current specifications with required ImAc features is given (section 5.1, page 125).
  • The provision of redundant AS signalling on various layers simultaneously can cause conflicts in the UI. As potentially various specifications / standards are involved, an additional harmonisation of implementation choices would be desirable. Design choices to handle these conflicts are described for the case of language settings (section 6.2) and subtitle style and positioning attributes (section 6.3).

The topic “access to access services” clearly requires additional research, specifically also regarding the realisation of UIs (whether for webplayers on PC/tablets, apps running on TV or other devices). Easy access solutions also for TV application (e.g. a “one button” on a remote control) would be ideal but need a support from involved market players.

Read Deliverable 3.4. Accessibility Interface

Deliverable 3.5. Player

Authors: Mario Montagud (i2CAT)

Summary: The ImAc project targets the specification and implementation of an end-to-end platform, comprised of different parts where production, editing, management, preparation, delivery and consumption of contents take place. A key component of the end-to-end platform is the ImAc player, as it is the interface through which end-users will consume the available immersive and accessibility content in an interactive and personalized manner.

This deliverable firstly provides an overview of the identified requirements and design criteria that have mostly determined the development of the player, its components and its features. Then, it describes the integration of the ImAc player within the end-to-end platform. The deliverable additionally explains how to access the source code as well as how to install and configure the player. Finally, the deliverable describes the different screens, User Interfaces (UIs), menus and interaction modalities to control the player, enabling a personalized presentation of immersive and accessibility content.

Read Deliverable 3.5-Player

Deliverable 3.6. Integration and Testing Report

Authors: Chris Hughes (USAL), Peter tho Pesch (IRT), Mario Montagud (i2CAT), Marc Brelot (MSE), Romain Bouqueau (MSE) and Enric Torres (ANG)

Summary: This document is published in two iterations to match the development cycle of the ImAc project. In this document we describe the integration and testing plan for the ImAc project. We describe a methodology for integration and system testing based on standard software engineering aproaches. The system components are defined in the Technical Architecture (D3.1) and this document describes how each of these components are integrated as well as the current status of this development. This document also defines a testing strategy based on the user requirements gathered (D2.3). Chapter 1 provides an overview of this document, describes the objectives and scope of the integration and testing report, and details how it fits into the larger ImAc project. Chapter 2 describes our approach to testing, discuses the theory of system integration testing and provides the methodology that we employ for testing the ImAc platform. Chapter 3 describes specific Integration points within the ImAc project the the activities that they relate to as well as an Integration and Testing plan which identifys current constraints within the system and the strategy for testing these points. Chapter 4 revisits the key size and performance requirements defined in the system architecture and eveluates the success of the ImAc system. Chapter 5 provides the results of acceptance testing of the ImAc platform in order to meet the proove that the system meets the user requirements defined in D2.3. Chapter 6 concludes the document with a summary of the current status of the software and provides a pathway for testing in the second iteration.

Read Deliverable 3.6. Integration and Testing Report

Work Package 4

Deliverable 4.1. Subtitle Production Tools

Author: Peter tho Pesch (IRT)

Summary This document describes the subtitles production tools developed in T4.1 and the accompanying work. The software tools that have been developed, are the actual deliverables – this document describes their concepts and functionalities.
In T4.1, the project partners explored how professional users can create and edit subtitle files for 360° videos. Additionally, T4.1 contributed to the design and concept for presentation manners of subtitles in 360° videos. Besides implementation of editor software, prototypes have been realized to run internal tests before the selected subtitle features have been put through end user tests.
The requirements for the work done in T4.1 derived mainly from the user requirements that have been discovered in T2.3 – for both, end users and professional editors. But in addition, researches from other parties have been studied. Their results were introduced into some of the subtitle presentation concepts that have been explored.

Read Deliverable 4.1. Subtitle Production Tools

Deliverable 4.2. Audio Production Tools

Authors: Theresa Liebl (IRT), Peter tho Pesch (IRT)

Summary: This deliverable documents the audio production tools developed in T4.2 and the accompanying work. The software tools that have been developed, are the actual deliverables – this document describes their concepts and functionalities. The tools that have been investigated and developed in the project (Web AD editor, cloud renderer, object-based audio editor) are described put into the context of the overall ImAc audio production workflow.
In T4.2 of the ImAc project, the consortium is looking for an environment in which professional users would be able to create and edit audio description files for 360° videos in an interactive manner.
The requirements for this environment were extracted from the work done in T2.3 and have been gradually completed during the project (see also ImAc deliverable D2.3 [5]).

Read Deliverable 4.2. Audio Production Tools

Deliverable 4.3. Sign Language Editor

Author: Kimiasadat Mirehbar (Anglatecnic)

Summary: This document is meant to be a comprehensive guide through the professional tools that are dedicated to production and edition of Sign Language access services for 360° videos in an interactive and user-friendly manner destined to professional users.
Professional users are defined as broadcasters and certified sign interpreters who can be either external entities or freelancers contracted by the broadcaster or an access service provision department at broadcasters facilities and this tool helps them edit sign language files for immersive media type.
The deliverable mainly reports the progress made in T4.3 (Sign language editor) of ImAc up to the date of submission and addresses issues such as structure, workflow, user manual and related activities in ImAc workframe.

Read Deliverable 4.3. Sign Language Editor

Deliverable 4.4. Report on new accessibility formats

Authors: Peter tho Pesch (IRT)

Summary In ImAc some new features have been introduced which require metadata in the delivered streams that are not covered by current standards. It is relatively simple to realize such new features in a closed environment like we have initially conducted within the ImAc project. However, one of the projects key objectives is to work towards standardization, and thus interoperable and standard-compliant solutions for providing access services for 360° media applications need to be specified and adopted.

This document is intended to help early adaptors who want to introduce similar services to their portfolio with standard-compliant solutions. We have carefully investigated the needs for integrating additional, i.e. new data, into the content stream(s). Extensions were only made if it wasn’t possible to use existing standards.

After evaluating existing standards (mainly MPEG DASH and TTML) and comparing their possibilities with the identified ImAc requirements, we found that the foundations for the different access services are quite diverse. While for the subtitle service, we mostly added 360° relevant data, we needed to find a more extensive solution for the signer service. These are the key findings by each access service:

  • Subtitles: Using the IMSC format for subtitles, additional data can be carried within the subtitle format. The adaptation of the subtitle service to a 360° environment requires reconsidering the positioning strategies of subtitles (refer to section 2.4.2) as well as the reference to the spatial position of the related speaker (refer to section 2.4.3). Only one exception: Signalling a subtitle stream as easy-to-read should happen at the MPEG DASH layer, but is currently not specified.
  • Audio Description and spoken subtitles: The ImAc specific extensions regarding audio description and spoken subtitles are processed and pre-rendered during the production process on the server side. In the distributed stream, relevant parameters of the various audio streams need to be signalled in order to enable the player application to select the desired stream. We proposed dedicated attributes within the DASH format to identify the different ImAc variations for these services (refer to section 3.3.2 and 4.1). An additional problem is the absence of any standardized solution for identifying an audio stream as Ambisonics (a coding for spatial audio). This is a requirement that we believe others will have as well and thus we have filed a corresponding proposal to MPEG (refer to section 3.3.1).
  • Sign Language: Realizing the sign language service was more challenging due to the number of additional information we need and the absence of suitable locations for such data in the media stream. We decided to create a sidecar (accompanying) metadata file carrying all relevant information timed to the media timeline. Embedding the metadata file in the MPEG DASH manifest is described in section 5.3.1. The separate data fields we introduced are described in sections 5.3.2 – 5.3.5.

Read Deliverable 4.4. Report on new accessibility formats

Deliverable 4.5. Accessibility Service Tools Report

Authors: Kimiasadat Mirehbar, Enric Torres Feixas (Anglatècnic)

Summary: This deliverable acts as a comprehensive report destined to description of the progress made in this work package (WP4 – Accessibility Service Tools), especially the final developments in the matter of access services production tools up to the moment of their submission.
The main objective is to present all the achievements in a concrete manner in order to make it easy for the reader to follow up the ImAc WP4 progress.
The objective of ImAc WP4 is to develop the technological basis required for access service production. The developed tools have derived their specifications from user requirements defined in previous work packages and introduce new results in the field of access service production in immersive media.
In first iteration the relevant research has been conducted and prototyped version of tools have been developed. In the second iteration the tools have migrated to an exploitation version of themselves while fulfilling the user requirements.
The deliverable addresses the developed tools, improvements in their iterations, use-cases, user requirements achievement, aimed stakeholders and conclusion.

Read Deliverable 4.5. Accessibility Service Tools Report

Work Package 5

Deliverable 5.1. Pilot operation Plan – First and Second Phase

Author: Francesc Mas (CCMA), Àngel Quero (CCMA)

Summary: This document defines the pilot operation plan describing the strategy for the operational phase of the platform, tools and access services developed in the Immersive Accessibility (ImAc) project. It defines which service components need to be integrated for their evaluation through a set of pilot actions. For this purpose, an execution plan is produced including the details of the different trials in Spain, Germany and England (T5.3, T5.4 and T5.5). According to T5.2, immersive media is produced to feed the pilots and enriched with accessibility content using the editing tools developed in the project. Professional users will validate these tools by means of specific pilots.

CCMA, RBB and RNIB participate in the definition of the pilot scenarios, the pilot objectives, the needed evaluation methodology in conjunction with UAB, and in the definition of an integrated roadmap for setting up the needed pilot execution and evaluation infrastructure, including both closed and open pilots.

Read Deliverable 5.1. Pilot operation Plan – First Phase

Deliverable 5.2. Pilot evaluation methodology and plan

Authors: Anna Matamala, Pilar Orero (UAB)

Summary: This document presents the evaluation methodology for the different pilot actions in the project: the criteria for defining tests, the procedure, and the evaluation plan. Two types of users and elements are identified: access services addressed to home users, and tools addressed to professional users. Regarding access services, a methodology for testing player interaction and the different presentation modes along the project is suggested. Regarding professional, a methodology for testing the tools in different project stages is proposed. The methodology is based on the concepts of usability, presence, and preferences.

Read Deliverable 5.2. Pilot evaluation methodology and plan

Deliverable 5.3. Pilot Content

Author: Zora Schärer Kalkandjiev (RBB)

Summary: This deliverable describes the content used for demo and pilot actions. As a first step, guidelines for the production of the 360° video content and the accessibility content were developed.

Regarding the 360°-video content (see section 2.1) we identified technical specifications based on the specifications on D3.1 Architecture and Design and derived content- and production related requirements from the pilot evaluation plan specified in D5.2 and the user requirements in D2.2.

Regarding the accessibility content, there are again technical specifications resulting from D3.1 as well as editorial properties resulting from the user requirements presented in D2.2.

A table lists the content data sets.

Read Deliverable 5.3. Pilot Content

Deliverable 5.4 – Pilot evaluation report

Authors: Anna Matamala (UAB)

Summary: This document presents and analyses the results of pilot actions in the ImAc project. It considers both pilot actions with professional users, who have assessed the tools, and with home users, who have assessed the interface and different presentation modes..

Read Deliverable 5.4 – Pilot evaluation report

Work Package 6

Deliverable 6.1 Dissemination and Standardisation Strategy

Authors:   Sonali Rai (RNIB), Ronald Mies (IRT)

Summary: Deliverable 6.1 was set up with the purpose of setting out the communication and dissemination strategy of the Immersive Accessibility Project (ImAc) with a list of key performance indicators that are used to track developments. This deliverable also reports on the standardisation activities planned within the project.

There are two iterations:

  • Report 1 submitted in Month 9 (January 2018) presented the plan for activities between M1 – 17
  • Report 2 scheduled for Month 18 (January 2018) presented the plan for activities between M18- 30

Given the diverse nature of the consortium which includes representatives from academia, industrial development, a user organisation and broadcasters, this communication plan has ensured the consistency of messaging from the project even though the channels and the language used have been tailored to the different stakeholder groups securing continued interest through course of the project.  Furthermore, these reports also provided detailed plans of the dissemination activities undertaken to inform the different stakeholder groups including standardisation experts, broadcasters, industrial stakeholders and third sector organisations.

Overall, the project has had an extraordinary level of interest from stakeholder groups due to the nature of developments in the project which were supported by an effective communication and dissemination plan.

Read Deliverable 6.1 Dissemination and Standardisation Strategy

Deliverable 6.2. Operational public website with logo

Authors: Pilar Orero (UAB)

Summary: This deliverable describes two key dissemination elements in ImAc: the logo and the website.

Both the web and the logo were designed to fulfil accessibility requirements.

The document describes how these requirements were met. It presents the process of selection of a suitable logo and it also describes how the website has been structured.

Read Deliverable 6.2. Operational public website with logo

Deliverable 6.4. Short Movie

Authors: Pilar Orero (UAB)

Summary: This deliverable describes the two videos produced in ImAc early stage: a video to inform about the project and its objectives, providing information about the different accessibility services. On the other hand, a video to inform on the ethical requirements for user testing.

Read Deliverable 6.4. Short Movie