The Verdione project

Verdione will develop a platform that allows artists who are distributed around the globe to interact with each other as though they were co-located. Our platform will be applicable in different artistic genres that require auditive and visual  interactions.  Our goal is to make the platform robust enough to be used in rehearsals and also in real performances which include a critical audience.

The experience of users of the platform can only be of high quality and life-like if the perception of remote co-users satisfies enough  senses, if the media types are presented in a well-coordinated and natural manner, and if they are of sufficient quality. Therefore, Verdione will develop technologies for network resilience and dependable communication infrastructures, information management and user interfaces and explore the artistic use of digital technology, especially in music and opera performances.

Novelty

Verdione will explore techniques that can open totally new levels of interaction between people over long distances. So two situations are the rule. In one such as video remote involves a very clear aspect of computer mediation in which users experience distance from communication partners. In the other we find virtual-reality. All try to reach a high level of immersion into virtual worlds and disassociation with their real environment. Verdione pushes forward towards cooperative applications that merge elements in a combination call mixed reality. The ultimate goal is for users to become immersed in a virtual reality that includes their local environment, and where users’ arbitrary actions can be disseminated to their remote communication partners.

Challenges

The World Opera consortium poses several very large challenges. Singers act together on a virtual stage that comprises virtual elements as well as all the real, local stages of each of the participating singers, and the virtual stage is concurrently projected into all of the real stages. The singers will interact as if they were located together on a single stage. At the same time, musicians located at the different operas will play together. A clear challenge to the multimodality of the system arises in the requirements that actors and musicians react to their remote peers. Although no computer keys are pressed by the actors and musicians, the distributed multimedia system that is to be built by Verdione mediates information about position, location, distance, timing and many other details to remote peers.

The media used include a large number of very high-quality audio and video streams that are transmitted in real-time, but may also include event streams for stage control and stored data for inclusion into the virtual stage. Verdione  members will address problems of user interface construction for live media recording and for the presentation of stages that consist of a physically present local stage environment, virtual representations of audiovisual information that are captured at several remote stages and even completely virtual stage elements.

Methodology

To facilitate the communication between stages Verdione will exploit contextual knowledge ranging from screenplay-defined interaction to location tracking that allows the discovery of changing dependencies between media streams to allocate limited network resources in the best way possible. The quality-of-service requirements for streams will be taken into account when communication facilities adapt to changes in context or user requirements. To allow the World Opera to rely on the Verdione platform, the reliability of communication will be increased by improved network resilience mechanisms, and the provided services will be subject to dependability mechanisms. Network simulations and an experimental lab will be used to investigate and evaluate our solutions. We finally aim for an implementation of a demonstration case, a real distributed opera performance – a task that involves both prototype implementations and production-grade system enigneering.