Firefox OS/1UA Multi-Screen

Introduction

Goal

Our mission is to support "1 UA multi-screen" rendering on FxOS. Try imagine the following scenario, you have a computer set up in duel monitors, launch a Firefox browser on your primary display, launch another Firefox browser(or open a link in new window), and drag that new window to the secondary monitor. Our concept is similar but slightly different, where the baseline of this project is to allow a FxOS device connect and render contents to any non-FxOS display, or device that doesn't support Firefox browser.

The picture shows a FxOS device(at the bottom) has connected to a dummy display(the Acer monitor) with HDMI cord, and to a tablet(at the lower right) with Wifi Display in the same time. Both Marketplace and Dialler are launched from the FxOS device and displayed to the external displays.

Requirements

• Connect with external displays
  • For wireless connection, we have implemented Wifi Display with Miracast protocol. That is, any device capable of being a Miracast Sink, can be connected with a FxOS device.
  • For wired connection, we have implemented the HDMI output. This requires the support of hotplug detection in Hardware Composer.
• Open multiple top-level windows on FxOS
  • Gecko supports opening multiple top-level windows on browser naturally, while the FxOS supports single top-level window only, thus we are going to bring this ability back for FxOS.
• Manage multi-screen in System App

Advantages

• Having multiple top-level windows means that we will have multiple DOM trees and layer trees exist in the system. The secondary window should have no difference regarding to functionality then the original one, which means that we should be able to launch WebApps, web pages, and multimedia on this secondary window.
• With the case of "1 UA multi-screen" rendering, contents of remote screen are rendered by primary device, but in the resolution of its display.
• Open arbitrary numbers of top-level windows and associate each of them to a display device is doable theoretically, but it depends on the support of hardware capability.

Terminology

• Top-level window - A Window object without a parent, currently in FxOS, document loaded into the one and only top-level window is shell.html.
• UA - In short of User Agent. A user agent is any software that retrieves, renders and facilitates end user interaction with Web content. -- From User Agent Accessibility Guidelines (UAAG) 2.0
• Display device/Screen - Refer to devices for presentation in visual, e.g. monitor, TV, projector, tablet, ...etc.
• Primary Display - Refer to the display of FxOS mobile device.
• Secondary/Remote/External/Virtual Display - Refer to displays connected with primary device.
  • To be more specific, HWC has categorized the display types into follows: Primary the default built-in display, External the display connected through HDMI cord, and Virtual as the display connected via Wifi Display or other wireless protocols.
• Wifi Display, Miracast -

Architecture Designs

Overview

An overview of multi-screen on FxOS

This is an overview structure design from top to down.

Gecko

Design structure of gecko's part.

Gaia

Design structure of gaia's part.