|
|
| (13 intermediate revisions by 6 users not shown) |
| Line 26: |
Line 26: |
| The Virtual Keyboard/IME API supports the following features: | | The Virtual Keyboard/IME API supports the following features: |
|
| |
|
| * Notifies the VKB app when the focus text field was changing in other | | * Notifies the VKB app when the focus text field was changing in other apps |
| apps | |
| * Allow user to manual hide the keyboard. Check {{bug|737110}}. | | * Allow user to manual hide the keyboard. Check {{bug|737110}}. |
| * The VKB app should be responsive to properties and the state of the input field (more than HTML5 input type, including current content, cursor position, x-inputmode {{bug|796544}}). | | * The VKB app should be responsive to properties and the state of the input field (more than HTML5 input type, including current content, cursor position, x-inputmode {{bug|796544}}). |
| Line 43: |
Line 42: |
| * [Line 4] a "role" field with value "keyboard" declares it's an IME app. Homescreen app will ignore some role types when displaying app icons, and "keyboard" is one of them. (see {{bug|892397}}) | | * [Line 4] a "role" field with value "keyboard" declares it's an IME app. Homescreen app will ignore some role types when displaying app icons, and "keyboard" is one of them. (see {{bug|892397}}) |
| * [Line 6-8] a "permissions" field that requests "keyboard" permission. All IME apps need this permission for sending input keys and updating the value of a input field. | | * [Line 6-8] a "permissions" field that requests "keyboard" permission. All IME apps need this permission for sending input keys and updating the value of a input field. |
| * [Line 9-30] a "entry_points" field specifies supported layouts. Each layout is described in a key-value pair, where the key represents the layout name (will be shown up on Settings app with the app name), and the value describes the detailed information of the layout, including launch path of the layout and supported input types. (See [[#Layout Matching Algorithm]]) | | * [Line 9-30] a "inputs" field specifies supported layouts. Each layout is described in a key-value pair, where the key represents the layout name (will be shown up on Settings app with the app name), and the value describes the detailed information of the layout, including launch path of the layout and supported input types. (See [[#Layout Matching Algorithm]]) |
| ** The allowed value in "types" field is a subset of [http://www.whatwg.org/specs/web-apps/current-work/multipage/the-input-element.html#attr-input-type type attribute of input element]: text, search, tel, number, url, email. Other types will be ignored by FxOS Gaia in the initial version because at this point UI for <select> and <input type=date> (called "value selectors") are not open for 3rd-party implementation. | | ** The allowed value in "types" field is a subset of [http://www.whatwg.org/specs/web-apps/current-work/multipage/the-input-element.html#attr-input-type type attribute of input element]: text, search, tel, number, url, email. Other types will be ignored by FxOS Gaia in the initial version because at this point UI for <select> and <input type=date> (called "value selectors") are not open for 3rd-party implementation. |
|
| |
|
| === IME App Manifest Example ===
| | { |
| | | "name": "3rd-party Keyboard", |
| 1 {
| | "description": "3rd-party Keyboard", |
| 2 "name": "MyKeyboard",
| | "type": "privilege", |
| 3 "description": "A 3rd Party Keyboard",
| | "role": "input", |
| 4 "role": "keyboard",
| | "launch_path": "/settings.html", |
| 5 "launch_path": "/settings.html",
| | "developer": { |
| 6 "permissions": {
| | "name": "developer's name", |
| 7 "keyboard": {}
| | "url": "https://keyboard.example.com" |
| 8 },
| | }, |
| 9 "entry_points": {
| | "permissions": { |
| 10 "English": {
| | "input": {} |
| 11 "launch_path": "/index.html#en",
| | }, |
| 12 "description": "English layout",
| | "inputs": { |
| 13 "types": ["url", "number"],
| | "en": { |
| "locales": {
| | "launch_path": "/index.html#en", |
| "ar":{
| | "name": "English", |
| "entry_points": {
| | "description": "English layout", |
| "English":{
| | "types": ["url", "text"], |
| "name":"الإنجليزية",
| | "locales": { |
| "description": "Gaia هاتف"
| | "en-US": { |
| }
| | "name": "English", |
| }
| | "description": "English layout" |
| },
| | }, |
| "en-US":{
| | "zh-TW": { |
| "entry_points": {
| | "name": "英文", |
| "English":{
| | "description": "英文鍵盤" |
| "name": "English",
| | } |
| "description": "English layout"
| | } |
| }
| | }, |
| }
| | "en-Dvorak": { |
| },
| | "launch_path": "/index.html#en-Dvorak", |
| "fr":{
| | "name": "English (Dvorak)", |
| "entry_points": {
| | "description": "Dvorak layout", |
| "English":{
| | "types": ["url", "text"] |
| "name": "Anglais",
| | }, |
| "description": "Anglais layout"
| | "es": { |
| }
| | "launch_path": "/index.html#es", |
| }
| | "name": "Spanish", |
| }
| | "description": "Spanish layout", |
| },
| | "types": ["url", "text"] |
| "default_locale": "en-US"
| | }, |
| 14 },
| | "pt-BR": { |
| 15 "English (Dvorak)": {
| | "launch_path": "/index.html#pt-BR", |
| 16 "launch_path": "/index.html#en-Dvorak",
| | "name": "Portuguese Brazilian", |
| 17 "description": "Dvorak layout",
| | "description": "Portuguese Brazilian layout", |
| 18 "types": ["text", "url", "number"]
| | "types": ["url", "text"] |
| "locales": {
| | }, |
| "ar":{
| | "pl": { |
| "entry_points": {
| | "launch_path": "/index.html#pl", |
| "English":{
| | "name": "Polish", |
| "name":"الإنجليزية",
| | "description": "Polish layout", |
| "description": "Gaia هاتف"
| | "types": ["url", "text"] |
| }
| | }, |
| }
| | "ca": { |
| },
| | "launch_path": "/index.html#ca", |
| "en-US":{
| | "name": "Catalan", |
| "entry_points": {
| | "description": "Catalan layout", |
| "English":{
| | "types": ["url", "text"] |
| "name": "English",
| | }, |
| "description": "English layout"
| | "cz": { |
| }
| | "launch_path": "/index.html#cz", |
| }
| | "name": "Czech", |
| },
| | "description": "Czech layout", |
| "fr":{
| | "types": ["url", "text"] |
| "entry_points": {
| | }, |
| "English":{
| | "fr": { |
| "name": "Anglais",
| | "launch_path": "/index.html#fr", |
| "description": "Anglais layout"
| | "name": "French", |
| }
| | "description": "French layout", |
| }
| | "types": ["url", "text"] |
| }
| | }, |
| },
| | "de": { |
| "default_locale": "en-US"
| | "launch_path": "/index.html#de", |
| 19 },
| | "name": "German", |
| 20 "Spanish": {
| | "description": "German layout", |
| 21 "launch_path": "/index.html#es",
| | "types": ["url", "text"] |
| 22 "description": "Spanish layout",
| | }, |
| 23 "types": ["text", "number"]
| | "nb": { |
| "locales": {
| | "launch_path": "/index.html#nb", |
| "ar":{
| | "name": "Norwegian Bokmal", |
| "entry_points": {
| | "description": "Norwegian Bokmal layout", |
| "Spanish":{
| | "types": ["url", "text"] |
| "name":"الإنجليزية",
| | }, |
| "description": "Gaia هاتف"
| | "sk": { |
| }
| | "launch_path": "/index.html#sk", |
| }
| | "name": "Slovak", |
| },
| | "description": "Slovak layout", |
| "en-US":{
| | "types": ["url", "text"] |
| "entry_points": {
| | }, |
| "Spanish":{
| | "tr-Q": { |
| "name": "English",
| | "launch_path": "/index.html#tr-Q", |
| "description": "English layout"
| | "name": "Turkish Q", |
| }
| | "description": "Turkish Q layout", |
| }
| | "types": ["url", "text"] |
| },
| | }, |
| "fr":{
| | "tr-F": { |
| "entry_points": {
| | "launch_path": "/index.html#tr-F", |
| "Spanish":{
| | "name": "Turkish F", |
| "name": "Anglais",
| | "description": "Turkish F layout", |
| "description": "Anglais layout"
| | "types": ["url", "text"] |
| }
| | }, |
| }
| | "ro": { |
| }
| | "launch_path": "/index.html#ro", |
| },
| | "name": "Romanian", |
| "default_locale": "en-US"
| | "description": "Romanian layout", |
| 24 },
| | "types": ["url", "text"] |
| 25 "number": {
| | }, |
| 26 "launch_path": "/index.html#numberLayout",
| | "ru": { |
| 27 "description": "Number layout",
| | "launch_path": "/index.html#ru", |
| 28 "types": ["number"]
| | "name": "Russian", |
| "locales": {
| | "description": "Russian layout", |
| "ar":{
| | "types": ["url", "text"] |
| "entry_points": {
| | }, |
| "number":{
| | "sr-Cyrl": { |
| "name":"الإنجليزية",
| | "launch_path": "/index.html#sr-Cyrl", |
| "description": "Gaia هاتف"
| | "name": "Serbian (Cyrillic)", |
| }
| | "description": "Serbian (Cyrillic) layout", |
| }
| | "types": ["url", "text"] |
| },
| | }, |
| "en-US":{
| | "sr-Latn": { |
| "entry_points": {
| | "launch_path": "/index.html#sr-Latn", |
| "number":{
| | "name": "Serbian (Latin)", |
| "name": "English",
| | "description": "Serbian (Latin) layout", |
| "description": "English layout"
| | "types": ["url", "text"] |
| }
| | }, |
| }
| | "ar": { |
| },
| | "launch_path": "/index.html#ar", |
| "fr":{
| | "name": "Arabic", |
| "entry_points": {
| | "description": "Arabic layout", |
| "number":{
| | "types": ["url", "text"] |
| "name": "Anglais",
| | }, |
| "description": "Anglais layout"
| | "he": { |
| }
| | "launch_path": "/index.html#he", |
| }
| | "name": "Hebrew", |
| }
| | "description": "Hebrew layout", |
| },
| | "types": ["url", "text"] |
| "default_locale": "en-US"
| | }, |
| 29 }
| | "hu": { |
| 30 }
| | "launch_path": "/index.html#hu", |
| 31 }
| | "name": "Hungarian", |
| | "description": "Hungarian layout", |
| | "types": ["url", "text"] |
| | }, |
| | "el": { |
| | "launch_path": "/index.html#el", |
| | "name": "Greek", |
| | "description": "Greek layout", |
| | "types": ["url", "text"] |
| | }, |
| | "zh-Hans-Pinyin": { |
| | "launch_path": "/index.html#zh-Hans-Pinyin", |
| | "name": "Pinyin", |
| | "description": "Pinyin", |
| | "types": ["url", "text"] |
| | }, |
| | "number": { |
| | "launch_path": "/index.html#numberLayout", |
| | "name": "Number", |
| | "description": "Number layout", |
| | "types": ["number"] |
| | } |
| | }, |
| | "locales": { |
| | "en-US": { |
| | "name": "3rd-party Keyboard", |
| | "description": "3rd-party Keyboard" |
| | }, |
| | "zh-TW": { |
| | "name": "第三方鍵盤", |
| | "description": "第三方鍵盤" |
| | } |
| | }, |
| | "default_locale": "en-US" |
| | } |
|
| |
|
| === Layout Matching Algorithm === | | === Layout Matching Algorithm === |
| Line 201: |
Line 234: |
| == Proposed API == | | == Proposed API == |
|
| |
|
| The input method API is available to web content who intend to implement an input method, or "input source", or "virtual keyboard". | | '''The API has made available to privileged apps. See the [http://dxr.mozilla.org/mozilla-central/source/dom/webidl/InputMethod.webidl WebIDL] for the current interface.''' |
| | |
| partial interface Navigator {
| |
| readonly attribute InputMethod inputMethod;
| |
| };
| |
| | |
| interface InputMethod: EventTarget {
| |
| // Input Method Manager contain a few global methods expose to apps
| |
| readonly attribute InputMethodManager mgmt;
| |
|
| |
| // Fired when the input context changes, include changes from and to null.
| |
| // The new InputContext instance will be available in the event object under |inputcontext| property.
| |
| // When it changes to null it means the app (the user of this API) no longer has the control of the original focused input field.
| |
| // Note that if the app saves the original context, it might get void; implementation decides when to void the input context.
| |
| attribute EventHandler oninputcontextchange;
| |
|
| |
| // An "input context" is mapped to a text field that the app is allow to mutate.
| |
| // this attribute should be null when there is no text field currently focused.
| |
| readonly attribute InputContext? inputcontext;
| |
| };
| |
| | |
| // Manages the list of IMEs, enables/disables IME and switches to an IME.
| |
| interface InputMethodManager {
| |
| // Ask the OS to show a list of available IMEs for users to switch from.
| |
| // OS should ignore this request if the app is currently not the active one.
| |
| void showAll();
| |
|
| |
| // Ask the OS to switch away from the current active Keyboard app.
| |
| // OS should ignore this request if the app is currently not the active one.
| |
| void next();
| |
|
| |
| // To know if the OS supports IME switching or not.
| |
| // Use case: let the keyboard app knows if it is necessary to show the "IME switching"
| |
| // (globe) button. We have a use case that when there is only one IME enabled, we
| |
| // should not show the globe icon.
| |
| boolean supportsSwitching();
| |
|
| |
| // Ask the OS to hide the current active Keyboard app. (was: |removeFocus()|)
| |
| // OS should ignore this request if the app is currently not the active one.
| |
| // The OS will void the current input context (if it exists).
| |
| // This method belong to |mgmt| because we would like to allow Keyboard to access to
| |
| // this method w/o a input context.
| |
| void hide();
| |
| };
| |
| | |
| // The input context, which consists of attributes and information of current input field.
| |
| // It also hosts the methods available to the keyboard app to mutate the input field represented.
| |
| // An "input context" gets void when the app is no longer allowed to interact with the text field,
| |
| // e.g., the text field does no longer exist, the app is being switched to background, and etc.
| |
| // [JJ] I doubt whether we should have 'name', 'type', etc. here. In the manifest we should
| |
| // have entry points where the keyboard specifies which view to load when going into a
| |
| // certain context. Requiring to do this manually will give extra work.
| |
| // The system should guarantee that the right view is rendered based on entry_points in
| |
| // in manifest (e.g. navigate keyboard to #text/en, or something, based on manifest.
| |
| // [Tim] I don't think they are exclusive. A keyboard app might choose to load the same page with the same hash
| |
| // for different types but only to deal with the |type| or |inputmode| difference later.
| |
| // [JS] I agree that exposing type etc is a good idea. It's quite likely that the same keyboard
| |
| // app will want to handle multiple different keyboards, for example both for latin text as well as
| |
| // numeric keyboard.
| |
| // But I agree that also enabling the keyboard to declare in the manifest which types it supports
| |
| // is a good idea.
| |
| interface InputContext: EventTarget {
| |
| // The tag name of input field, which is enum of "input", "textarea", or "contenteditable"
| |
| readonly DOMString type;
| |
|
| |
| // The type of the input field, which is enum of text, number, password, url, search, email, and so on.
| |
| // See http://www.whatwg.org/specs/web-apps/current-work/multipage/states-of-the-type-attribute.html#states-of-the-type-attribute
| |
| readonly DOMString inputType;
| |
|
| |
| /*
| |
| * The inputmode string, representing the input mode.
| |
| * See http://www.whatwg.org/specs/web-apps/current-work/multipage/association-of-controls-and-forms.html#input-modalities:-the-inputmode-attribute
| |
| */
| |
| readonly DOMString inputMode;
| |
|
| |
| /*
| |
| * The primary language for the input field.
| |
| * It is the value of HTMLElement.lang.
| |
| * See http://www.whatwg.org/specs/web-apps/current-work/multipage/elements.html#htmlelement
| |
| */
| |
| readonly DOMString lang;
| |
|
| |
| /*
| |
| * Get specified range of text content from the input field.
| |
| * @param offset The offset from the cursor position where the range starts. Defaults to 0.
| |
| * @param length The range length. Defaults to the end of the text.
| |
| */
| |
| Promise<DOMString> getText([optional] offset, [optional] length);
| |
|
| |
| // The start and stop position of the selection.
| |
| readonly attribute long selectionStart;
| |
| readonly attribute long selectionEnd;
| |
|
| |
| // The text before and after the begining of the selected text.
| |
| readonly attribute DOMString textBeforeCursor;
| |
| readonly attribute DOMString textAfterCursor;
| |
|
| |
|
| /*
| | [https://wiki.mozilla.org/index.php?title=WebAPI/KeboardIME&oldid=1029753#Proposed_API History of this section] |
| * Set the selection range of the the editable text.
| |
| * Note: This method cannot be used to move the cursor during composition. Calling this
| |
| * method will cancel composition.
| |
| * @param start The beginning of the selected text.
| |
| * @param length The length of the selected text.
| |
| *
| |
| * Note that the start position should be less or equal to the end position.
| |
| * To move the cursor, set the start and end position to the same value.
| |
| *
| |
| * [JJ] I think that this method should return the same info as the selectionchange event
| |
| * rather than a boolean.
| |
| * [yxl] I don't think so. We could get selection range info by checking the attributes of
| |
| * selectionStart and selectionEnd.
| |
| */
| |
| <strike>Promise<boolean> setSelectionRange(long start, long length);</strike>
| |
| Promise<void> setSelectionRange(long start, long length);
| |
|
| |
| /* User moves the cursor, or changes the selection with other means. If the text around
| |
| * cursor has changed, but the cursor has not been moved, the IME won't get notification.
| |
| */
| |
| attribute EventHandler onselectionchange;
| |
|
| |
| /*
| |
| * Commit text to current input field and replace text around cursor position. It will clear the current composition.
| |
| *
| |
| * @param text The string to be replaced with.
| |
| * @param offset The offset from the cursor position where replacing starts. Defaults to 0.
| |
| * @param length The length of text to replace. Defaults to 0.
| |
| */
| |
| <strike>Promise<boolean> replaceSurroundingText(DOMString text, [optional] long offset, [optional] long length);</strike>
| |
| Promise<void> replaceSurroundingText(DOMString text, [optional] long offset, [optional] long length);
| |
|
| |
| /*
| |
| *
| |
| * Delete text around the cursor.
| |
| * @param offset The offset from the cursor position where deletion starts.
| |
| * @param length The length of text to delete.
| |
| */
| |
| <strike>Promise<boolean> deleteSurroundingText(long offset, long length);</strike>
| |
| Promise<void> deleteSurroundingText(long offset, long length);
| |
|
| |
| /*
| |
| * Notifies when the text around the cursor is changed, due to either text
| |
| * editing or cursor movement. If the cursor has been moved, but the text around has not
| |
| * changed, the IME won't get notification.
| |
| */
| |
| // [JS] Can you describe how the cursor can be moved without the surrounding text
| |
| // also changing? Is that really something that can happen?
| |
| // [yxl] For example, if the text field is filled with 'a', wherever the cusor movies the surrounding text is always 'aa...'. Another exmaple, the selection range is changed, but the cursor isn't and the surrouding text won't be changed.
| |
| attribute EventHandler onsurroundingtextchange;
| |
|
| |
| /*
| |
| * send a character with its key events.
| |
| * @param modifiers see http://mxr.mozilla.org/mozilla-central/source/dom/interfaces/base/nsIDOMWindowUtils.idl#206
| |
| * @return true if succeeds. Otherwise false if the input context becomes invalid.
| |
| * Alternative: sendKey(KeyboardEvent event), but we will likely waste memory for creating the KeyboardEvent object.
| |
| */
| |
| <strike>Promise<boolean> sendKey(long keyCode, long charCode, [optional] long modifiers);</strike>
| |
| Promise<void> sendKey(long keyCode, long charCode, [optional] long modifiers);
| |
|
| |
| /*
| |
| * Set current composing text. This method will start composition or update composition if it
| |
| * has started. The composition will be started right before the cursor position and any
| |
| * selected text will be replaced by the composing text. When the composition is started,
| |
| * calling this method can update the text and move cursor winthin the range of the composing
| |
| * text.
| |
| * @param text The new composition text to show.
| |
| * @param cursor The new cursor position relative to the start of the composition text. The cursor should
| |
| * be positioned within the composition text. This means the value should be >= 0 and <= the length of
| |
| * composition text. Defaults to the lenght of composition text, i.e., the cursor will be positioned after
| |
| * the composition text.
| |
| *
| |
| * The composing text, which is shown with underlined style to distinguish from the existing text, is used
| |
| * to compose non-ASCII characters from keystrokes, e.g. Pinyin or Hiragana. The composing text is the
| |
| * intermediate text to help input complex character and is not committed to current input field. Therefore
| |
| * if any text operation other than composition is performed, the composition will be automatically
| |
| * canceled and the composing text will be cleared. Same apply when the inputContext is lost during a
| |
| * unfinished composition session.
| |
| *
| |
| * To finish composition and commit text to current input field, an IME should call |endComposition|.
| |
| */
| |
| <strike>Promise<boolean> setComposition(DOMString text, [optional] long cursor);</strike>
| |
| Promise<void> setComposition(DOMString text, [optional] long cursor);
| |
|
| |
| /*
| |
| * End composition, clear the composing text and commit given text to current input field. The text will
| |
| * be committed before the cursor position.
| |
| * @param text The text to commited before cursor position. If empty string is given, no text will be
| |
| * committed.
| |
| *
| |
| * Note that composition always ends automatically with nothing to commit if the composition does not
| |
| * explicitly end by calling |endComposition|, but is interrupted by |sendKey|, |setSelectionRange|,
| |
| * |replaceSurroundingText|, |deleteSurroundingText|, user moving the cursor, changing the focus, etc.
| |
| */
| |
| <strike>Promise<boolean> endComposition([optional] DOMString text);</strike>
| |
| Promise<void> endComposition(DOMString text);
| |
| };
| |
|
| |
|
| === Use cases for each of the methods === | | === Use cases for each of the methods === |
|
| |
|
| * For a simple virtual keyboard action (send a character and key events w/ each user action), use <code>sendKey()</code>. TODO: should we allow backspace key to be sent from the method? If not, how do send these non-printable characters and it's effect with key events? | | * For a simple virtual keyboard action (send a character and key events w/ each user action), use <code>sendKey()</code>. TODO: should we allow backspace key to be sent from the method? If not, how do send these non-printable characters and it's effect with key events? |
| *[yxl] I perfer to allowing non-printable character, such as backspace key, to be sent, if there is no security issue. This | | **[yxl] I perfer to allowing non-printable character, such as backspace key, to be sent, if there is no security issue. This would give the IME more flexibility. |
| * would give the IME more flexibility.
| |
| * For spellcheck, autocomplete etc, use surrounding text methods. | | * For spellcheck, autocomplete etc, use surrounding text methods. |
| * For cursor moment helper features, use <code>setSelectionRange()</code> and related attributes. | | * For cursor moment helper features, use <code>setSelectionRange()</code> and related attributes. |
| Line 475: |
Line 315: |
|
| |
|
| http://developer.chrome.com/trunk/extensions/input.ime.html | | http://developer.chrome.com/trunk/extensions/input.ime.html |
| | |
| | [[Category:Web APIs]] |