WebAPI/KeboardIME: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
mNo edit summary
(Remove the IDL draft)
 
(83 intermediate revisions by 10 users not shown)
Line 1: Line 1:
= Virtual Keyboard/IME API =


__TOC__
__TOC__
Line 5: Line 4:
== Introduction ==
== Introduction ==


Virtual Keyboard/IME API aims to implement the system IME as a Web App.
Virtual Keyboard/IME API aims to implement the system IME as a Web App. It will be used in Firefox OS and use cases could be found in the
[https://wiki.mozilla.org/images/6/61/Gaia_Keyboard_20130417.pdf Firefox OS Keyboard UX document(WIP)].


The API provides the communication channel between the IME App and the other App that receives user's inputs.
The API provides the communication channel between the IME App and the other App that receives user's inputs.
Line 17: Line 17:
# [https://groups.google.com/forum/?fromgroups#!topic/mozilla.dev.webapi/Vs3-HGv9NNw WebAPI mailing list post]
# [https://groups.google.com/forum/?fromgroups#!topic/mozilla.dev.webapi/Vs3-HGv9NNw WebAPI mailing list post]
# [https://groups.google.com/forum/?fromgroups=#!topic/mozilla.dev.webapi/A7dIBaR3lpU Extended API mailing list post]
# [https://groups.google.com/forum/?fromgroups=#!topic/mozilla.dev.webapi/A7dIBaR3lpU Extended API mailing list post]
# [https://groups.google.com/forum/#!msg/mozilla.dev.webapi/nZLOEvEL3Tk/M-nSnP6HZgoJ Polished Keyboard API]


See {{bug|737110}} for the implementation.
Implementation:
#{{bug|737110}} - Bug 737110 - Virtual Keyboard API


== Features ==
== Features ==
The Virtual Keyboard/IME API supports the following features:
* Notifies the VKB app when the focus text field was changing in other apps
* Allow user to manual hide the keyboard. Check {{bug|737110}}.
* The VKB app should be responsive to properties and the state of the  input field (more than HTML5 input type, including current content,  cursor position, x-inputmode {{bug|796544}}).
* Sends trust
* The VKB app should be able to send trusted key events such as they are considered by the other apps as user' inputs.
* The VKB app should be able to send a character or a string to the current input cursor position.
* Keyboard should be able to overwrite the current value of the input field of the input field and set the cursor position.
* The VKB app should be able to move the cursor position or select a specified range of text.
* The VKB should be able to switch the focus onto the previous/next input field.
* The return key label of the VKB can be customized.
== Proposed Manifest of a 3rd-Party IME ==
Just like any other apps, keyboard apps register themselves in the same apps registry. We extend the manifest syntax here to describe layout(s) available in a given keyboard app. Gaia will be paring the manifest. There are 3 special fields to distinguish and describe a 3rd-party IME:
* [Line 4] a "role" field with value "keyboard" declares it's an IME app. Homescreen app will ignore some role types when displaying app icons, and "keyboard" is one of them. (see {{bug|892397}})
* [Line 6-8] a "permissions" field that requests "keyboard" permission. All IME apps need this permission for sending input keys and updating the value of a input field.
* [Line 9-30] a "inputs" field specifies supported layouts. Each layout is described in a key-value pair, where the key represents the layout name (will be shown up on Settings app with the app name), and the value describes the detailed information of the layout, including launch path of the layout and supported input types. (See [[#Layout Matching Algorithm]])
** The allowed value in "types" field is a subset of [http://www.whatwg.org/specs/web-apps/current-work/multipage/the-input-element.html#attr-input-type type attribute of input element]: text, search, tel, number, url, email. Other types will be ignored by FxOS Gaia in the initial version because at this point UI for <select> and <input type=date> (called "value selectors") are not open for 3rd-party implementation.
{
  "name": "3rd-party Keyboard",
  "description": "3rd-party Keyboard",
  "type": "privilege",
  "role": "input",
  "launch_path": "/settings.html",
  "developer": {
    "name": "developer's name",
    "url": "https://keyboard.example.com"
  },
  "permissions": {
    "input": {}
  },
  "inputs": {
    "en": {
      "launch_path": "/index.html#en",
      "name": "English",
      "description": "English layout",
      "types": ["url", "text"],
      "locales": {
        "en-US": {
          "name": "English",
          "description": "English layout"
        },
        "zh-TW": {
          "name": "英文",
          "description": "英文鍵盤"
        }
      }
    },
    "en-Dvorak": {
      "launch_path": "/index.html#en-Dvorak",
      "name": "English (Dvorak)",
      "description": "Dvorak layout",
      "types": ["url", "text"]
    },
    "es": {
      "launch_path": "/index.html#es",
      "name": "Spanish",
      "description": "Spanish layout",
      "types": ["url", "text"]
    },
    "pt-BR": {
      "launch_path": "/index.html#pt-BR",
      "name": "Portuguese Brazilian",
      "description": "Portuguese Brazilian layout",
      "types": ["url", "text"]
    },
    "pl": {
      "launch_path": "/index.html#pl",
      "name": "Polish",
      "description": "Polish layout",
      "types": ["url", "text"]
    },
    "ca": {
      "launch_path": "/index.html#ca",
      "name": "Catalan",
      "description": "Catalan layout",
      "types": ["url", "text"]
    },
    "cz": {
      "launch_path": "/index.html#cz",
      "name": "Czech",
      "description": "Czech layout",
      "types": ["url", "text"]
    },
    "fr": {
      "launch_path": "/index.html#fr",
      "name": "French",
      "description": "French layout",
      "types": ["url", "text"]
    },
    "de": {
      "launch_path": "/index.html#de",
      "name": "German",
      "description": "German layout",
      "types": ["url", "text"]
    },
    "nb": {
      "launch_path": "/index.html#nb",
      "name": "Norwegian Bokmal",
      "description": "Norwegian Bokmal layout",
      "types": ["url", "text"]
    },
    "sk": {
      "launch_path": "/index.html#sk",
      "name": "Slovak",
      "description": "Slovak layout",
      "types": ["url", "text"]
    },
    "tr-Q": {
      "launch_path": "/index.html#tr-Q",
      "name": "Turkish Q",
      "description": "Turkish Q layout",
      "types": ["url", "text"]
    },
    "tr-F": {
      "launch_path": "/index.html#tr-F",
      "name": "Turkish F",
      "description": "Turkish F layout",
      "types": ["url", "text"]
    },
    "ro": {
      "launch_path": "/index.html#ro",
      "name": "Romanian",
      "description": "Romanian layout",
      "types": ["url", "text"]
    },
    "ru": {
      "launch_path": "/index.html#ru",
      "name": "Russian",
      "description": "Russian layout",
      "types": ["url", "text"]
    },
    "sr-Cyrl": {
      "launch_path": "/index.html#sr-Cyrl",
      "name": "Serbian (Cyrillic)",
      "description": "Serbian (Cyrillic) layout",
      "types": ["url", "text"]
    },
    "sr-Latn": {
      "launch_path": "/index.html#sr-Latn",
      "name": "Serbian (Latin)",
      "description": "Serbian (Latin) layout",
      "types": ["url", "text"]
    },
    "ar": {
      "launch_path": "/index.html#ar",
      "name": "Arabic",
      "description": "Arabic layout",
      "types": ["url", "text"]
    },
    "he": {
      "launch_path": "/index.html#he",
      "name": "Hebrew",
      "description": "Hebrew layout",
      "types": ["url", "text"]
    },
    "hu": {
      "launch_path": "/index.html#hu",
      "name": "Hungarian",
      "description": "Hungarian layout",
      "types": ["url", "text"]
    },
    "el": {
      "launch_path": "/index.html#el",
      "name": "Greek",
      "description": "Greek layout",
      "types": ["url", "text"]
    },
    "zh-Hans-Pinyin": {
      "launch_path": "/index.html#zh-Hans-Pinyin",
      "name": "Pinyin",
      "description": "Pinyin",
      "types": ["url", "text"]
    },
    "number": {
      "launch_path": "/index.html#numberLayout",
      "name": "Number",
      "description": "Number layout",
      "types": ["number"]
    }
  },
  "locales": {
    "en-US": {
      "name": "3rd-party Keyboard",
      "description": "3rd-party Keyboard"
    },
    "zh-TW": {
      "name": "第三方鍵盤",
      "description": "第三方鍵盤"
    }
  },
  "default_locale": "en-US"
}
=== Layout Matching Algorithm ===
When an input field is focused, if its <code>type</code> attribute is one of the allowed values stated above, it will be used to filter a set of candidate layouts. A candidate layout means it can handle this input type or is possible to let user input all characters that this input field can accept.
For example, if the type of a input is "url", then a layout with "url" or "text" listed in the <code>types</code> of its manifest will be matched. However, if a input field with type "text", then all layouts that support "text" will be matched, but those layouts that only support "url" will not. This is because we believe layouts that can handle "text" could be a fallback for "url" input type, but not vice versa. We also believe "text" could be a fallback for all allowed types stated above.
The matching algorithm of keyboard manager in System app is as follows:
# With the given type, find all layouts claims to support the said type and put it into the list.
# Next, find layouts claims to support "text" and put it into the list. Layouts do not get duplicated listing even if it supports both types.
# Present the user with the choice of the layouts available to handle the input field. The order of presenting list is depend on UX design and/or user preferences in Settings.


== Proposed API ==
== Proposed API ==


partial interface Navigator {
'''The API has made available to privileged apps. See the [http://dxr.mozilla.org/mozilla-central/source/dom/webidl/InputMethod.webidl WebIDL] for the current interface.'''
    readonly attribute InputMethodManager mozInputMethodManager;
 
    readonly attribute InputMethodConnection mozInputMethodConnection;
[https://wiki.mozilla.org/index.php?title=WebAPI/KeboardIME&oldid=1029753#Proposed_API History of this section]
};
 
=== Use cases for each of the methods ===
 
* For a simple virtual keyboard action (send a character and key events w/ each user action), use <code>sendKey()</code>. TODO: should we allow backspace key to be sent from the method? If not, how do send these non-printable characters and it's effect with key events?
**[yxl] I perfer to allowing non-printable character, such as backspace key, to be sent, if there is no security issue. This would give the IME more flexibility.
* For spellcheck, autocomplete etc, use surrounding text methods.
* For cursor moment helper features, use <code>setSelectionRange()</code> and related attributes.
* For Asian IMEs that sends characters and composition along with the composition events, use <code>setComposition()</code> and <code>endComposition()</code>.
 
It is important to stick with the given use cases because the web application might need to react with what the user actually do. To test the events currently sent to the web, see http://jsfiddle.net/timdream/YDGgk/ .
 
== Examples ==
 
The following "snowman filler" Keyboard app will start filling snowman character ("☃") and follow by characters "SNOW" with key events to the input field whenever the user is focus on a input field and switch to the keyboard app.


  interface InputMethodConnection {
If the field is a numeric field, it will fill "1337".
     // User begins or finishes editing a field.
 
     // If the user change input directly from one input to another, only one event will be dispatched.
  var timer;
     attribute Function oninputmethodstatechange;
function startTyping(inputContext) {
  clearTimeout(timer);
  timer = setInterval(function typing() {
     /* [JJ] So I think that this code shouldn't be here, because you'll get lots of clutter
      *     as you'll also have to take languages into account.
      *     Rather rely on entry points in manifest...
      */
   
   
     // Whether the user begins editing.
     if (inputContext.inputmode === 'numeric' || inputContext.type === 'number') {
     readonly attribute boolean inputStarted;  
      ['1', '3', '3', '7'].forEach(function (k) {
        // For numbers, keyCode is same as the charCode.
        inputContext.sendKey(k.charCodeAt(0), k.charCodeAt(0));
      });
     } else {
      // It's not a good idea to commit text w/o sending events. So we should first send composition events.
      inputContext.setComposition('☃');
      // end the composition and commit the text.
      inputContext.endComposition('☃');
      ['S', 'N', 'O', 'W'].forEach(function (k) {
        // For capital Latin letters, keyCode is same as the charCode.
        inputContext.sendKey(k.charCodeAt(0), k.charCodeAt(0));
      });
  }, 1000);
}
   
   
    // User moves the cursor, changes the selection, or alters the composing text length
function stopTyping() {
    attribute Function onselectionchange;
  clearTimeout(timer);
}
   
   
    // Send a keyevent to the input field that is currently attached. It exists in the current interface
var im = navigator.inputMethod;
    void sendKey(in long keyCode, in long charCode, in long modifiers);
   
   
    // Insert the given text at the cursor position. Replace the selected text if any and clear the composing text.
im.addEventListener('inputcontextchange', function contextchanged(evt) {
    void commitText(in DOMString text);
  if (evt.inputcontext) {
      // Got a new context, start working with it.
      startTyping(evt.inputcontext);
  } else {
      // The user have removed the focus, we are not allow to type into the text field anymore.
      stopTyping();
  }
});
   
   
    // Set the text value of the current input field.
  if (im.inputcontext) {
    void setText(in DOMString text);
  // The webpage here is loaded *after* the user has place the focus on the text field,
   
  // let's start typing now.
    // Get the text value of the current input field asynchronously.
  startTyping(im.inputcontext);
    void getText(in long start, in long end, in Function callback);
  }
    // Length of the content
    readonly attribute long textLength;
    // Delete text around the cursor
    void deleteSurroundingText(in long beforeLength, in long afterLength);
    // Set the composing text before the current cursor position.
    // To clear the composing text, set the text paramter to the empty string.
    void setComposingText(in DOMString text);
    // The start and stop position of the selection.
    readonly attribute long selectionStart;
    readonly attribute long selectionEnd;
    // Set the selection range of the the editable text.
    // Note that the start position should be less or equal to the end position.
    // To move the cursor, set the start and end position to the same value.
    void setSelectionRange(in long start, in long end);
    // Clear the focus of the current input field and hide the keyboard.
    void removeFocus();
    // Focus the next text field.
    void advanceFocus();
    // Focus the previous text field.
    void rewindFocus();
    // This read only attribute is true if user can advance the focus to the next
    // text field.
    readonly attribute boolean canAdvanceFocus;
    // This readonly attribute is true if user can rewind the focus back to the
    // previous text field.
    readonly attribute boolean canRewindFocus;
    // The input mode string.
    // https://bugzilla.mozilla.org/show_bug.cgi?id=796544
    // It can be one of the following values:
    // "none"
    // "verbatim"  - no capitalization, no word suggestions
    // "latin"          - word suggestions but no capitalization
    // "latin-prose" - word suggestions and capitalization at the start of sentences
    // "latin-name" - word suggestions and capitalize each word
    readonly attribute DOMString inputMode;
    // The type of the input field, including text, number, password, url, tel and email.
    readonly attribute DOMString inputType;
    // Get the return key type, which determines the title of the key displayed in the keyboard and action
    // performed when it is  pressed. The value could be "done", "next", "go", "search" and "send".
    // If the value is "next", when pressing the enter key, the focus should be move to the next input field.
    readonly attribute DOMString returnKeyType;
    // Get the customized label for the return key.
    readonly attribute DOMString returnKeyLabel;
  };


== Examples ==
== Related ==


var conn = navigator.mozInputMethodConnection;
Android IME API:


// Called when the user starts or finishes editing an input field
http://developer.android.com/guide/topics/text/creating-input-method.html#IMEAPI
conn.oninputmethodstatechange= function(event) {
    if (conn.inputStarted) {
        // Text input should start
    } else {
        // Text input should end
    }
};


// Insert a string at the current cursor position
iOS Keyboard Management:
conn.commitText('Hello world');


// Get the selected text
http://developer.apple.com/library/ios/#documentation/StringsTextFonts/Conceptual/TextAndWebiPhoneOS/KeyboardManagement/KeyboardManagement.html#//apple_ref/doc/uid/TP40009542-CH5-SW1
conn.getText(conn.selectionStart, conn.selectionEnd, function callback(text) {
    var selectedText = text;
});


// Move the cursor position
Chrome Extensions API:
var position = 10;
conn.setSelectionRange(position, position);


// Hide the keyboard
http://developer.chrome.com/trunk/extensions/input.ime.html
conn.removeFocus();


// Switch the focus onto the next input field if possible
[[Category:Web APIs]]
if (conn.canAdvanceFocus) {
    conn.advanceFocus();
}
 
== Related ==

Latest revision as of 03:35, 3 November 2014

Introduction

Virtual Keyboard/IME API aims to implement the system IME as a Web App. It will be used in Firefox OS and use cases could be found in the Firefox OS Keyboard UX document(WIP).

The API provides the communication channel between the IME App and the other App that receives user's inputs.

It is very different from the IME API from Google that aims to re-use the system's IME in a web page.

Status

API discussion:

  1. WebAPI mailing list post
  2. Extended API mailing list post
  3. Polished Keyboard API

Implementation:

  1. bug 737110 - Bug 737110 - Virtual Keyboard API

Features

The Virtual Keyboard/IME API supports the following features:

  • Notifies the VKB app when the focus text field was changing in other apps
  • Allow user to manual hide the keyboard. Check bug 737110.
  • The VKB app should be responsive to properties and the state of the input field (more than HTML5 input type, including current content, cursor position, x-inputmode bug 796544).
  • Sends trust
  • The VKB app should be able to send trusted key events such as they are considered by the other apps as user' inputs.
  • The VKB app should be able to send a character or a string to the current input cursor position.
  • Keyboard should be able to overwrite the current value of the input field of the input field and set the cursor position.
  • The VKB app should be able to move the cursor position or select a specified range of text.
  • The VKB should be able to switch the focus onto the previous/next input field.
  • The return key label of the VKB can be customized.

Proposed Manifest of a 3rd-Party IME

Just like any other apps, keyboard apps register themselves in the same apps registry. We extend the manifest syntax here to describe layout(s) available in a given keyboard app. Gaia will be paring the manifest. There are 3 special fields to distinguish and describe a 3rd-party IME:

  • [Line 4] a "role" field with value "keyboard" declares it's an IME app. Homescreen app will ignore some role types when displaying app icons, and "keyboard" is one of them. (see bug 892397)
  • [Line 6-8] a "permissions" field that requests "keyboard" permission. All IME apps need this permission for sending input keys and updating the value of a input field.
  • [Line 9-30] a "inputs" field specifies supported layouts. Each layout is described in a key-value pair, where the key represents the layout name (will be shown up on Settings app with the app name), and the value describes the detailed information of the layout, including launch path of the layout and supported input types. (See #Layout Matching Algorithm)
    • The allowed value in "types" field is a subset of type attribute of input element: text, search, tel, number, url, email. Other types will be ignored by FxOS Gaia in the initial version because at this point UI for <select> and <input type=date> (called "value selectors") are not open for 3rd-party implementation.

{

 "name": "3rd-party Keyboard",
 "description": "3rd-party Keyboard",
 "type": "privilege",
 "role": "input",
 "launch_path": "/settings.html",
 "developer": {
   "name": "developer's name",
   "url": "https://keyboard.example.com"
 },
 "permissions": {
   "input": {}
 },
 "inputs": {
   "en": {
     "launch_path": "/index.html#en",
     "name": "English",
     "description": "English layout",
     "types": ["url", "text"],
     "locales": {
       "en-US": {
         "name": "English",
         "description": "English layout"
       },
       "zh-TW": {
         "name": "英文",
         "description": "英文鍵盤"
       }
     }
   },
   "en-Dvorak": {
     "launch_path": "/index.html#en-Dvorak",
     "name": "English (Dvorak)",
     "description": "Dvorak layout",
     "types": ["url", "text"]
   },
   "es": {
     "launch_path": "/index.html#es",
     "name": "Spanish",
     "description": "Spanish layout",
     "types": ["url", "text"]
   },
   "pt-BR": {
     "launch_path": "/index.html#pt-BR",
     "name": "Portuguese Brazilian",
     "description": "Portuguese Brazilian layout",
     "types": ["url", "text"]
   },
   "pl": {
     "launch_path": "/index.html#pl",
     "name": "Polish",
     "description": "Polish layout",
     "types": ["url", "text"]
   },
   "ca": {
     "launch_path": "/index.html#ca",
     "name": "Catalan",
     "description": "Catalan layout",
     "types": ["url", "text"]
   },
   "cz": {
     "launch_path": "/index.html#cz",
     "name": "Czech",
     "description": "Czech layout",
     "types": ["url", "text"]
   },
   "fr": {
     "launch_path": "/index.html#fr",
     "name": "French",
     "description": "French layout",
     "types": ["url", "text"]
   },
   "de": {
     "launch_path": "/index.html#de",
     "name": "German",
     "description": "German layout",
     "types": ["url", "text"]
   },
   "nb": {
     "launch_path": "/index.html#nb",
     "name": "Norwegian Bokmal",
     "description": "Norwegian Bokmal layout",
     "types": ["url", "text"]
   },
   "sk": {
     "launch_path": "/index.html#sk",
     "name": "Slovak",
     "description": "Slovak layout",
     "types": ["url", "text"]
   },
   "tr-Q": {
     "launch_path": "/index.html#tr-Q",
     "name": "Turkish Q",
     "description": "Turkish Q layout",
     "types": ["url", "text"]
   },
   "tr-F": {
     "launch_path": "/index.html#tr-F",
     "name": "Turkish F",
     "description": "Turkish F layout",
     "types": ["url", "text"]
   },
   "ro": {
     "launch_path": "/index.html#ro",
     "name": "Romanian",
     "description": "Romanian layout",
     "types": ["url", "text"]
   },
   "ru": {
     "launch_path": "/index.html#ru",
     "name": "Russian",
     "description": "Russian layout",
     "types": ["url", "text"]
   },
   "sr-Cyrl": {
     "launch_path": "/index.html#sr-Cyrl",
     "name": "Serbian (Cyrillic)",
     "description": "Serbian (Cyrillic) layout",
     "types": ["url", "text"]
   },
   "sr-Latn": {
     "launch_path": "/index.html#sr-Latn",
     "name": "Serbian (Latin)",
     "description": "Serbian (Latin) layout",
     "types": ["url", "text"]
   },
   "ar": {
     "launch_path": "/index.html#ar",
     "name": "Arabic",
     "description": "Arabic layout",
     "types": ["url", "text"]
   },
   "he": {
     "launch_path": "/index.html#he",
     "name": "Hebrew",
     "description": "Hebrew layout",
     "types": ["url", "text"]
   },
   "hu": {
     "launch_path": "/index.html#hu",
     "name": "Hungarian",
     "description": "Hungarian layout",
     "types": ["url", "text"]
   },
   "el": {
     "launch_path": "/index.html#el",
     "name": "Greek",
     "description": "Greek layout",
     "types": ["url", "text"]
   },
   "zh-Hans-Pinyin": {
     "launch_path": "/index.html#zh-Hans-Pinyin",
     "name": "Pinyin",
     "description": "Pinyin",
     "types": ["url", "text"]
   },
   "number": {
     "launch_path": "/index.html#numberLayout",
     "name": "Number",
     "description": "Number layout",
     "types": ["number"]
   }
 },
 "locales": {
   "en-US": {
     "name": "3rd-party Keyboard",
     "description": "3rd-party Keyboard"
   },
   "zh-TW": {
     "name": "第三方鍵盤",
     "description": "第三方鍵盤"
   }
 },
 "default_locale": "en-US"

}

Layout Matching Algorithm

When an input field is focused, if its type attribute is one of the allowed values stated above, it will be used to filter a set of candidate layouts. A candidate layout means it can handle this input type or is possible to let user input all characters that this input field can accept. For example, if the type of a input is "url", then a layout with "url" or "text" listed in the types of its manifest will be matched. However, if a input field with type "text", then all layouts that support "text" will be matched, but those layouts that only support "url" will not. This is because we believe layouts that can handle "text" could be a fallback for "url" input type, but not vice versa. We also believe "text" could be a fallback for all allowed types stated above.

The matching algorithm of keyboard manager in System app is as follows:

  1. With the given type, find all layouts claims to support the said type and put it into the list.
  2. Next, find layouts claims to support "text" and put it into the list. Layouts do not get duplicated listing even if it supports both types.
  3. Present the user with the choice of the layouts available to handle the input field. The order of presenting list is depend on UX design and/or user preferences in Settings.

Proposed API

The API has made available to privileged apps. See the WebIDL for the current interface.

History of this section

Use cases for each of the methods

  • For a simple virtual keyboard action (send a character and key events w/ each user action), use sendKey(). TODO: should we allow backspace key to be sent from the method? If not, how do send these non-printable characters and it's effect with key events?
    • [yxl] I perfer to allowing non-printable character, such as backspace key, to be sent, if there is no security issue. This would give the IME more flexibility.
  • For spellcheck, autocomplete etc, use surrounding text methods.
  • For cursor moment helper features, use setSelectionRange() and related attributes.
  • For Asian IMEs that sends characters and composition along with the composition events, use setComposition() and endComposition().

It is important to stick with the given use cases because the web application might need to react with what the user actually do. To test the events currently sent to the web, see http://jsfiddle.net/timdream/YDGgk/ .

Examples

The following "snowman filler" Keyboard app will start filling snowman character ("☃") and follow by characters "SNOW" with key events to the input field whenever the user is focus on a input field and switch to the keyboard app.

If the field is a numeric field, it will fill "1337".

var timer;
function startTyping(inputContext) {
  clearTimeout(timer);
  timer = setInterval(function typing() {
    /* [JJ] So I think that this code shouldn't be here, because you'll get lots of clutter
     *      as you'll also have to take languages into account.
     *      Rather rely on entry points in manifest...
     */

    if (inputContext.inputmode === 'numeric' || inputContext.type === 'number') {
      ['1', '3', '3', '7'].forEach(function (k) {
        // For numbers, keyCode is same as the charCode.
        inputContext.sendKey(k.charCodeAt(0), k.charCodeAt(0));
      });
    } else {
      // It's not a good idea to commit text w/o sending events. So we should first send composition events.
      inputContext.setComposition('☃');
      // end the composition and commit the text.
      inputContext.endComposition('☃');
      ['S', 'N', 'O', 'W'].forEach(function (k) {
        // For capital Latin letters, keyCode is same as the charCode.
        inputContext.sendKey(k.charCodeAt(0), k.charCodeAt(0));
      });
  }, 1000);
}

function stopTyping() {
  clearTimeout(timer);
}

var im = navigator.inputMethod;

im.addEventListener('inputcontextchange', function contextchanged(evt) {
  if (evt.inputcontext) {
     // Got a new context, start working with it.
     startTyping(evt.inputcontext);
  } else {
     // The user have removed the focus, we are not allow to type into the text field anymore.
     stopTyping();
  }
});

if (im.inputcontext) {
  // The webpage here is loaded *after* the user has place the focus on the text field,
  // let's start typing now.
  startTyping(im.inputcontext);
}

Related

Android IME API:

http://developer.android.com/guide/topics/text/creating-input-method.html#IMEAPI

iOS Keyboard Management:

http://developer.apple.com/library/ios/#documentation/StringsTextFonts/Conceptual/TextAndWebiPhoneOS/KeyboardManagement/KeyboardManagement.html#//apple_ref/doc/uid/TP40009542-CH5-SW1

Chrome Extensions API:

http://developer.chrome.com/trunk/extensions/input.ime.html