Input Method Framework

Design Specification

Version 1.0
October 5, 1998

Copyright © 1997-1998 Sun Microsystems, Inc. All rights reserved.


Contents

  1. Introduction
  2. Design Requirements
  3. Functional Requirements
  4. Framework Architecture
  5. Input Method API
  6. Changes to Other Frameworks
  7. Future Enhancements


Abstract

This document describes the input method framework for the Java Development Kit, release 1.2. This framework enables developers of text editing components to interact with input methods that enable text input for languages with large character sets. JDK 1.2 uses native input methods of the host platform as well as (through a IIIMP adapter) server-based input methods. The framework is designed to be easily extended in a future JDK version to enable input methods written in the Java programming language.

References

This document does not provide a complete overview of input methods and their implementations on the different platforms. This information can be found in the following literature:

API Definitions and Sample Code

For detailed API definitions, refer to the separate API documents generated by javadoc. To see how to use these APIs, refer to the sample code.

Feedback

Please send feedback to java-intl@java.sun.com.


1. Introduction

Input methods are software components that interpret user operations such as typing keys, speaking, or writing using a pen device to generate text input for applications. The most common input methods are the ones that let users type text in Chinese, Japanese, or Korean, languages that use thousands of different characters, on a regular-sized keyboard. The text is typed in a form that can be handled by regular-sized keyboards, for example, in a romanized form, and then converted into the form that's really intended. Typically a sequence of several characters needs to be typed and then converted in one chunk, and conversion may have to be retried because there may be several possible translations. Similarly, for hand-writing recognition, the user may write a series of characters, convert them, and select the correct text from several possible conversion results.

While this "composition" process is going on, the text logically belongs to the input method, but still needs to be displayed to the user. Input method frameworks cooperate with input methods to provide at least two ways to do this: They enable modern text editing components to display the text in the context of the document that it will eventually belong to, albeit in a style that indicates that the text still needs to be converted or confirmed by the input method. This is called "on-the-spot editing". And they provide a separate window to display the text as a fall-back if a not-so-modern text editing component can't deal with the text until it's confirmed. We call this "root-window editing".

In the JDK 1.1 release, text input using input methods was only supported for the AWT TextField and TextArea components, using the native platform input method functionality. Starting with JDK 1.1.6, lightweight text components (such as the Swing text components) can receive text input from input methods using root-window style input.

The Input Method Framework in JDK 1.2 defines classes and interfaces that enable all text editing components to receive text through input methods. Components can actively use the input method API to support the on-the-spot input style, otherwise the framework provides root-window style input as a fallback. JDK 1.2 uses native input methods of the host platform as well as (through an adapter for the Internet-Intranet Input Method Protocol) server-based input methods. The framework is designed to be easily extended in a future JDK version to enable input methods written in the Java programming language.


2. Design Requirements

"Write Once, Run Anywhere in the World" for Text Components

Text editing components that use the input method framework run on any Java application environment and support any text input methods available on that Java application environment without modifying or recompiling the text editing component. The framework provides text editing components with an API that is platform independent, and maps to the platform APIs if necessary to use native input methods.

Language and Engine Independence

The input method framework provides a language independent API for handling input to satisfy the needs of fully international applications. Although input methods are generally used for entering East Asian text, they may be useful for other languages as well. For example, a transliteration input method might be developed to allow Greek to be entered using Latin characters.

The framework handles input methods for different languages at the same time for truly multilingual text editing. The text can have language attributes so that applications can perform language sensitive operations on the text.

Likewise, the framework isolates text editing components from differences in various input method engines. In particular, it provides only one programming model for interacting with input methods. This lets developers write to a single input method API and users to choose whatever input method they prefer.

Although language and engine independence will be important to most developers, some high-end developers may want access to language specific or engine specific features. The framework allows this access, although at the obvious cost of generality for the application.

Input Device Independence

The input method framework is designed to accommodate many possible input devices. Although keyboards are the predominant way of entering text today, other devices are gaining importance. For some platforms, pen-based input is the only way to enter text. Also, voice input is becoming more and more popular. In future releases, it will be possible to support these alternate devices.

Close Integration with Other Frameworks

The input method framework is designed as an integral part of the Java platform. New classes that are necessary to exchange text between input methods and text components are designed to also support communication between other text-related frameworks, such as Java 2D and the Swing text components. Support for input method highlight drawing is integrated into Java 2D, so that text components can treat composed text just like any other text and the highlights just like any other style. Java 2D interacts with the input method framework to determine appropriate visual styles for input method highlights depending on the platform. The Swing text components use the input method framework to implement the best possible user interface with minimal additional programming.


3. Functional Requirements

Text Input Model

Input Text Composition

In general, some languages require editing of text before actually incorporating the text into the application. This editing is called composition (or preedit). When composing input text, input methods require that segments of the input text be highlighted to indicate which part of the text is being processed. The process of inputting text with highlighting is called composing input text and the produced intermediate text is called composed (input) text.

When composition of input text is complete by the user, the final input text is sent to the application. This action is called committing input text and the produced final input text is called committed (input) text.

Visible Position

When drawing composed input text, it may not be possible to keep the entire composed text visible. For this case, an input method may provide the application with hints which parts of the composed text is currently most important to keep visible. The framework passes this information on from the input method to the application.

Caret Position

In general, the caret position represents an insertion position in the application's backing store. However, when composing input text, the caret position is located within composed text and the position is managed by input method. The input method informs the application about the current caret position.

Multiple Service Levels

The Input Method Framework must be able to support different levels of application needs. Some text applications might not want to deal with input methods, while some want to provide the best user interfaces to support them. A few programs, such as games, want direct low-level keyboard input without interference from an input method. The framework lets programs choose one of the following levels of text input support.

Integrated Text Input User Interface

Modern programs intended to be used with East Asian languages usually provide a fully integrated user interface for text input operations and show the text being composed embedded in the document text. While converting text, the current candidate is highlighted, and when the user chooses a different candidate, the previous candidate text is replaced with the new one. This user interface support for input methods is known as on-the-spot mode (or inline mode). The input method framework API lets text editing components implement such a user interface.

Non-Integrated Text Input User Interface

For programs that do not want to deal with the user interface for text input operations, only final input text is sent to the application. In this case, the framework provides a user interface for input operations in a separate composition window outside the application. This is known as root-window mode (an X Window term). It is less convenient because the user has to move his/her eyes between the input window and the application window.

In this case, the application receives the input text as a sequence of key events. Therefore, there is no way to receive any information associated with the input text, such as grammatical information. Also, surrogate pairs (as defined in the Unicode Standard, version 2.0) are received as two separate key events.

No Input Method Support

Some applications, such as game software, may need only raw key input and no input method support. The framework provides a way to explicitly disable input method support.

Multiple Concurrent Input Operations

The Input Method Framework allows applications to support multiple concurrent input operations. The state of each input operation is maintained separately. For example, in the middle of entering text into a document, the user might want to deal with a file and enter the file's name in a separate input operation.


4. Framework Architecture

The following picture shows the overall structure of the framework and its clients. Note that only parts outlined in bold are actually integrated with the input method framework as part of JDK 1.2.

The Input Method API defines the classes and interfaces that text editing components can use to implement on-the-spot text input. The AWT text components TextArea and TextField provide on-the-spot or over-the-spot composition, depending on implementation. The Swing text components provide on-the-spot composition. The context management manages communication paths between the text editing components and input methods. JDK 1.2 uses primarily native input methods integrated with the host input method manager. An adapter translates information between the data models used by native input methods and the input method framework. The framework also works with the native input method managers to provide parts of the user interface for text composition, especially the root window. JDK 1.2 also includes an adapter that makes it possible to use server-based input methods through the Internet-Intranet Input Method Protocol (IIIMP). In future versions of the JDK, the Input Method Engine API will support input methods written in the Java programming language.

Client Components

The clients of the input method framework are text editing components of all kinds. Applications don't really exist for the framework. Corresponding to the service levels described above, there are three kinds of components:

Text components that support on-the-spot editing and others that do not may coexist in the same window. The input method framework, therefore, needs to determine the capabilities of text components and switch between on-the-spot and root-window editing as the focus changes from one component to another.

Text components are not responsible for drawing candidate lists or for user interface elements that control the input method's behavior. Depending on the platform, this responsibility rests with input methods, the host input method manager, or the input method framework.

Context Management

InputContext instances manage communication contexts between client components and input methods. The main task of an input context is to maintain the connection from the current text component to its current input method. It dispatches input events such as key and mouse events from the component to the current input method, and input method events from the input method to the client component. It also dispatches requests for information from the input method to the client component.

In addition, the following functionality is provided by this class:

By default, one InputContext instance is created per Window instance, and this input context is shared by all components within the window's containment hierarchy. If necessary, components can create private input contexts. A component that doesn't have its own input context uses the one used by its parent.

The input context creates an input method instance for each input method engine that its client components need to use. The instances are kept until the window is closed, so that the instances can keep information about the text that was entered in the window.

In many situations, no input composition is desired, and key events should be handled directly by the component's key event listener. If this is a temporary situation - for example, German text is typed on the keyboard -, this is handled in JDK 1.2 by switching from an input method to regular keyboard handling using the host's switching mechanism. If a component never wants to use input composition - for example, a game wants to interpret keys directly -, it can disable input methods.

For text components that do not support on-the-spot editing, the input context provides a default user interface for text composition. When an input method starts composing text for this text component, the input context opens an input window for root-window style input. When the text is committed, the text is sent to the text component as a series of key events. When all text in the input window is committed, the window is closed. If the window containing the focused text component loses the focus while text is being composed, the input window is hidden until the window regains the focus, so that the user will never have two input windows in view at the same time.

Event Flow

The following diagrams show the flow of events in the input method framework. There are three alternatives, for active clients, for passive clients, and for non-clients.

Non-clients don't have input contexts, so all events go straight to the component's listeners (here, the key listener):

Event flow for non-client

For both active and passive clients, input events such as KeyEvent and MouseEvent are sent to the current input method through the InputContext object. If the input event is used to perform the input text service, the input event is consumed by the input method and will result in generating an input method event to the component.

Event flow for active client

Active clients must have registered an InputMethodListener so they can handle InputMethodEvents coming from the input method. Composed and committed text is sent to the input method listener as input method events.

Event flow for passive client

Passive clients don't have input method listeners, so the input method event is given to the root window, which simulates an active client. The root window translates committed text into key events for the actual client's key event listener. Only KEY_TYPED events are sent. Note that for host input methods the root window is typically provided by the host input method manager.

Host Input Method Adapters

JDK 1.2 supports native input methods integrated with the host input method managers. These are: the Input Method Manager on Win32, the Text Services Manager on MacOS, and XIM on Solaris. The host input method adapter plays the role of an input method within the input method framework, and translates events and requests between the data models used by AWT and the input method framework on one side and the host's input method manager on the other side. It also cooperates with the input context in managing the root window - for host input methods, typically the root window provided by the host input method manager is used. The adapter passes on requests for particular input methods to the host, but users can also use the host's selection mechanism to select input methods.

Separate host adapter instances are created for each InputContext instance, so that each InputContext instance is mirrored by its own instance of the host system's input context data structure.

IIIMP Client-Side Adapter

For server-based input methods, JDK 1.2 provides a client-side adapter for the Internet-Intranet Input Method Protocol. IIIMP is a network protocol that lets client applications use server-based input methods over the network. The first client-side implementation of IIIMP is currently used in JavaOS. The adapter in JDK 1.2 is a subset of the implementation in JavaOS - it provides a full implementation of the protocol as well as user interface support for status and lookup choice information, but does not support lightweight input methods. The IIIMP adapter in JDK 1.2 is provided as a separate extension, and may not be included in all Java application environments derived from JDK 1.2.

Users need to provide the client adapter with information about the IIIMP server in a properties file. The file can be named ".iiimp" and located in the user's home directory, or "iiimp.properties" and located in the JDK's "lib" directory. If both files exist, the one in the user's home directory takes precedence. The file has a one-line entry:

iiimp.server=iiimp://servername:portid

where servername is the name of the host running the IIIMP server, and portid the port id used by the IIIMP server.

If the extension is included in the Java application environment and valid server information is found, the JDK provides a user interface that lets users choose between host platform input methods and server-based input methods. On Windows and Motif systems, a menu item is added to the menu attached to window titles (called the "system menu" on Windows and "window menu" on Motif). Choosing this menu item brings up a popup-menu, from which host input methods (as a group) or server-based input methods (by locale) can be selected. The user still uses the host system's selection mechanism to choose between different host input methods.

Separate IIIMP adapter instances are created for each InputContext instance, so that each InputContext instance is mirrored by its own instance of the server's input context data structure.

Client Method Invocation

The framework needs mechanisms to invoke a component's methods (a.k.a. callback) from an input method to notify the component of changes in input text or to request information from the component. The framework uses two different invocation models:

Text Representation

The framework uses the java.text.AttributedCharacterIterator interface to represent text moving in either direction between client component and input method. Both character attributes and range attributes may be used to provide input method highlights. Also, range attributes can be used to represent information depending on an input method, language, etc.


5. Input Method API

The Input Method API makes it possible for text components to:

Becoming a Non-Client

By default, all components that process key events are clients of the input method framework, that is, input method support is enabled for them. In some cases, components may not want to have their input processed by input methods. For example, games may want to interpret keyboard events directly. These components should call enableInputMethods(false), so that events do not get forwarded to input methods.

Becoming an Active Client Component

The Input Method API makes it possible for client components to implement on-the-spot editing. The API defines events and methods that facilitate the communication between a client component and an input method. It also lets the client component request an input method for a particular language.

Since the API doesn't make any assumptions about how and where the text is drawn, it can also be used to implement other input styles such as over-the-spot editing. In this style, the composed text is drawn over surrounding text and covers it instead of being integrated and formatted with it.

Any client component class can become an active client of the input method API and thus support on-the-spot editing by performing the following steps:

Optionally, the client component can also use the following functionality:

Client components don't need to deal with setting up input contexts, activating or deactivating contexts, or dispatching events to input methods, since all this is handled automatically by AWT.

Handling Input Method Events

In addition to the JDK 1.1 AWT events, the input method framework provides a new event class, InputMethodEvent, to support the communication between input methods and text components. The class has two separate event kinds: text changed and caret changed. A new event listener interface, InputMethodListener, supports these two events. An active client component must implement the InputMethodListener interface, register the listener, and handle both kinds of events.

InputMethodEvent instances are sent to the client component when there is a change to the user's input text or to the caret location within the composed text. The event sent for caret-only changes is a simplified version of the one for text changes (it just doesn't have text information), so the following discussion assumes the text-changed event.

An event reporting a text change has a reference to an instance of AttributedCharacterIterator that represents either composed text or committed text or both together.

The cases above can be determined by examining the value of the committed character count. If the value is 0 (zero), the entire text is composed text. If the value is equal to the length of the iterator's text, the entire text is committed. Otherwise, the value (in number of characters) represents the length of partially committed text, and the rest of the text is composed text.

The event also contains information about the current caret location within the the composed text (null if no caret is to be displayed), and about the part of the composed text that is most important to keep in view (null if the input method doesn't have a recommendation).

Handling Input Method Highlight Attributes

The text component generally draws the composed text as part of the text being edited, using its regular text layout and drawing functionality. However, it needs to add certain highlight style attributes to the composed text to indicate the current state of the composition. The framework defines these style attributes as abstract styles (for example, "unconverted unselected text" or "converted selected text"), and maps them internally to platform-dependent concrete styles (for example, 2-pixel gray underline).

Abstract highlight styles are represented by the InputMethodHighlight class. Instances of this class are used as attribute values of the AttributedCharacterIterator instances representing composed text. Text components must store these attributes with the composed text and pass them on to the drawing routines when drawing composed text. They can use either the new drawString methods that accept AttributedCharacterIterator, or create a TextLayout from the iterator and use its draw method. These new drawing methods interact with the input method framework to map abstract to concrete highlight styles. Text components using these methods therefore generally do not need to be concerned with the internal details of the input method highlights.

Here's information for text components that need more detail: InputMethodHighlights have three fields: a Boolean selected, an integer state, and an integer variation. Selected indicates whether the text range is the one that the input method is currently working on, for example, the segment for which conversion candidates are currently shown in a menu. State represents the conversion state. State values are defined by the input method framework and should be distinguished in all mappings from abstract to concrete styles. Currently defined state values are unconverted and converted. The variation field allows future input methods written in the Java programming language to express additional information about the conversion results. Renderers should negotiate the concrete style with the input method or ignore this field.

Some input methods may treat highlights as "annotations". Annotations are attributes that apply to a specified range of text, but not to subranges or the concatenation of ranges. They are represented by wrapping the InputMethodHighlight instance into a java.text.Annotation instance. Input methods may use annotation highlights to separate text segments that will be converted as separate units. On some platforms, these highlights are rendered so as to make the segments visible, for example, by using underlines with short breaks between the segments. Text components have to be able to handle input method highlights whether they are wrapped in Annotation instances or not. If a text component implements line wrapping, special care needs to be taken when the range to which a highlight annotation applies crosses a line boundary: The normal behavior (implemented, for example, in java.text.AttributedString) would be to discard the attribute because it doesn't apply to subranges. But, since in this case there's only a visual break and not a logical break, the highlight needs to be preserved - it has to be treated as if it applied to the subranges that are rendered on separate lines. One way to do this is by implementing the java.text.AttributedCharacterIterator in a way that returns highlight annotations even for subranges of the intended range.

Handling Input Method Requests

An input method needs to access component information to perform input operations. For example, an input method needs to know the location where a list of possible choices can be shown.

An active client component therefore must implement the InputMethodRequests interface, and override getInputMethodRequests to return the request handler. The interface includes methods to:

Ending Input Operations

Input methods typically recognize some user actions that end input operations, for example, an operation that commits all uncommitted text. However, there are also user actions that start operations for which input operations should be ended, but that an input method cannot recognize. Saving the document containing the text is one such example. In these cases, the component has to explicitly call the input context's endComposition method.

Handling Additional Text Attributes

In addition to the input method highlight information, input methods may also attach other attributes to the text they send to a text component. These attributes may be useful information for the component. They may also improve the input method's performance if returned by the InputMethodRequest methods. For the latter reason, it is recommended that text components keep this attribute information around while the text is being edited, and return it with any text requested.

The AttributedCharacterIterator.Attribute class defines the following common attributes:

Future input methods written in the Java programming language may define additional attributes.

Creating Private Input Contexts

By default, one InputContext instance is created per Window instance, and this input context is shared by all components within the window's containment hierarchy. This reduces the number of instances created overall, and lets input methods combine information about all the text entered in this window (input methods often use information about previously entered text to improve their conversion accuracy). It means, however, that only one input operation is possible at any one time within a window, and that the text needs to be committed when moving the focus from one text component to another. If this is not desired, text components can create their own input context instances and override getInputContext to return them. A component that doesn't have its own input context uses the one used by its parent.

Selecting Input Methods

Text components can use the input context's selectInputMethod operation to select an input method for a given language or locale. This may be helpful, for example, if the user clicks in text that is written in that language, since it's likely that she wants to continue in the same language. Or, the text component may know that the application only allows text in a certain language to be entered.

Setting an Expected Character Subset

Text components can use the input context's setCharacterSubsets operation to tell input methods which characters can be meaningfully entered. For example, a database application may know that certain fields should only receive input in hiragana (one of the syllabic subscripts used in Japanese), another one only Latin characters, a third one any kind of characters. Passing on this information to input methods may allow the input methods to limit the range of characters that can be entered, or to switch to a different input mode that particularly supports the specified character subsets. However, applications cannot rely on this call having the desired effect, since this setting cannot be passed on to all host input methods - applications still need to apply their own character validation.


6. Changes to Other Frameworks

This chapter summarizes the changes made to other Java platform frameworks in JDK 1.2 in order to integrate the input method framework.

Text

The AttributedCharacterIterator interface is added as a standard way of communicating text information with attributes between frameworks. It lets text readers access text without having to know how the text is stored in the information provider. Attribute information may include font and style attributes as well as language tags and grammatical annotations.

Abstract Window Toolkit

The Window class is modified to create an initial input context for the window, and to dispose this input context for the window when the window is disposed. Explicitly disposing input contexts allows to free up the (often substantial) resources allocated by native input methods.

The Component class has new methods to handle input contexts and input method request handlers. Newly created Component instances other than windows initially share the input context of their containing window. The event handling in the Component class is modified to redirect incoming events to the input context associated with the component, and only pass them on to the component's listeners if they have not been consumed by an input method. Focus changes are communicated to the input context so it can activate or deactivate input methods.

The classes that have knowledge of event or listener classes are modified to handle the new InputMethodEvent and InputMethodListener classes.

The Graphics and Graphics2D classes have new drawString methods that accept AttributedCharacterIterator instances as input. Instances of the new TextLayout class can be constructed from AttributedCharacterIterator instances and used to draw text with input method highlights. Both ways of drawing text recognize input method highlights as range attributes for the text, and call back into the input method framework to convert abstract into concrete highlight styles.

Swing

The Swing text components in JDK 1.2 by default are active clients of the input method framework. That means, applications using these text components will use the on-the-spot input mode by default. Applications developers are still responsible for ending input operations when some other operation is started that requires the text to be committed. They may also use the input method framework's methods to create a private input context, to select an input method, or to set an expected character subset.


7. Future Enhancements

The following enhancements are planned for future JDK releases. Note that the order of enhancement items does not prioritize the functionality.

Input Method Engine API

This API enables the implementation of input method engines in the Java programming language. This API will also include a facility to manage multiple input methods in the Java application environment.

Using non-native Input Methods for TextArea and TextField

Currently, TextArea and TextField use platform peers and therefore the platform text input systems for text input. They can therefore not use input methods implemented in the Java programming language. Eventually, as Swing and AWT get merged, TextArea and TextField will become lightweight components. At that point, input through input methods implemented in the Java programming language will magically become available.

Multi-Layer Input Method Support

The framework has to be able to support multi-layer input methods. For example, text entered through a hand-writing recognition input method may need to also go through kana-kanji conversion before becoming committed text.