Windows Automation API 3.0 Overview

While general accessibility requirements (such as font colors in UI rendering) are important, programmatic access to the graphical user interface (GUI) is a crucial element to improving accessibility. On the Windows® operating system, Microsoft® Active Accessibility® and User Interface (UI) Automation support this programmatic access. This article provides a quick overview of Windows Automation API 3.0 featured in Windows 7.

Windows Automation API 3.0: a Bit of History

Today, the Windows operating system offers two application programming interfaces (API) specifications for user interface accessibility and software test automation. The legacy API, Microsoft Active Accessibility, was introduced to Windows 95 as a platform add-on in 1996. The new API is a Windows implementation of the User Interface Automation specification called UI Automation. UI Automation was introduced in Windows Vista® and .NET Framework 3.0.

Although the two technologies are different, the basic design principles are similar. Both expose the UI object model as a tree hierarchy rooted at the desktop. Microsoft Active Accessibility represents individual UI elements as accessible objects, and UI Automation represents them as automation elements. Both refer to the accessibility tool or software automation program as the client. However, Microsoft Active Accessibility refers to the application or control offering the UI for accessibility as the server, while UI Automation refers to this as the provider.

Microsoft Active Accessibility offers a single COM interface with a fixed, small set of properties. UI Automation offers a richer set of properties, as well as a set of extended interfaces called control patterns to manipulate automation elements in ways Microsoft Active Accessibility cannot.

While UI Automation previously had both managed and unmanaged API for providers, the original release had no unmanaged interfaces for clients. With Windows Automation API 3.0, you can finally write UI Automation clients entirely in unmanaged code.

The new API also provides support for transitioning from Microsoft Active Accessibility servers to UI Automation providers. The IAccessibleEx interface enables legacy Microsoft Active Accessibility servers to add support for specific UI Automation patterns and properties without rewriting their whole implementation. The specification also allows in-process Microsoft Active Accessibility clients to access UI Automation provider interfaces directly, rather than through UI Automation client interface.

The ecosystem of Windows automation technologies, now called Windows Automation API, includes classic Microsoft Active Accessibility and Windows implementations of the UI Automation specification. At Microsoft, the UI Automation specification is implemented on Windows Vista, Windows Server 2008, Windows Presentation Foundation (WPF), XPS Viewers, and many other upcoming Microsoft products. Windows 7, Windows Internet Explorer 8, and Silverlight 2.0 are joining the pack soon.

The Architecture: Microsoft Active Accessibility, UI Automation, and Interoperability

The goal of Microsoft Active Accessibility is to expose basic information about custom controls such as control name, location on screen, and type of control, as well as state information such as visibility and enabled/disabled status. The UI is represented as a hierarchy of accessible objects; changes and actions are represented as WinEvents. The following components comprise the Microsoft Active Accessibility architecture:

• Accessible Object-A logical UI element (such as a button) that is represented by an IAccessible COM interface and an integer ChildID.
• WinEvents-An event system that enables servers to notify clients when an accessible object changes.
• OLEACC.dll-A run-time dynamic-link library that provides the Microsoft Active Accessibility API and the accessibility system framework.

For Microsoft Active Accessibility, the system component of the accessibility framework (OLEACC.dll) helps the communication between accessibility tools and applications (Figure 1). The applications (Microsoft Active Accessibility servers) provide UI accessibility information to tools (Microsoft Active Accessibility clients), which interact with the UI on behalf of users. The code boundary can be a programmatic or process boundary.

The goal of UI Automation is similar but broader, as described later in this article. From an architecture point of view, UI Automation loads the UI Automation Core component into both the accessibility tools’ and applications’ processes (Figure 2). This component manages cross-process communication and provides higher level services. This core component enables bulk fetching or caching of properties, which improves the cross-process performance over Microsoft Active Accessibility implementation.

Interoperability between Microsoft Active Accessibility-based and UI Automation-based Applications

The UIA-to-MSAA Bridge enables Microsoft Active Accessibility clients to access UI Automation providers by converting the UI Automation object model to a Microsoft Active Accessibility object model (Figure 3). Similarly, the MSAA-to-UIA Proxy (Figure 4) translates Microsoft Active Accessibility-based server object models for UI Automation clients.

Now with IAccessibleEx, you can also improve existing Microsoft Active Accessibility server implementations by adding only required UI Automation object model information. The MSAA-to-UIA Proxy takes care of incorporating the added UI Automation object model.

Limitations of Microsoft Active Accessibility

Microsoft designed the Microsoft Active Accessibility object model about the same time as Windows 95 released. The model is based on “roles” defined a decade ago, and you cannot support new UI behaviors or merge two or more roles together. There is no text object model, for example, to help assistive technologies deal with complex Web content.

Another limitation involves navigating the object model. Microsoft Active Accessibility represents the UI as a hierarchy of accessible objects. Clients navigate from one accessible object to another using interfaces and methods available from the accessible object. Servers can expose the children of an accessible object with properties of the accessible object or with the IEnumVARIANT COM interface. Clients, however, must be able to deal with both approaches for any server. This ambiguity means extra work for client implementers, and the complexity can contribute unforeseen problems of server implementations.

Just as important is the inability to extend Microsoft Active Accessibility properties or functions without breaking or changing the IAccessible COM interface specification. The result is that you cannot expose new control behavior or property through the object model. The object model tends to be both static and stagnant.

By: Masahiko Kaneko

Masahiko Kaneko is a Senior Program Manager on the User Interface Platform Team in the Windows Experience Division. A program manager in accessibility at Microsoft for more than 10 years, he has been involved with several releases of the Windows Operating System as well as many other Microsoft products.

Masahiko enjoys road biking, XC skiing, music and movies in his free time but most of it is now dedicated to his first son who is turning 18 months in September.

mkaneko@microsoft.com