Firefox and OS X’s VoiceOver … Reading the Magic 8 Ball

Intro

Macs are quickly gaining popularity — they simply rock for mainstream users. Macs continue to make inroads for people with disabilities as well, via Apple’s proprietary built-in tools such as VoiceOver. However, while Firefox has recently made amazing strides on the Mac, Apple’s assistive technology tools are not compatible with it. Firefox users with disabilities are simply not included in the OS X party.

Apple users with disabilities should be able to enjoy the same level of Firefox support that they have on Windows and Linux. For example, VoiceOver users should be able to take advantage of accessibility features such as Web 2.0 accessibility support via WAI-ARIA. They should have the advantage of the WebVisum extension, which enables the community to share accessibility repairs across the web, performs OCR on images missing alt attributes, and even solves captchas. These are just a few of the accessibility benefits of Firefox, and users should be able to take advantage of them regardless of their chosen operating system.

Developers would also benefit from Firefox accessibility on OS X. If the Mozilla platform were accessible on all major desktops, developers would be able to implement accessible UI’s with XUL, HTML or JavaScript/ARIA widgets, while remaining insulated from the vastly different architectures on each desktop.

But how do we get there? The community needs to mull over some difficult decisions regarding how we implement Firefox accessibility on OS X.

Approaches to OS X Accessibility

We have three choices:

  1. Continue to invest in utilizing the native Mac OS X accessibility protocol: this protocol is comprised of two parts: NSAccessibility, which Cocoa applications use to expose their UIs and content, and the AX API, which assistive technologies use to access that information. Using these native APIs is how OS X’s built-in VoiceOver screen reader operates. Other OS X features such as Screen Zoom, Sticky Keys and the built-in dictionary can also take advantage of this support, and there is no doubt that other future tools will as well. Firefox 3 already has support for the Mac OS X accessibility protocol, thanks to Håkan Waara and a grant from the Mozilla Foundation. Unfortunately, VoiceOver compatibility issues remain unaddressed, and development is on hold. Håkan made multiple attempts from 2006-2008 to get assistance from Apple, and received replies but no follow-up.
  2. Utilize an open source solution on Macs (help improve a DOM solution such as Fire Vox or WebAnywhere, or port the NVDA or Orca screen reader). The DOM based solutions would still rely on VoiceOver for access to the actual OS X desktop and native applications. Porting an open source screen reader would either need to do the same, or require that those screen readers have a way of dealing with Universal Access.
  3. Prioritize the open source solution (option #2) but also continue to request help from Apple on the issues in native API mappings (option #1) for at least moderate support of Apple’s suite of built-in tools. This is the hybrid approach.

So what does our magic 8 ball say? Any of the approaches has pros and cons, and no matter which we chose, there is no escape from the profusion of work necessary to make that approach succeed.

However, one major difference is how far we can ultimately take each approach. The issue with using the Mac OS X accessibility protocol, as we have seen, is that the Mozilla accessibility team does not have adequate access to the VoiceOver team.

Firefox can’t just use NSAccessibility and hope that everything will just work with VoiceOver. In the history of making browsers hyper-accessible, close cooperation with the screen reader developers has always been required. We have already hit major snags just trying to make basic document content accessible. But, this is just the tip of the iceberg for Firefox, which pushes the forefront of accessibility. As Firefox pushes Web 2.0 accessibility forward, there is a need to work with the VoiceOver team to help define new ways of using the Mac OS X accessibility protocol to expose new capabilities introduced by WAI-ARIA or XUL. Properly exposing all the nuances for something like live regions requires working directly with the developers. In addition, it is necessary to influence development priorities such that quality support for the new semantics is incorporated into the actual VoiceOver software.

The web is like the wild west of technologies – it’s a chaotic, unpredictable situation at times, and is only growing in complexity. Having usable screen reader support always depends on addressing hundreds of annoying, detailed fixes in both the screen reader and the browser.

Painful isn’t a strong enough word to describe the situation when screen reader developers have another browser that matters more to them. I know this firsthand – we had this problem for years on Windows.

Linux

Contrast this with the situation on Linux — Orca is an open source screen reader and Firefox is a crucial browser under Gnome. Between Sun, volunteers such as the unsung heroine Joanie Diggs and Mozilla grantees, Firefox access keeps shooting forward under Orca, and we’re able to push the known limits of accessibility on that platform. The developers are easily reachable and we never reach “sorry, there are other projects we need to prioritize”. The Mozilla team reciprocates; it takes the job of fixing bugs that affect Orca very seriously.

Windows

Evaluating the situation on Windows for a moment, it wasn’t always good for Mozilla. In the beginning, it was extremely difficult for Firefox to be noticed, despite Mozilla doing everything it could to lay the red carpet out for Windows screen reader developers. Fortunately, Window-Eyes decided to support Firefox early in the game, even letting me work side by side with their developers in Fort Wayne, Indiana. However, Mozilla ultimately had to account for JAWS users being the majority and it has taken years to bring a satisfactory experience to those users.

Now things are getting interesting, because NVDA is also part of the race, and is moving forward with excellent Firefox support. At first NVDA did this because they generally care about other open source products and because Firefox simply had the best accessibility API support. The goals of both projects line up nicely, and the NVDA work is accelerating under a Mozilla Foundation grant.

What Succeeded?

What have we learned? We’re successful at accessibility when we become important enough to the screen reader developers to get their full attention. When we don’t get that attention, it’s just a slow, maddening road to unsatisfactory support.

Here are three ways to get screen reader developers to work closely on compatibility with your application:

  1. Become the main browser or application of that class on the given platform (IE on Windows, Firefox on Linux, and Safari/WebKit on OS X).
  2. Contract with a 3rd party screen reader vendor. This involves offsetting the vendor’s cost of doing the work. What you pay them, plus the income from upgrades they will sell due to support for your product, must be a fair amount more than the development costs. There may be side benefits for the vendor, such as the improvement of code they utilize elsewhere, which can factor in the price. On the other hand, hidden costs exist; for example, the difficulty of finding and managing an additional developer. Furthermore, because the revenue stream is short term, the vendor must consider the project’s aftermath. In the end, contracting with a third party vendor can be expensive, but it is effective. The need for vendors to offset costs is understandable. The market is small and the complexity involved in making screen readers work is significant and not generally appreciated. Moreover, as much as we wish otherwise, it is not easy to sell enough screen readers or upgrades because of Firefox support, to warrant extra development costs.
  3. Assist an open source screen reader project. This is an ideal approach to make Firefox accessible on OSX.

No options on OS X

Unfortunately, Mozilla does not currently have any of the above choices on OS X. The bottom line is that there is no incentive for Apple to spend resources supporting someone else’s browser, and there is no possibility for Mozilla to contribute to a solution. This is not a reflection on Apple — it’s just a realistic statement.

Mozilla’s goals of quality access to both tradition web content and Web 2 are in jeopardy if it depends on Apple to devote significant time to support those goals. And, in the case of advanced topics such as WAI-ARIA, Mozilla is usually in the lead. For leading edge accessibility there is often nothing available yet in WebKit for Mozilla to imitate.

Let’s take an example. Think about full featured access to online office suites. What incentive would there be for Apple to spend significant amounts of time making these complex pieces of software, which have never been made 100% accessible in any browser-screen reader combination, work cleanly in Firefox with VoiceOver?

Would a third party make it different? Probably. If Berkeley Access was still around making Outspoken for the Mac, we could pursue them as a potential partner to make things really work. (For those who knew Berkeley Access back in the day, they were a ragtag bunch and lots of fun at conferences.) Unfortunately, this is not the situation. Apple owns both the screen reader and the main browser for the platform, and they have enough problems dealing with their own products to worry about Mozilla’s needs.

What if VoiceOver was open source?

We can’t expect this of course; but we can hope. It would mean that the Mozilla Foundation could provide grants to individuals to fix Firefox-related bugs in VoiceOver.

Apple may be reluctant to open source VoiceOver. There is some setup required for Apple to do this, and to be honest open source may not gel well with the way the Apple accessibility team works comfortably.

Before people cry “open source zealot”, let me assert that this is a practical issue, both for Mozilla and for Apple. Apple would gain much more than it would lose. It has been shown that external developers are of great benefit to Apple. WebKit originally came from an open source project called KHTML. And, WebKit continues to benefit from being open source, for example via the recent development of an amazingly fast JavaScript engine. In the end, open source costs less to develop while both the product reach and quality increase.

You don’t believe that Apple accessibility will receive help from the open source community? People like Alp Toker are already contributing. Alp is working to make WebKit expose ATK/AT-SPI so that WebKit can be accessible on Linux.

So what are the actual disadvantages for Apple? VoiceOver can’t really be used anywhere but on Apple, so I don’t see a concern about third party redistribution. It is not even necessary to open source the entire codebase – we only need the parts that process the AX API. Anything else, such as modules that are an integral part of the OS, could remain proprietary (although ideally the architecture has a clean separation from OS internals).

The open accessibility community would love to work together as a team with Apple on improving accessibility for all. The question is, would the VoiceOver team also thrive on working with the community? There is no doubt that they would, as the community respects Apple’s leadership and goals for VoiceOver.

Let us imagine the possibilities if the community were allowed to help improve VoiceOver. The community at large could help add new features, support for new types of content, help fix bugs, improve performance or usability, and in Mozilla’s case, add support for advanced web applications. Apple would remain in control over what goes into the main distribution that comes with OS X.

Summary

For Mozilla and the community to help push forward accessibility on the future of the Web, we need to be able to get bugs fixed along the entire chain of technology, all the way to the user. The community needs to influence fixes in authoring tools, UI toolkits, testing tools, browsers and certainly in assistive technologies.

A healthy situation on OS X would present no barriers to Mozilla developing innovations in web accessibility. Either a third party should be responsible for screen reader customizations or, an open source screen reader is necessary. At this time, the Mozilla accessibility team would have to consider solid proposals to develop an alternative open source screen reader for OS X. However, that would be an unfortunate waste of resources. The cleanest solution for everyone involved would be if VoiceOver simply became an open source project. This would maximize the potential for beneficial collaboration. It is very clear that users would benefit the most from the scenario where VoiceOver is open source.

An open source VoiceOver would accelerate the evolution of accessibility for everyone’s benefit while keeping control in Apple’s hands. Come on Apple! Invite the community to join your efforts!

About these ads

18 Responses to Firefox and OS X’s VoiceOver … Reading the Magic 8 Ball

  1. Bogus Bogus says:

    As I understand it,any webkit accessibility bugs can be filed against the WebKit project, and any VoiceOver/Accessibility bugs or questions should be sent to Accessibility@apple.com. I believe these emails do get forwarded to VOiceoVer engineers, as well as other engineers working on Accessibility within Mac OS X.

  2. cmtalbert says:

    Great overview post, Aaron. I’m really happy to have a full outline of where each OS is w.r.t. accessibility. Thanks!

  3. aaronlev says:

    To Bogus Bogus:
    Perhaps my post didn’t clarify exactly how Hakan Waara attempted to get help from Apple. Over the last 2 years, he did in fact email back and forth with the Voice Over team, and bugs have filed.

    But that’s not really the point — you need to understand the complexity of developing a truly usable accessible Web 2.0 browsing solution. I think if you read the post carefully and try to understand our point of view, you’ll see we’re not just trying to lay blame. This is not just a matter of a few bugs here and there. Connecting two complex applications make new kinds of dynamic web content accessible requires new ideas, new semantics and a dedicated, coordinated effort between the two teams. If you think that’s realistic with the present situation, then we’ll just have to respectfully disagree with each other :)

  4. Eitan says:

    Maybe the political situation in Mac OS X is similar to early Windows days? Like you outlined, there used to be a time when no major vendor would give Mozilla the time of day. Maybe with just a bit of persistence we will get more Apple attention and more transparency. After all, accessibility is not usually where companies take pleasure in fierce competition.

    Also, good Firefox support on OS X will not cause Apple to sell less Macs, on the contrary.

  5. aaronlev says:

    Eitan, thanks for the comment. A major difference with the Windows situation is that Microsoft has never shipped a full built-in screen reading solution for Windows (Narrator is not intended to be a complete solution). Therefore, Application developers can pick which of the third party screen reader vendors they wish to approach if they want to advance the state of the art. For example, we were able to develop a new API (IAccessible2), which supports new technologies such as WAI-ARIA live regions, and then ask the various Windows screen reader vendors to support these endeavors. The fact that they are independent helps us. It would have been difficult or impossible to incent this development if there was only one vendor that developed everything: the platform, the main browser and the main screen reader.

    I don’t think Apple would ever ignore Mozilla’s accessibility questions intentionally or as a matter of competition. It’s just the nature of development. There’s always too much work, and the vendor’s own strategic direction must take precedence over working with third parties. That’s just business. There’s nothing wrong with that, but it also doesn’t help us push Mozilla’s Web 2.0 accessibility agenda forward.

  6. Eitan says:

    Aaron, about ARIA and live regions:
    Webkit is on the path to support those standards, Safari respectively. So Apple will need to provide API and VoiceOver enhancements to support their own browser, in one way or another. But I guess that means that we cannot beat them to it, and we would have to wait.

    The fact that it is all in-house at Apple is pretty intimidating, they provide a tight package that consumers love. But this entire conversation is about improving their own platform, so I hope they do the right thing.

  7. aaronlev says:

    Eitan, that’s right. They are on that path, and may eventually support live regions and the rest of WAI-ARIA for WebKit, but it remains to be seen what they do. There are no guarantees.

    Ultimately we need to be able to support authoring environments on the web, diagrams, custom widgets, math and compound documents.

    We really have no idea how far Apple’s commitment will go. Meanwhile, uses of dynamic web techniques will continue to become more interesting.

  8. Pratik Patel says:

    response to the first comment and several general points.

    Apple has really shown a lack of response to the community feedback until a significant economic incentive is placed in front of their face. Take ITunes/ITunesU as a perfect example of this. Apple agressively marketed the ITunes U product to colleges and universities and continues to do so despite having significant interface accessibility challenges on WIndows. While ITune client being almost accessible on OSX, it lack similar access on the Windows Side. Promises have come from Apple on making that interface accessible one component at a time. But, such promises did not come until significant economic pressure was brought on Apple. Many people forget that Apple’s renewed focus on accessibility with Voice Over is the result of school districts stopping purchases due to the lack of access – and I even recall a law suit.

    That having been said, however, convincing Apple to work with the FF dev community is going to be a challenge. Partly this challenge is related to FF’s role in somewhat undermining it’s own Browser. The more Apple can do to keep users, with or without disabilities, to it’s own browser, it’s better for the brand and numbers. And, this is pure speculation, but by working with FF, Apple will not devote internal resources to what it has convinced itself its priorities are.

    I would argue a different approach to the solution of FF access on the OSX platform. Until/unless Apple shows a serious commitment to making FF and other open source products more accessible, porting NVDA to OSX seems like an approach that could bring more benefits to consumers. …

  9. [...] to arise. Especially when they offer such sexy and irresistible technology. Aaron Leventhal asks an important one, regarding web accessibility. « Introducing [...]

  10. Vadi says:

    It should work, since Apple only uses oss when it’s profitable for their platform, and such is the case here.

  11. [...] Leventhal: Firefox and OS X’s VoiceOver … Reading the Magic 8 Ball. OSX has some nice accessibility tools like their VoiceOver screen reader. Unfortunately [...]

  12. [...] Firefox and OS X’s VoiceOver … Reading the Magic 8 Ball  Aaron Leventhal details the barriers faced in getting the Firefox browser to work with [...]

  13. Phill says:

    hmm, reading the posts from Aaron, Pratik, and others seems to put much of the problem on Apple’s priorities. Apple is both a single company and a bunch of separate dev teams. Apple has separate pressures (priorities) on OSX, VoiceOver, Safari/WebKit, and all the apps. As Aaron points out, all the pressure points are at the intersections or support points between the platform, browser, and screen reader – ignoring the authoring tools and other support stuff for a moment. Even if we get Apple to open source VoiceOver or port NVDA to OS X, Apple’s OS X team still has its pressures and priorities which are not going to magically go away or be reduced. One browser and one screen reader on the OS X platform should REDUCE the bugs and need for co-development, but it doesn’t change the priority – as Pratik observed. What’s the motivation to change the priority? Reducing the workload isn’t in my opinion a way to increase the priority, it may actually backfire and reduce the resources in that area to be able to accomplish even less. I agree with the approach of making VoiceOver open source which seems more efficient than porting NVDA in the short term (maybe in the long term they could be co-developed). But the real issue is to raise the priority of accessibility at APPLE specifically related to OS X, VoiceOver, and Safari/WebKit to keep up with the open source capabilities on the other platforms – or join forces and go open source on VoiceOver and better support Firefox. Well, how do we raise the priority? $6 million would go a long way towards that!

  14. [...] within the Firefox developer team. He recently wrote a detailed article about the problems with Firefox and OS X’s VoiceOver and why these are so difficult to resolve. There are both technical and political difficulties to [...]

  15. The story goes that Apple has per-developer technology evangelists who care if you do something cool with the developer API they are tasked with evangelizing. Also, Mac developers say that when an ISV becomes ‘important’ enough, Apple assigns someone to manage developer support relationship with the ISV.

    One would think that Firefox as a product and Mozilla Corporation and IBM as companies would be ‘important’ enough to merit the attention of a VoiceOver developer evangelist who could poke the VoiceOver team for answers. I wonder if individual Mozilla contributors haven’t seemed to have enough clout from the point of view of Apple developer relations and if being more corporate about the liaison would better match the way Apple runs developer relations.

  16. AlastairC says:

    Presumably it would be a lot of effort for Apple to match the accessibility API they have for Cocoa apps with Mozilla’s framework?

    Would it not make more sense for Mozilla to concentrate on the version of Firefox that aims to be a native Apple app? (Camino)

    The interface for Camino works fine with Voiceover (from a quick test), you just can’t get to the web page!

  17. aaronlev says:

    Alastair, it’s the always the web content that’s hardest to make accessible.

    That said, the Mozilla team is planning to give VoiceOver compatibility another shot this year.

  18. david says:

    I agree with:
    “Would it not make more sense for Mozilla to concentrate on the version of Firefox that aims to be a native Apple app? (Camino)”.

    Regarding porting nvda to osx, I would not use a screen reader just for one app so there would have to be highly compelling reasons for me to use and support another screen reader on the Mac besides ff.

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: