If you know of any tools that can convert TeX or MathML to a format useful for people who are blind, I’d like to get a list. This could include conversion to one or more Braille math codes (unfortunately there are many Braille math formats, even several within German). Or, it could be conversion to good text-to-speech output. I’m looking for both open source or proprietary.
For example, I’d like to convert the alt text in math images on Wikipedia.org. For example, see the article on the quadratic equation. An example of the markup used is:
<img class="tex" alt="x = \frac{-b \pm \sqrt {b^2-4ac}}{2a}"
src="http://upload.wikimedia.org/math/8/c/5/8c58ae2d322a33f3036800d96db0e91a.png">
And here is the image directly inserted into my blog, so that you don’t have to read the article to see it in action:
We need to talk about making images such as these accessible since MathML is still not widely used. The obvious reason is that MathML is not yet supported in IE. However, images are used for math a lot, and can have useful alt text as the Wikipedia math images used. WAI-ARIA even provides a math role to do this. Perhaps we can convince Wikipedia to add this role to math equations, since it helps with accessibility, although it does not validate. For now, someone could use AxsJax or Greasemonkey to add this role wherever class=”tex” exists. This will cause the image to be exposed as ROLE_EQUATION in Firefox’s MSAA support, so screen readers could recognize them. However, what should screen readers do with the TeX, if anything?
Maybe some screen reader users find the TeX from Wikipedia article readables without any changes? Or, perhaps a few tweaks to what their screen reader does with TeX would help? That would be useful information as well.
Links, quick tool reviews or any thoughts appreciated.
Access Garage 
Math accessibility is something I am very interested in and maths on the web is one thing where there is no really good solution I know of which currently exists. Currently I would say the LaTeX alt tag for images is the best solution for me, but I do know that this isn’t good for all (eg. those who use large print and may be want certain colour schemes, may be not all people know LaTeX although I think the math part isn’t too hard to learn, etc). MathML is meant to overcome many of those issues but it hasn’t been taken up widely, and while mathplayer seems to work reasonably well I don’t like how it provides access with a screen reader (spelling everything out in words, makes itt difficult to navigate an equation and isn’t good for reading on a Braille display). This is why LaTeX alt tags are best for me, I can use the screen reader and browser of my choice (Orca and firefox) and it isn’t too verbose to read on a Braille display.
As for tools, I have done some work of my own on converting LaTeX to British Braille BrlTex (http://brltex.sf.net). The original version worked well for what I actually needed, then I went on to try and generalise it and possibly bit off more than I could chew at the time, so that generalised version (not released only in SVN) could do with some rewriting. I don’t believe BrlTex will help for accessibility in the way you describe as it deals with document translation. Possibly a better tool for the task of LaTeX fragment translation would be the LaTeX-access project scripts (http://latex-access.sf.net). The scripts have been written in python and I believe that some intergration with jaws has been done, and because the scripts are python should work on linux (they did when I tried them some time ago) so potentially orca could make use of them, may be we just need a way for the screen reader to know when it is dealing with LaTeX source so that it can pass it through the LaTeX-access scripts. This still doesn’t help screen magnifiers though.
One thing also I feel is missing in math accessibility on the web is decent equation navigation. I believe the firevox plugin for firefox can handle mathml (I found its support limited but that may be trivial to add the extra characters, etc) but when I contacted the author he seemed to suggest that the firevox plugin knows about how the equation is split up and the reason firevox has no equation navigation is that he hasn’t added the user interface stuff (eg. hotkeys for it).
I will be watching this thread and if I can I would like to help where possible in improving math accessibility.
http://www.math.union.edu/~dpvc/jsmath/welcome.html
This jsMath library is interesting. It takes a fallback approach from MathML to images when the appropriate TeX or Unicode fonts are not available in a user’s browser. The user can also select their preferred rendering options using a floating button in the bottom right of the page (see http://www.math.union.edu/~dpvc/jsmath/examples/Henrici.html for an example).
I’m not claiming this library produces accessible renderings right now. Instead I’m saying if someone took this library and improved it to be accessible (e.g., use mathml when possible, proper alt tags when images get used, make the preference button focusable, etc.) it would serve as a nice ref. implementation of producing accessible math.
Michael, I see that your license is Reciprocal Public License. Can it be included in Orca (LGPL) or NVDA (GPL)?
What about expanding the project so it can also convert to other kinds of Braille math, such as Nemeth?
Have you ever looked into integrating your work into something like liblouis? This is currently used by Orca and NVDA. It would be a great start to get those screen readers to support math. I think that would help finally push the commercial vendors to build in math support.
Michael, also, latex-access looks interesting. Written in Python, just like Orca and NVDA. GPL. Seems like someone could make a project out of making Wikipedia math accessible in Orca and NVDA, and get something working pretty quickly.
Hmm, John Boyer tells me that liblouis can convert MathML to Nemeth or Marburg Braille. I hope he comments on this post.
Meanwhile, I see from the BrlTex page that it uses liblouis.
Then there is latex-access.
How come everyone isn’t working together?
BTW, anyone ever hear what happened to the MAVIS project? I can’t find it anymore.
So, John Boyer says liblouis can translate MathML to Nemeth or Marburg Braille. How about using that in conjunction with MathTran when we run into TeX? MathTran can convert TeX to MathML and vice-versa. See http://sourceforge.net/projects/mathtran/
It’s GPL and written in Python.
This means we could support TeX or MathML equally. For a MathML subtree, Mozilla could expose the subtee as a string.
Just brainstorming here. Orca or NVDA could pass a math equation to liblouis, BrlText or latex-access in the way they need it: MathML or TeX depending on the tool.
Of course, it’s doubtful that Orca and NVDA would want to touch so many different tools. It would be good to integrate the liblouis, BrlTex and latex-access efforts into 1 project. That way Orca and NVDA would just deal with 1 tool for Braille math conversion.
This doesn’t speak about conversion to TTS. I’m not sure what they’d need to do for that yet, although latex-access says it does something. That probably isn’t compatible with what NVDA and Orca need, but it could be useful to look at, since it’s also in Python.
This also doesn’t consider navigation through math, or authoring math, but it would definitely be a start.
Aaron,
For TTS equation reporting and navigation, it’s probably worth the time revisiting what T.V. Raman’s ASTER did for equations in TeX.
Here’s a relatively old link. He might have something newer for you.
http://www.cs.cornell.edu/home/raman/aster/aster-toplevel.html
Aaron, I don’t know fully about why projects aren’t working together, but here is why BrlTex is separate. BrlTex was initially written for my own use, I purely wanted British Braille, I was new to programming and was still getting to grips with python (at the time python being the only language I had any reasonable knowledge of). There was no other project which I felt I could build upon (LiblouisXML required learning about C and as it was I was struggling with understanding how I might just create a table for it to do British Braille, and all the notes I was dealing with were in LaTeX not MathML). LaTeX-access as I said deals with a different problem, I wanted to do documents and that’s what BrlTex does (I don’t know whether it could do a fragment, that would be determined by PlasTeX (http://plastex.sf.net)), where as LaTeX-access originally started as making jaws work better with LaTeX so had to deal with fragments.
Aaron, as for the licensing of BrlTex, I chose RPL as it (in my mind) prevents any development being done closed for someone’s commercial gain (eg. from what I understand with the GPL a Braille transcription service might take the GPL, modify it (improve it) and then use it for the transcriptions they charge people for, with the RPL that internal version would need to be published with source code). May be this is over protective.
As for intergrating my work with others (eg. NVDA, orca, other GPL code) I don’t know whether it would be possible with the RPL, but personally I wouldn’t have a problem so long as I could guarantee my code would be protected as I described, so if developers do wish to intergrate with it then I will discuss it. On the other hand, as I said BrlTex really needs rewriting and if the RPL would be a barrier then I could look at other licenses for a completely new project (the rewrite could be very close to a complete rewrite).
Aaron,
First off, there are many reasons for MathML not being widely seen on the web. I although not having MathML support built in to IE is definitely a problem, most people seem to be OK with using MathPlayer in IE. The bigger issue is that MathML hung its hat on XHTML, and IE does not support XHTML. This makes is MUCH harder to distribute pages with MathML in them if you want to them to work in most people’s browsers — you can’t even have javascript to alert people that they need MathPlayer if they use IE. Perhaps things will change if and when HTML5 comes out — MathML is currently in the HTML5 draft.
As for braille translators, you have already discovered liblious (which the next version of MathPlayer makes use of). In addition, there is UMCL (see http://www.ascience.eu/?q=en/finalConf/abstractArchambault) and in Europe, LAMBDA is used (www.lambdaproject.org) and like UMCL, it uses MathML as the intermediary language when doing conversions. If you are starting with TeX, there is Eitan Gurari’s (author of tex4ht) work (www.cse.ohio-state.edu/~gurari/proposal/nsf-02.html). The project might have died short of being ready for prime time — you’d have to ask him.
I think the biggest challenge you will have is getting screen readers to do anything special to work with you. Even open source projects such as NVDA don’t seem to care (I’ve sent them info on what it takes to add support for MathML accessibility in IE and offered to send them source code, but never got a response). Sadly, Math accessibility is not high up on most people radar screens, and even those who use math, don’t seem to care enough to make a stink about it to implementers. It’s really a shame since every day, in every school, every child takes a math class; lack of access to materials is a pretty big barrier facing children who are blind that want to learn math.
Hi Neil, thanks for the info. Why are there so many projects to translate MathML into Braille and so few of them seem to be working together? I find that totally confusing.
I agree that MathML requiring XHTML is a huge issue, and that support in HTML 5 could be a big boost.
FWIW, it’s not that NVDA doesn’t care about math. They’re just a couple of years old and still working on core things. For example, they’re currently adding support for Braille devices. I think it’s fair that general Braille support comes before math. There is a lot to do in a screen reader. Also, I believe that once one screen reader has built-in support for math, and there is content that uses that, other screen readers will feel the need to follow. Remember, we were able to get screen readers, web browser vendors and JavaScript toolkits to care about dynamic content. No one thought we’d get this far. I know we can do math too.
Speaking as an open source developer working on MathML-Nemeth interconversion, I believe that one reason that there are so many projects is that this is a VERY hard problem. The last revision of the Nemeth code was in 1972 so — if this were an easy problem — it likely would have already been solved.
And don’t forget that even in the case of contracted English literary braille, we still don’t have fully automatic accurate transcription of documents such as textbooks. (BTW, the “first computer-aided automatic braille production scheme” was developed by IBM in 1965! You can see a scan of the sample output here: http://www.dotlessbraille.org/bidissues.htm#intro )
Automatically converting MathML to Nemeth or vice versa seems about as difficult to me as automatically converting Fortran, say, to readable C or Java or converting LaTeX to HTML+MathML.
I think it is a good sign that there are so many different projects. Some are focusing on the 90% that takes 10% of the effort; others are focusing on especially difficult issues such as tables. Some of the non-US projects are attempting to handle multiple braille math codes.
Aaron, you asked what happened to MAVIS.
The MAVIS LaTeX to Nemeth translator has been incorporated into Duxbury’s DBT. The MAVIS Nemeth backtranslator was the basis for Logisoft’s new product, TransBraille:
http://www.logicalsoft.net/TransBraille.html
The MAVIS personnel have continued a wide variety of research into math access, much of which is summarized in this report:
Click to access mathaccsurvey.pdf
Here’s an update on Karshmer’s current research in this area:
http://www.usfca.edu/usfnews/news_stories/Art_Karshmer.html
The newest MAVIS-inspired research of which I am aware is this recent NSF award:
http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0754525
Thanks to everyone providing info and their thoughts. I found another one. Labrador, developed by the University of Linz, translates LaTeX to Marburg code.
Susan, having working on Braille software, I can agree that it’s complex. What I’m looking for as a first step is not hardcopy Braille formatting nor math editing capability. I just want screen reader users on the Web to be able to read some math.
The first step in ensuring accessibility of math on the Web is to require that the math be in MathML. The technical community already supports this for a number of reasons including that it is the basis for making math searchable.
IMHO, putting effort into support for the outdated use of images of equations, e.g. doing anything with alt text, would work against achieving accessible math on the Web.
Between MathPlayer and FireVox there is already fairly decent support for converting MathML to spoken math. And, as has already been pointed out, a number of researchers are working on conversion of MathML to one or more of the various braille math codes.
Susan, I think there is a point of debate there. I’m not saying I have all the answers but this reminds me of how we waited for XForms and XHTML. I’m not trying to go against people with more experience in this area but I do think there is another possible approach.
IMO, we need a 2-pronged solution. We can have a solution now. We start putting by supportng for both MathML and alt=”[LaTeX]” into screen readers and push the industry into taking math seriously. This also gets users something in their hands today. The LaTeX can either be supported natively or by something like a greasemonkey script that converts the LaTeX alt text into MathML.
Third party solutions are ok, but anything that requires a user to know about special software and install it generally has much poorer uptake. Math support just needs to work, and be built into the core screen reading and Braille embossing/translation tools that users install.
So, I think it’s great that MathPlayer does support MathML, but let me ask … how many more years will it be before MathPlayer/IE, Firefox and WebKit support MathML in HTML, and sites like Wikipedia actually start using it?
Why do we think that the presence of a short term solution will prevent authors from using MathML in the long term? In the long run, authors will still prefer use something where they don’t have to generate and serve image files.
So, I think we should take a similar approach to ARIA. We couldn’t wait for HTML to have native support for tree views, spreadsheets, sliders, progress meters and other advanced widgets. We know that at some point, it will, and authors will prefer to use that because it will just work better. But for now, authors are getting around the problem and users need access.
I think the open source math tools are important enough to have a mailing list for people that work on it to share information. Here it is:
http://groups.google.com/group/free-math
And here is the information I have so far, on math formats and translation libraries:
https://wiki.mozilla.org/Accessibility/Math_Accessibility
Coming back to this some time later, I have been thinking what could be done. If taking a two stage approach, what have we now and where would we like to go. Now I don’t know how screen readers interface with browsers, but I hope the following idea could fit.
Possibly a plugin for firefox and may be other browsers could be developed to be able to convert TeX to mathml and allow that mathml to be passed to a screen reader. For this I am thinking of snuggletex http://www.ph.ed.ac.uk/snuggletex (I don’t know how complete the TeX support is but it certainly can deal with TeX fragments such as those in a equation image’s alt text). The second part of the solution would be a library for converting mathml to Braille and speech, designed in such a way it could be used by various applications. May be this part would be python written so that it could be intergrated into NVDA and orca easily and with the COM scripting of window-eyes possibly there as well. This library possibly should take in strings of mathml and/or an object DOM and should be able to work on fragments as well eg. from the element inwards.
The advantages I can think of for the above is that having the TeX to mathml translation in a browser plugin then the screen reader/library part only needs to deal with mathml which is where I think we should be going. The library component for inclusion in screen readers might be useable else where eg. in a document Braille translator, and can be reused by many products.
The question of whether we should support TeX, well my view is it would be like webvisum (eg. webvisum with captures), we may not support people using what we are working around but unfortunately it is there and so we would like the access and so could do with a solution.
One disadvantage I can possibly think of is that object navigation of maths might be lost if the library returns strings from the input (eg. strings won’t contain enough information to jump to denominators, etc).
Any thoughts. What might help on the library translation side, eg. liblouisxml might do Braille (although I am unsure about the BAUK possibilities of liblouisxml) and what about the speech output?
My previous posting didn’t show the element I was meaning the translation library might work on because I included the less than and greater than signs around it and the blog took it as html. I was of course refering to the math tag. Hope that clears it up.