xmlhack: Microsoft heads performance rankings for XSLT

In benchmarks published yesterday on XML.com, MSXML 3 was significantly ahead on performance, the nearest competitor being James Clark's XT processor.

The tests, conducted by DataPower, show Java-based processors ahead of their C/C++ counterparts (with the exception of MSXML). XT, the grandaddy of XSLT processors, outperformed all but Microsoft. Oracle's was the next-best performing processor in production (Sun's XSLT compiler performed well, but is only alpha technology).

In their analysis DataPower observed that while each processor tended to excel in certain areas, coverage of the entire XSLT 1.0 specification was in general incomplete. According to the report, Michael Kay's Saxon processor offers the best coverage of XSLT 1.0 features, as well as very respectable performance.

The strong showing offered by open source processors is a confirmation of the key role such software has played in XML, and in particular that played by James Clark.

Links: explanation of the benchmarks, and full breakdown of results.

1. There was an error in some of the drivers, it did not affect the ultimate conclusions; it was explained and corrected in the article two days later.

2. XSLTMark was developed with help and cooperation from the XML community, it is open-source and it has been available for download since October 2000.

3. Indeed, it is its open-source nature that allowed Dan Helmsman & Mike Kay to find the bug mentioned in point 1 above!

4. XSLTMark actually uses three tests of Kevin Jones' own xslbench benchmark and 37 other testcases.

5. Note the DataPower's XML acceleration technology is neither included in the results we published nor hidden from the public. We do have an interest in there being an open-source comprehensive XSLT benchmarking system.

(David, it might be of interest to declare in your post that you work for Lotus, who have a significant interest in the Xalan processor).

Yes, on reflection I think I should have noted that DataPower developed XSLTMark as part of the development of their own processor. The fact that they released the source, and that it has been implemented in at least one other language by an independent developer already, gave me some assurance that they were not playing games.

-- Edd Dumbill, Editor XML.com and XMLhack.com

Journalists, probe more deeply!

If you go to http://www.datapower.com and follow the "XSLJIT" link, you'll see that this company intends to compete in the XSLT processing space!

While this does not invalidate their benchmarking methodology per se, it should be noted.