Re: [saxon] Saxon 9.5 is available

On 19/04/2013 21:57, Michael Kay wrote:
> Some of you will have already picked up from Twitter that Saxon 9.5 is now available. Or you may have seen that we've already had a bug reported on this list. It might sound odd, but that gives me great pleasure. The quality of Saxon owes so much to the users who do amazing things with it, and not only tell us when it doesn't work, but produce simple test cases that make the problems easy to fix. Having users who download the software on day 1 to test it is a great asset. So a big thank you to all our regular and loyal power users.
>
> ......
>
I've managed to convert my document layout system (some 30 stylesheets,
half-a-dozen extension functions) from 3.0 under 9.4.0.7 to 3.0 under
9.5.0.1 with only a few minor problems:
1. Tightening to the XSLT 'spec' required some minor XSLT code changes
- all alerted by the compiler or runtime: serialisation paremeters
now required on a very specific element, context-item on
xsl:evaluate..... (Some of the documentation will eventually need to
follow .... I'll try to record them.)
2. Changes in the System Programming Interfaces (i.e. internal
structures) were pretty straighforward when you found out about
them. Axis -> AxisInfo was simple, and the dropping of fingerprint
access to attributes in NodeInfo didn't cause much problem. The new
Sequence seems easier for the extension call interface, but the
example of the full interface in the documentation
(http://www.saxonica.com/documentation/index.html#!extensibility/integratedfunctions/ext-full-J
<http://www.saxonica.com/documentation/index.html#%21extensibility/integratedfunctions/ext-full-J&gt;)
will need altering some time ;-)
3. There are a number of extra nice 'warning' touches that have
appeared in warning about conditions that are 'useless', e.g. mode
#current in functions (I have an example that I used to illustrate
XSLT code examining and reporting on itself...).
4. I have two (XSLT compile/execution) bugs that seem to have appeared
in the change (i.e. all worked well under 9.4.0.7). As usual the
problem is to try to isolate the bug manifestation into a small
example from the extensive operational context within which it appears.
* An NPE (XSLNextIteration.java:135) in the compilation of an
xsl:iterate within a function that is itself called recursively
(laying out a graphical tree, if you're interested)
* An ArrayOutOfBounds: "local variable $mid uses slot 2 but only 1
slots are allocated on the stack frame".
(LocalVariableReference.java:105) within the runtime of the same
function (but not inside the xsl:iterate - this does not depend
on the presence of the iterate).
When I've tracked down simplified examples of the two bugs I'll post to
the bug tracker....
As you said Mike - treading on the bleeding edge. It took me about a day
to do the complete conversion, /pace/ the two bugs, which I can work
around for the present.
--
John Lumley MA PhD CEng FIEE
john@...
+44 (0)1275 845284
mobile: +44 7765 717073

Thread view

Some of you will have already picked up from Twitter that Saxon 9.5 is now available. Or you may have seen that we've already had a bug reported on this list. It might sound odd, but that gives me great pleasure. The quality of Saxon owes so much to the users who do amazing things with it, and not only tell us when it doesn't work, but produce simple test cases that make the problems easy to fix. Having users who download the software on day 1 to test it is a great asset. So a big thank you to all our regular and loyal power users.
This is the first functionality release for almost 18 months, and you can be assured that this isn't because we've been spending our time on other things.
There's a large number of changes, all carefully documented. Some of them are relatively small things in response to user feedback, such as improving the diagnostics from the schema validator; some are continued implementation of the evolving W3C specs (XQuery 3.0 and XPath 3.0 are now essentially finished, and Saxon 9.5 is 100% conformant as far as we know - we pass every single one of the 23000 or so test cases). Some changes are more strategic, like the continued improvement in streaming capability for large documents, and new in this release, automatic support for parallel execution of stylesheets that have multiple input or multiple output documents.
The product is now pretty big (around 300K lines of code, I think), and I'm convinced that when you are managing such a large code base, you have to work hard on preventing structural decay. So there are internal changes that some of you will notice if you poke around the innards, for example the introduction of a new Sequence class to model all XPath values, replacing the old clumsy Value/ValueRepresentation split.
Bytecode generation which we introduced in 9.4 has held up pretty well. I was worried it might be a source of bugs and support hassle, but it has proved very reliable (well done O'Neil Delpratt, since this was largely his work). In this release, my two anxieties are probably the introduction of multi-threading for xsl:result-document (simply because multi-threading is so difficult to test thoroughly, and to debug if there are problems), and the introduction of a new regular expression engine. Both are indicators that we're not prepared to sit back on our laurels and take a low-risk "leave well alone" strategy.
Multithreading has always been part of the promise of using declarative languages, and it's increasingly important to take advantage of the fact that the hardware is now always multi-core. It's exciting to be finally delivering on this. Publishing workflows that generate hundreds or thousands of output documents from a single stylesheet will be the ones that benefit most.
As for the new regular expression engine (which is derived from Apache Jakarta), the existing approach which translated regular expressions from the XPath regex dialect to the Java regex dialect was getting increasingly cumbersome. Java hasn't moved forward in terms of Unicode support, and we wanted to be in more control of our destiny. The new regex engine is probably less code than the translator we were using before. I doubt it will deliver an immediate performance boost, but it does give us a lot more opportunity to introduce optimizations that take advantage of the way regular expressions are used in XSD and XPath.
I know there will be a few bugs in the first month or two, there always are; I know that although we are now running probably half a million tests before we release, it's not nearly enough. Those of you with mission-critical applications will probably want to hold back and exercise caution, and quite rightly so, because we know how dependent you are on the software's reliability. For those of you more keen to stay in front of the curve, please do try it out and give us your feedback, and as always we will do our very best to respond.
I hope you like the new Saxon-CE based documentation system. It was originally built by Phil Fearon, but I've been doing work on it over the last couple of weeks, and I have to say I've found it a very positive experience developing this application. I can't imagine ever writing something like this in Javascript, though I know people do.
Oh, and finally: I'm interested in expanding the team. If you're interested and qualified, or know someone who is, then come and talk. We're based in Reading, UK, and I do think that co-location is important to create an effective team.
Michael Kay
Saxonica

On 04/19/2013 04:57 PM, Michael Kay wrote:
> Some of you will have already picked up from Twitter that Saxon 9.5 is now available. Or you may have seen that we've already had a bug reported on this list. It might sound odd, but that gives me great pleasure. The quality of Saxon owes so much to the users who do amazing things with it, and not only tell us when it doesn't work, but produce simple test cases that make the problems easy to fix. Having users who download the software on day 1 to test it is a great asset. So a big thank you to all our regular and loyal power users.
>
>
In my (lux) tests, after accounting for API changes, all is well. I see
some new optimizations, and one that is no more, but all results are
correct for me so far. Although I'm not really exercising the newly
changed features, so it's probably not much of a test. Still, looks
like a solid release - thank you! I do hope you (or O'Neill?) will be
deploying the latest Saxon-HE to maven central, too.
-Mike Sokolov

Hi Michael,
Just to inform you and the mailing list that I have have deployed
Saxon-HE 9.5 to maven central, which should be available shortly.
Kind regards,
O'Neil
On 21/04/13 00:01, Michael Sokolov wrote:
> On 04/19/2013 04:57 PM, Michael Kay wrote:
>> Some of you will have already picked up from Twitter that Saxon 9.5 is now available. Or you may have seen that we've already had a bug reported on this list. It might sound odd, but that gives me great pleasure. The quality of Saxon owes so much to the users who do amazing things with it, and not only tell us when it doesn't work, but produce simple test cases that make the problems easy to fix. Having users who download the software on day 1 to test it is a great asset. So a big thank you to all our regular and loyal power users.
>>
>>
> In my (lux) tests, after accounting for API changes, all is well. I see
> some new optimizations, and one that is no more, but all results are
> correct for me so far. Although I'm not really exercising the newly
> changed features, so it's probably not much of a test. Still, looks
> like a solid release - thank you! I do hope you (or O'Neill?) will be
> deploying the latest Saxon-HE to maven central, too.
>
> -Mike Sokolov
>
> ------------------------------------------------------------------------------
> Precog is a next-generation analytics platform capable of advanced
> analytics on semi-structured data. The platform includes APIs for building
> apps and a phenomenal toolset for data science. Developers can use
> our toolset for easy data analysis & visualization. Get a free account!
> http://www2.precog.com/precogplatform/slashdotnewsletter
> _______________________________________________
> saxon-help mailing list archived at http://saxon.markmail.org/
> saxon-help@...
> https://lists.sourceforge.net/lists/listinfo/saxon-help
>
--
O'Neil Delpratt
Software Developer, Saxonica Limited
Email: oneil@... <mailto:oneil@...>
Tel: +44 118 946 5894
Web: http://www.saxonica.com
Saxonica Community Site: http://dev.saxonica.com Saxonica Bug tracking
System: https://saxonica.plan.io/

On 19/04/2013 21:57, Michael Kay wrote:
> Some of you will have already picked up from Twitter that Saxon 9.5 is now available. Or you may have seen that we've already had a bug reported on this list. It might sound odd, but that gives me great pleasure. The quality of Saxon owes so much to the users who do amazing things with it, and not only tell us when it doesn't work, but produce simple test cases that make the problems easy to fix. Having users who download the software on day 1 to test it is a great asset. So a big thank you to all our regular and loyal power users.
>
> ......
>
I've managed to convert my document layout system (some 30 stylesheets,
half-a-dozen extension functions) from 3.0 under 9.4.0.7 to 3.0 under
9.5.0.1 with only a few minor problems:
1. Tightening to the XSLT 'spec' required some minor XSLT code changes
- all alerted by the compiler or runtime: serialisation paremeters
now required on a very specific element, context-item on
xsl:evaluate..... (Some of the documentation will eventually need to
follow .... I'll try to record them.)
2. Changes in the System Programming Interfaces (i.e. internal
structures) were pretty straighforward when you found out about
them. Axis -> AxisInfo was simple, and the dropping of fingerprint
access to attributes in NodeInfo didn't cause much problem. The new
Sequence seems easier for the extension call interface, but the
example of the full interface in the documentation
(http://www.saxonica.com/documentation/index.html#!extensibility/integratedfunctions/ext-full-J
<http://www.saxonica.com/documentation/index.html#%21extensibility/integratedfunctions/ext-full-J&gt;)
will need altering some time ;-)
3. There are a number of extra nice 'warning' touches that have
appeared in warning about conditions that are 'useless', e.g. mode
#current in functions (I have an example that I used to illustrate
XSLT code examining and reporting on itself...).
4. I have two (XSLT compile/execution) bugs that seem to have appeared
in the change (i.e. all worked well under 9.4.0.7). As usual the
problem is to try to isolate the bug manifestation into a small
example from the extensive operational context within which it appears.
* An NPE (XSLNextIteration.java:135) in the compilation of an
xsl:iterate within a function that is itself called recursively
(laying out a graphical tree, if you're interested)
* An ArrayOutOfBounds: "local variable $mid uses slot 2 but only 1
slots are allocated on the stack frame".
(LocalVariableReference.java:105) within the runtime of the same
function (but not inside the xsl:iterate - this does not depend
on the presence of the iterate).
When I've tracked down simplified examples of the two bugs I'll post to
the bug tracker....
As you said Mike - treading on the bleeding edge. It took me about a day
to do the complete conversion, /pace/ the two bugs, which I can work
around for the present.
--
John Lumley MA PhD CEng FIEE
john@...
+44 (0)1275 845284
mobile: +44 7765 717073

Thanks for the feedback. Most of these points are fairly minor but the two bugs are certainly something we want to know about. Don't worry too much if you can't make the repro simple, so long as we can reproduce the problem, we'll get to the bottom of it.
I don't like that "only 1 slots are" in an error message. I'm usually obsessive about making them grammatical, even if it's a condition that should never arise!
Michael Kay
Saxonica
On 25 Apr 2013, at 09:10, John Lumley wrote:
> On 19/04/2013 21:57, Michael Kay wrote:
>> Some of you will have already picked up from Twitter that Saxon 9.5 is now available. Or you may have seen that we've already had a bug reported on this list. It might sound odd, but that gives me great pleasure. The quality of Saxon owes so much to the users who do amazing things with it, and not only tell us when it doesn't work, but produce simple test cases that make the problems easy to fix. Having users who download the software on day 1 to test it is a great asset. So a big thank you to all our regular and loyal power users.
>>
>> ......
>>
> I've managed to convert my document layout system (some 30 stylesheets, half-a-dozen extension functions) from 3.0 under 9.4.0.7 to 3.0 under 9.5.0.1 with only a few minor problems:
>
> Tightening to the XSLT 'spec' required some minor XSLT code changes - all alerted by the compiler or runtime: serialisation paremeters now required on a very specific element, context-item on xsl:evaluate..... (Some of the documentation will eventually need to follow .... I'll try to record them.)
> Changes in the System Programming Interfaces (i.e. internal structures) were pretty straighforward when you found out about them. Axis -> AxisInfo was simple, and the dropping of fingerprint access to attributes in NodeInfo didn't cause much problem. The new Sequence seems easier for the extension call interface, but the example of the full interface in the documentation (http://www.saxonica.com/documentation/index.html#!extensibility/integratedfunctions/ext-full-J) will need altering some time ;-)
> There are a number of extra nice 'warning' touches that have appeared in warning about conditions that are 'useless', e.g. mode #current in functions (I have an example that I used to illustrate XSLT code examining and reporting on itself...).
> I have two (XSLT compile/execution) bugs that seem to have appeared in the change (i.e. all worked well under 9.4.0.7). As usual the problem is to try to isolate the bug manifestation into a small example from the extensive operational context within which it appears.
> An NPE (XSLNextIteration.java:135) in the compilation of an xsl:iterate within a function that is itself called recursively (laying out a graphical tree, if you're interested)
> An ArrayOutOfBounds: "local variable $mid uses slot 2 but only 1 slots are allocated on the stack frame". (LocalVariableReference.java:105) within the runtime of the same function (but not inside the xsl:iterate - this does not depend on the presence of the iterate).
> When I've tracked down simplified examples of the two bugs I'll post to the bug tracker....
>
> As you said Mike - treading on the bleeding edge. It took me about a day to do the complete conversion, pace the two bugs, which I can work around for the present.
> --
> John Lumley MA PhD CEng FIEE
> john@...
> +44 (0)1275 845284
> mobile: +44 7765 717073
>
> ------------------------------------------------------------------------------
> Try New Relic Now & We'll Send You this Cool Shirt
> New Relic is the only SaaS-based application performance monitoring service
> that delivers powerful full stack analytics. Optimize and monitor your
> browser, app, & servers with just a few lines of code. Try New Relic
> and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr_______________________________________________
> saxon-help mailing list archived at http://saxon.markmail.org/
> saxon-help@...
> https://lists.sourceforge.net/lists/listinfo/saxon-help

On 25 April 2013 09:24, Michael Kay <mike@...> wrote:
>
> Thanks for the feedback. Most of these points are fairly minor but the two
> bugs are certainly something we want to know about. Don't worry too much if
> you can't make the repro simple, so long as we can reproduce the problem,
> we'll get to the bottom of it.
>
> I don't like that "only 1 slots are" in an error message. I'm usually
> obsessive about making them grammatical, even if it's a condition that
> should never arise!
>
> Michael Kay
> Saxonica
For which thanks. Saxon clear error reporting has always been one of
its strengths.
regards
--
Dave Pawson
XSLT XSL-FO FAQ.
Docbook FAQ.
http://www.dpawson.co.uk

On 04/22/2013 10:27 AM, O'Neil Delpratt wrote:
> Hi Michael,
>
> Just to inform you and the mailing list that I have have deployed
> Saxon-HE 9.5 to maven central, which should be available shortly.
>
> Kind regards,
> O'Neil
>
O'Neil - I just checked
http://mvnrepository.com/artifact/net.sf.saxon/Saxon-HE, but I don't see
9.5 there yet; it seems something may have slipped 'twixt cup and lip
-Mike

Hi Michael,
Just to update you on this issue. Please see below the reply from sonatype:
[https://issues.sonatype.org/browse/OSSRH-3083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=187212#comment-187212 ]
Joel Orlina commented on OSSRH-3083:
------------------------------------
Maven Central has no formal relationship with mvnrepository.com. I can't speak to the contents of that site or how often that site is updated. If an artifact is search-able on search.maven.org, that guarantees that it's accessible via Maven and other build tools which reference repo1.maven.org.
On 10/05/2013 13:05, Michael Sokolov wrote:
> Thanks O'Neil -
> in fact I'm able to resolve the dependencies when I compile, so all
> is well, aside from the listing on the maven site, I guess
>
> -Mike
>
> On 5/10/2013 4:26 AM, O'Neil Delpratt wrote:
>> Hi Michael,
>>
>> I am not quite sure why Saxon-HE 9.5 has not found it's way on
>> mvnrepository. I have raised this issue with sonatype.
>>
>> When I search for 9.5 with the following URL it is present:
>>
>> http://search.maven.org/#artifactdetails%7Cnet.sf.saxon%7CSaxon-HE%7C9.5.0.1%7Cjar
>>
>> There seems to be some syncing issue.
>>
>> regards,
>>
>> O'Neil
>>
>> On 10/05/13 03:46, Michael Sokolov wrote:
>>> On 04/22/2013 10:27 AM, O'Neil Delpratt wrote:
>>>> Hi Michael,
>>>>
>>>> Just to inform you and the mailing list that I have have deployed
>>>> Saxon-HE 9.5 to maven central, which should be available shortly.
>>>>
>>>> Kind regards,
>>>> O'Neil
>>>>
>>> O'Neil - I just checked
>>> http://mvnrepository.com/artifact/net.sf.saxon/Saxon-HE, but I don't
>>> see 9.5 there yet; it seems something may have slipped 'twixt cup
>>> and lip
>>>
>>> -Mike
>>
>>
>> --
>> O'Neil Delpratt
>> Software Developer, Saxonica Limited
>> Email: oneil@... <mailto:oneil@...>
>> Tel: +44 118 946 5894
>> Web: http://www.saxonica.com
>> Saxonica Community Site: http://dev.saxonica.com Saxonica Bug
>> tracking System: https://saxonica.plan.io/
>
> No virus found in this message.
> Checked by AVG - http://www.avg.com <http://www.avg.com&gt;
> Version: 2013.0.3336 / Virus Database: 3162/6312 - Release Date: 05/09/13
>