Re: [saxon] importing an XML Schema

>
> Thanks a lot, running as a servlet does indeed improve
> performance *by a lot*.
> There are although a couple of questions I came across: The
> first query I run still takes some time, I guess it's
> retrieving the module and compiling it (this module in turn
> imports the XSD schema). Is it possible to load a module at
> the start of the servlet?
Saxon-EE supports separate compilation of query modules, using the method
XQueryCompiler.compileLibrary() in the s9api package. Once it is compiled,
any module that attempts to import the precompiled library module URI will
automatically pick up the compiled version. Any schema that is imported by
the library module should also be processed at the time you compile the
library.
Note that the compiled module "belongs" to the XQueryCompiler, and will only
be used if you compile the main module using the same XQueryCompiler
instance. It will be kept in memory so long as the XQueryCompiler is kept in
memory.
>Looking at the API I found the StandardModuleURIResolver class
I don't think this interface is particularly relevant to your requirement.
Regards,
Michael Kay
http://www.saxonica.com/http://twitter.com/michaelhkay

Thread view

Hello all,
We've run into some performance issues when importing an XML Schema, namely a query that simply imports [1], takes around 7 seconds on a Dual Xeon 3.4GHz 2GB RAM.
Is there a way we can speed this up? Possibly having Saxon running as a daemon that takes queries from a socket?
thanks for any hints.
--
Nuno Lopes
[1] http://www.w3.org/2007/SPARQL/result.xsd

I'll try and investigate why this schema should be taking a long time to
load - perhaps it's fetching the schema for the XML namespace from the W3C
web site - though Saxon has built-in knowledge of this namespace so that
shouldn't be necessary.
However, in this kind of situation the important thing is to ensure that
schema documents, once loaded and compiled, are cached and reused. This is
achieved by having multiple queries use the same Saxon Configuration object.
(or s9api Processor, or equivalent).
Regards,
Michael Kay
http://www.saxonica.com/http://twitter.com/michaelhkay
> -----Original Message-----
> From: Nuno Lopes [mailto:nuno.lopes@...]
> Sent: 04 February 2010 16:23
> To: saxon-help@...
> Subject: [saxon] importing an XML Schema
>
> Hello all,
>
> We've run into some performance issues when importing an XML
> Schema, namely a query that simply imports [1], takes around
> 7 seconds on a Dual Xeon 3.4GHz 2GB RAM.
>
> Is there a way we can speed this up? Possibly having Saxon
> running as a daemon that takes queries from a socket?
>
> thanks for any hints.
> --
> Nuno Lopes
>
>
> [1] http://www.w3.org/2007/SPARQL/result.xsd
> --------------------------------------------------------------
> ----------------
> The Planet: dedicated and managed hosting, cloud storage,
> colocation Stay online with enterprise data centers and the
> best network in the business Choose flexible plans and
> management services without long-term contracts Personal 24x7
> support from experience hosting pros just a phone call away.
> http://p.sf.net/sfu/theplanet-com
> _______________________________________________
> saxon-help mailing list archived at
> http://saxon.markmail.org/ saxon-help@...
> https://lists.sourceforge.net/lists/listinfo/saxon-help

Hello,
On 5 Feb 2010, at 20:01, Michael Kay wrote:
> iIf you're concerned about performance then you almost certainly shouldn't
> be running Saxon from the command line. Starting up the Java VM can often
> take longer than actually executing the query. If you're running any kind of
> application that involves multiple queries (as suggested by your remark
> about a deomon process) then you will get much higher throughput if you
> execute the queries from some kind of framework involving a single
> Configuration. The simplest is usually to run an application accepting
> requests within an HTTP servlet container.
Thanks a lot, running as a servlet does indeed improve performance *by a lot*.
There are although a couple of questions I came across: The first query I run still takes some time, I guess it's retrieving the module and compiling it (this module in turn imports the XSD schema). Is it possible to load a module at the start of the servlet? Looking at the API I found the StandardModuleURIResolver class but, although from the description it seems to be the class I want, using this at load doesn't improve the time of the first query. Basically the idea would be that the servlet would contain the module and XSD loaded at all times (since all incoming queries will require it), this also lead me to think if there is some timeout for caching the module and XSD in Saxon.
Thanks for any help,
best regards,
--
Nuno Lopes
> Regards,
>
> Michael Kay
> http://www.saxonica.com/
> http://twitter.com/michaelhkay
>
>> -----Original Message-----
>> From: Nuno Lopes [mailto:nuno.lopes@...]
>> Sent: 05 February 2010 10:03
>> To: Mailing list for the SAXON XSLT and XQuery processor
>> Subject: Re: [saxon] importing an XML Schema
>>
>> Hello,
>>
>> On 5 Feb 2010, at 03:09, Michael Kay wrote:
>>
>>> I'll try and investigate why this schema should be taking a
>> long time
>>> to load - perhaps it's fetching the schema for the XML
>> namespace from
>>> the W3C web site - though Saxon has built-in knowledge of this
>>> namespace so that shouldn't be necessary.
>>>
>>> However, in this kind of situation the important thing is to ensure
>>> that schema documents, once loaded and compiled, are cached and
>>> reused. This is achieved by having multiple queries use the
>> same Saxon Configuration object.
>>> (or s9api Processor, or equivalent).
>>
>> Forgot to mention that we're using Saxon-EE 9.2.0.3J from the
>> command line. Is it possible to do what you say in this setting?
>>
>> best regards,
>> --
>> Nuno Lopes
>>
>>
>>
>>
>>
>>> Regards,
>>>
>>> Michael Kay
>>> http://www.saxonica.com/
>>> http://twitter.com/michaelhkay
>>>
>>>> -----Original Message-----
>>>> From: Nuno Lopes [mailto:nuno.lopes@...]
>>>> Sent: 04 February 2010 16:23
>>>> To: saxon-help@...
>>>> Subject: [saxon] importing an XML Schema
>>>>
>>>> Hello all,
>>>>
>>>> We've run into some performance issues when importing an
>> XML Schema,
>>>> namely a query that simply imports [1], takes around
>>>> 7 seconds on a Dual Xeon 3.4GHz 2GB RAM.
>>>>
>>>> Is there a way we can speed this up? Possibly having Saxon
>> running as
>>>> a daemon that takes queries from a socket?
>>>>
>>>> thanks for any hints.
>>>> --
>>>> Nuno Lopes
>>>>
>>>>
>>>> [1] http://www.w3.org/2007/SPARQL/result.xsd
>>>> --------------------------------------------------------------
>>>> ----------------
>>>> The Planet: dedicated and managed hosting, cloud storage,
>> colocation
>>>> Stay online with enterprise data centers and the best
>> network in the
>>>> business Choose flexible plans and management services without
>>>> long-term contracts Personal 24x7 support from experience hosting
>>>> pros just a phone call away.
>>>> http://p.sf.net/sfu/theplanet-com
>>>> _______________________________________________
>>>> saxon-help mailing list archived at
>>>> http://saxon.markmail.org/ saxon-help@...
>>>> https://lists.sourceforge.net/lists/listinfo/saxon-help
>>>
>>>
>>>
>> ----------------------------------------------------------------------
>>> -------- The Planet: dedicated and managed hosting, cloud storage,
>>> colocation Stay online with enterprise data centers and the best
>>> network in the business Choose flexible plans and
>> management services
>>> without long-term contracts Personal 24x7 support from experience
>>> hosting pros just a phone call away.
>>> http://p.sf.net/sfu/theplanet-com
>>> _______________________________________________
>>> saxon-help mailing list archived at http://saxon.markmail.org/
>>> saxon-help@...
>>> https://lists.sourceforge.net/lists/listinfo/saxon-help
>>
>>
>> --------------------------------------------------------------
>> ----------------
>> The Planet: dedicated and managed hosting, cloud storage,
>> colocation Stay online with enterprise data centers and the
>> best network in the business Choose flexible plans and
>> management services without long-term contracts Personal 24x7
>> support from experience hosting pros just a phone call away.
>> http://p.sf.net/sfu/theplanet-com
>> _______________________________________________
>> saxon-help mailing list archived at
>> http://saxon.markmail.org/ saxon-help@...
>> https://lists.sourceforge.net/lists/listinfo/saxon-help
>
>
> ------------------------------------------------------------------------------
> The Planet: dedicated and managed hosting, cloud storage, colocation
> Stay online with enterprise data centers and the best network in the business
> Choose flexible plans and management services without long-term contracts
> Personal 24x7 support from experience hosting pros just a phone call away.
> http://p.sf.net/sfu/theplanet-com
> _______________________________________________
> saxon-help mailing list archived at http://saxon.markmail.org/
> saxon-help@...
> https://lists.sourceforge.net/lists/listinfo/saxon-help

>
> Thanks a lot, running as a servlet does indeed improve
> performance *by a lot*.
> There are although a couple of questions I came across: The
> first query I run still takes some time, I guess it's
> retrieving the module and compiling it (this module in turn
> imports the XSD schema). Is it possible to load a module at
> the start of the servlet?
Saxon-EE supports separate compilation of query modules, using the method
XQueryCompiler.compileLibrary() in the s9api package. Once it is compiled,
any module that attempts to import the precompiled library module URI will
automatically pick up the compiled version. Any schema that is imported by
the library module should also be processed at the time you compile the
library.
Note that the compiled module "belongs" to the XQueryCompiler, and will only
be used if you compile the main module using the same XQueryCompiler
instance. It will be kept in memory so long as the XQueryCompiler is kept in
memory.
>Looking at the API I found the StandardModuleURIResolver class
I don't think this interface is particularly relevant to your requirement.
Regards,
Michael Kay
http://www.saxonica.com/http://twitter.com/michaelhkay

You are starting to get outside the scope of this particular list but, using
Tomcat (I've been working with 6) you can implement
javax.servlet.ServletContextListener and hook it via the configuration
(web.xml) file.
You can then create/destroy objects outside of any particular servlet
instance and attach them to the "ServletContext" where they can then be
retrieved by your individual (shared) servlets.
As with any servlet based application you need to be aware of threading
issues.
Hope this helps and gives you an idea of where to look further.
I guess another alternative is to operate statically. From Michael's most
recent comment this appears to be how the XQueryCompiler operates.
Dave J
-----Original Message-----
From: Nuno Lopes [mailto:nuno.lopes@...]
Sent: Wednesday, February 10, 2010 1:30 PM
To: Mailing list for the SAXON XSLT and XQuery processor
Subject: Re: [saxon] importing an XML Schema
Hello,
On 5 Feb 2010, at 20:01, Michael Kay wrote:
> iIf you're concerned about performance then you almost certainly
> shouldn't be running Saxon from the command line. Starting up the Java
> VM can often take longer than actually executing the query. If you're
> running any kind of application that involves multiple queries (as
> suggested by your remark about a deomon process) then you will get
> much higher throughput if you execute the queries from some kind of
> framework involving a single Configuration. The simplest is usually to
> run an application accepting requests within an HTTP servlet container.
Thanks a lot, running as a servlet does indeed improve performance *by a
lot*.
There are although a couple of questions I came across: The first query I
run still takes some time, I guess it's retrieving the module and compiling
it (this module in turn imports the XSD schema). Is it possible to load a
module at the start of the servlet? Looking at the API I found the
StandardModuleURIResolver class but, although from the description it seems
to be the class I want, using this at load doesn't improve the time of the
first query. Basically the idea would be that the servlet would contain the
module and XSD loaded at all times (since all incoming queries will require
it), this also lead me to think if there is some timeout for caching the
module and XSD in Saxon.
Thanks for any help,
best regards,
--
Nuno Lopes
> Regards,
>
> Michael Kay
> http://www.saxonica.com/
> http://twitter.com/michaelhkay
>
>> -----Original Message-----
>> From: Nuno Lopes [mailto:nuno.lopes@...]
>> Sent: 05 February 2010 10:03
>> To: Mailing list for the SAXON XSLT and XQuery processor
>> Subject: Re: [saxon] importing an XML Schema
>>
>> Hello,
>>
>> On 5 Feb 2010, at 03:09, Michael Kay wrote:
>>
>>> I'll try and investigate why this schema should be taking a
>> long time
>>> to load - perhaps it's fetching the schema for the XML
>> namespace from
>>> the W3C web site - though Saxon has built-in knowledge of this
>>> namespace so that shouldn't be necessary.
>>>
>>> However, in this kind of situation the important thing is to ensure
>>> that schema documents, once loaded and compiled, are cached and
>>> reused. This is achieved by having multiple queries use the
>> same Saxon Configuration object.
>>> (or s9api Processor, or equivalent).
>>
>> Forgot to mention that we're using Saxon-EE 9.2.0.3J from the command
>> line. Is it possible to do what you say in this setting?
>>
>> best regards,
>> --
>> Nuno Lopes
>>
>>
>>
>>
>>
>>> Regards,
>>>
>>> Michael Kay
>>> http://www.saxonica.com/
>>> http://twitter.com/michaelhkay
>>>
>>>> -----Original Message-----
>>>> From: Nuno Lopes [mailto:nuno.lopes@...]
>>>> Sent: 04 February 2010 16:23
>>>> To: saxon-help@...
>>>> Subject: [saxon] importing an XML Schema
>>>>
>>>> Hello all,
>>>>
>>>> We've run into some performance issues when importing an
>> XML Schema,
>>>> namely a query that simply imports [1], takes around
>>>> 7 seconds on a Dual Xeon 3.4GHz 2GB RAM.
>>>>
>>>> Is there a way we can speed this up? Possibly having Saxon
>> running as
>>>> a daemon that takes queries from a socket?
>>>>
>>>> thanks for any hints.
>>>> --
>>>> Nuno Lopes
>>>>
>>>>
>>>> [1] http://www.w3.org/2007/SPARQL/result.xsd
>>>> --------------------------------------------------------------
>>>> ----------------
>>>> The Planet: dedicated and managed hosting, cloud storage,
>> colocation
>>>> Stay online with enterprise data centers and the best
>> network in the
>>>> business Choose flexible plans and management services without
>>>> long-term contracts Personal 24x7 support from experience hosting
>>>> pros just a phone call away.
>>>> http://p.sf.net/sfu/theplanet-com
>>>> _______________________________________________
>>>> saxon-help mailing list archived at http://saxon.markmail.org/
>>>> saxon-help@...
>>>> https://lists.sourceforge.net/lists/listinfo/saxon-help
>>>
>>>
>>>
>> ---------------------------------------------------------------------
>> -
>>> -------- The Planet: dedicated and managed hosting, cloud storage,
>>> colocation Stay online with enterprise data centers and the best
>>> network in the business Choose flexible plans and
>> management services
>>> without long-term contracts Personal 24x7 support from experience
>>> hosting pros just a phone call away.
>>> http://p.sf.net/sfu/theplanet-com
>>> _______________________________________________
>>> saxon-help mailing list archived at http://saxon.markmail.org/
>>> saxon-help@...
>>> https://lists.sourceforge.net/lists/listinfo/saxon-help
>>
>>
>> --------------------------------------------------------------
>> ----------------
>> The Planet: dedicated and managed hosting, cloud storage, colocation
>> Stay online with enterprise data centers and the best network in the
>> business Choose flexible plans and management services without
>> long-term contracts Personal 24x7 support from experience hosting
>> pros just a phone call away.
>> http://p.sf.net/sfu/theplanet-com
>> _______________________________________________
>> saxon-help mailing list archived at
>> http://saxon.markmail.org/ saxon-help@...
>> https://lists.sourceforge.net/lists/listinfo/saxon-help
>
>
> ----------------------------------------------------------------------
> -------- The Planet: dedicated and managed hosting, cloud storage,
> colocation Stay online with enterprise data centers and the best
> network in the business Choose flexible plans and management services
> without long-term contracts Personal 24x7 support from experience
> hosting pros just a phone call away.
> http://p.sf.net/sfu/theplanet-com
> _______________________________________________
> saxon-help mailing list archived at http://saxon.markmail.org/
> saxon-help@...
> https://lists.sourceforge.net/lists/listinfo/saxon-help
----------------------------------------------------------------------------
--
SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
http://p.sf.net/sfu/solaris-dev2dev
_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
saxon-help@...
https://lists.sourceforge.net/lists/listinfo/saxon-help

>
> You can then create/destroy objects outside of any particular
> servlet instance and attach them to the "ServletContext"
> where they can then be retrieved by your individual (shared) servlets.
>
> As with any servlet based application you need to be aware of
> threading issues.
>
> Hope this helps and gives you an idea of where to look further.
>
> I guess another alternative is to operate statically. From
> Michael's most recent comment this appears to be how the
> XQueryCompiler operates.
>
No, the XQueryCompiler won't use static. You can have as many
XQueryCompilers as you like and each can hold a cache of compiled query
modules. However, looking at the code, this hasn't been designed to allow
the XQueryCompiler to be shared across threads, unless all the compiling of
library modules (and initialisation of compiler options such as namespace
declarations) is done during single-threaded initialization. I don't think I
was envisaging quite this scenario. I'd recommend using one XQueryCompiler
per thread.
Regards,
Michael Kay
http://www.saxonica.com/http://twitter.com/michaelhkay