You need to be aware that the first 2 bullets/steps in that Algorithm
are JVM-wide... meaning for ALL webapps, all code running inside the JVM.
So using either of those first 2 techniques can effect other webapps (even the SE admin UI webapp).
Remember that ServletExec already comes with XML parsing classes. Most folks just let their code use those.
Out-of-the-box, any calls to a newInstance() method will result in the 3rd bullet/step being used.
This is because:

By default, SE does NOT set any of the JVM System Properties mentioned in the first bullet

By default, the JVM does NOT have a file named "jaxp.properties" in the lib folder of the JRE directory

SE 5.0 satisfies the requirements of the 3rd bullet, since:

For DocumentBuilderFactory.newInstance():
The xercesImpl.jar file that comes with SE contains the following file:
/META-INF/services/javax.xml.parsers.DocumentBuilderFactory
which names the DocumentBuilderFactory Implementation to use, whose class file also happens to be in that JAR.

For SAXParserFactory.newInstance():
The xercesImpl.jar file that comes with SE contains the following file:
/META-INF/services/javax.xml.parsers.SAXParserFactory
which names the SAXParserFactory Implementation to use, whose class file also happens to be in that JAR.

For TransformerFactory.newInstance():
The xalan.jar file that comes with SE contains the following file:
/META-INF/services/javax.xml.transform.TransformerFactory
which names the TransformerFactory Implementation to use, whose class file also happens to be in that JAR.

Setting one of these System Properties, or creating a jaxp.properties will cause the 3rd step to NOT be used anymore by any code running in that JVM. If you've done that in order to specify a class that is only inside your webapp (not visible to other webapps) then that may be fine for your own webapp, but may break other webapps if/when they try to use a newInstance() method do to some XML-related work.
If you need your app to use a DocumentBuilderFactory and/or a SAXParserFactory and/or a TransformerFactory that differs
from the one that comes with SE then consider one of these 2 possible options:

Replace the parsing JARs provided by SE (in SE's lib folder) with files of the same name whose versions are what you prefer. Note that with is option, you're changing SE and all code running inside SE to use a parser (brand and/or version)
that has not been tested by New Atlanta and is therefore not guaranteed to work.

Place the XML parsing/transforming/etc... JARs that you prefer your app to use, into the WEB-INF/lib folder of your own webapp. Then create a JAR file that meets the requirement of bullet/step #3 and place it into your WEB-INF/lib folder as well. This way when your app calls newInstance() it should receive the Instance that you prefer, and when other apps call
newInstance() they will receive the Instance that they prefer.

If you are using versions of SE prior to SE 5.0 (or if you just want more gritty details):
Here is some general knowledge to help you grasp the more specific information that will follow:

ServletExec comes with and uses XML parsing classes in order to parse the XML files used for Custom Tag Libraries (TLD files), and also to parse XML files for Web Applications (web.xml).
The first version of ServletExec to include XML parsing classes was SE 3.0.
At that time Sun's Project X parser (xml.jar) was used.
More information about this parser can be viewed under the heading:
What is Java Project X? at:http://java.sun.com/xml/faq.html#other
Beginning with ServletExec 4.0b1 Sun's Project X parser was no longer used. Instead the JAXP API and the JAXP 1.1 classes with its default parser (crimson.jar) were used. This was not done sooner because JAXP 1.1 was still an early access release prior to ServletExec 4.0b1.
JAXP 1.1 supports SAX-2 and DOM level 2.

Regardless of the version of ServletExec, when you are running a version of SE that operates in-process with the web server (ISAPI or NSAPI) certain JAR files (ServletExecXX.jar, and the XML parsing JARs for example) are added to ServletExec's classpath (the Main System Classpath) automatically upon startup. You won't see these JARs listed on the Classpath page of SE's main admin UI, and there is no way for you to prevent ServletExec ISAPI or NSAPI from trying to add these JARs to its Main Classpath. These JARs are found in SerlvetExec's lib folder.
Note: if you are using SE 4.0 and trying to do your own XML parsing in a JSP then be sure to put your parsing code into a JavaBean or some helper class rather than using scriptlets directly in the JSP. This works around a classloader bug that was fixed in SE 4.1. Alternatively, you can simply upgrade to SE 4.1 or higher to avoid this bug altogether.
Now here is some specific information for using your own XML parsing classes. The information varies depending on the version and flavor (in-process or out-of-process) of ServletExec that you have:

If you are running your code within the context of a Web
application, then:

Leave the XML parsing mechanism being used by ServletExec 3.0 or newer, in place.

Add the classes for the parser you want to use, to your web application. For example, put xerces.jar into the lib folder of your web app (Do not add them to the main SE classpath because if your web app code looks outside the web app for xml classes it needs, it will first see the XML parsing JARs being used by SE).

If you are using SE 4.0 or SE 4.1 then:

When you write your code to use xerces.jar, do not call
DocumentBuilderFactory.newInstance().

Instead, find out what the fully qualified class name is for the DocumentBuilderFactory Implementation provided by the parser in your web app, and use the 'new' operator to create an instance of it and use it.

If you are using version 4.1.1 of ServletExec, ServletExec still needs Crimson.jar, but it uses it in such a way that you are free to use
whatever XML parser you wish in your code.
You can use:
DocumentBuilderFactory.newInstance() to obtain an instance of the parser you wish to use, however you must be aware of and fully understand the ORDERED LOOKUP ALGORITHM described by the JavaDocs for that method [and the other related newInstance() methods... see top of this FAQ for more info].

If you are NOT running your code within the context of a Web
Application, but rather in a legacy manner, then:

If your version of SE uses xml.jar (SE 3.0 & SE 3.1) then here is an example of how to get all of ServletExec to use a different parser such as xerces.jar:

Add xerces.jar to the main ServletExec classpath.

Stop ServletExec

remove all the org.* packages from xml.jar leaving all other classes intact.
(these are DOM level 1 packages)
Note:
Everything should still work because the new org.* classes
found in xerces.jar are backward compatible with the parser
code that SE uses.

If you are using version 4.0, or 4.1 of SE (which use crimson.jar) you must use crimson.jar in your code. You cannot change the XML parser that the ServletExec Engine uses since there is a crimson-specific call that ServletExec code is relying upon in order to write web.xml files. If you call DocumentBuilderFactory.newInstance() in your code you will receive a parser instance implemented by crimson. This problem was caused by bug #260 which was fixed in SE 4.1.1. Therefore, upgrading to SE 4.1.1 will get you past it.
Here is more information about this case:
The documentation for:
javax.xml.parsers.DocumentBuilderFactory.newInstance()
in the JavaDocs for JAXP
http://java.sun.com/xml/jaxp/dist/1.1/docs/api
discusses an ordered lookup procedure to determine which DocumentBuilderFactory implementation class will be returned by DocumentBuilderFactory.newInstance().
ServletExec 4.0b1 and newer use the Services API technique described in that JavaDoc (step #3).
Since ServletExec (v.4.1, & v. 4.1.1) code relies on the Crimson implementation for writing out web.xml files, the only workaround (other than upgrading to SE 4.1.1 or higher)for doing your own parsing with a different parser is to run your code inside the context of a Web Application. Doing this is
discussed in step #2 of this FAQ entry.

If you are using version 4.1.1 of ServletExec, ServletExec still needs Crimson.jar, but it uses it in such a way that you are free to use
whatever XML parser you wish in your code.
You can use:
DocumentBuilderFactory.newInstance() to obtain an instance of the parser you wish to use, however you must be aware of and understand the ordered lookup algorithm described by the JavaDocs for that method. You will likely need to set the appropriate System Property in the JVM so that step #1 of that algorithm will be used to locate your XML parser class.

Note that ServletExec 4.2 does not use crimson.jar and does not rely on any crimson-specific calls. ServletExec 4.2 uses standard XML APIs, so any compliant XML parser can be used.

NOTE: If you are using JDK 1.4 (or possibly newer versions too) you should read FAQ #241 also