XCC allows you to create multi-tier applications that communicate with MarkLogic Server as the underlying content repository. This chapter describes some of the basic programming concepts used in XCC. It includes the following sections:

Use the Admin Interface to set up an XDBC server, specifying a name, port, a database to access, and other configuration parameters. For detailed instructions how to configure an XDBC Server, see the Administrator's Guide. You need an XDBC Server for an XCC program to communicate with MarkLogic Server. Alternately, you can use set up a REST instance to accept XDBC requests by setting the xdbc-enabled option to true in your REST instance; for details, see Administering REST Client API Instances in the REST Application Developer's Guide.

XCC programs use the Session interface to set up and control communication with MarkLogic Server. XCC automatically creates and releases connections to MarkLogic Server as needed, and automatically pools the connections so that multiple requests are handled efficiently.

A Session handles authentication with MarkLogic Server and holds a dynamic state, but it is a lightweight object. It is OK to create and release Session objects as needed and as makes logical sense for your program. Do not expend effort to pool and reuse them, however, because they are not expensive to create. For example, if your program is doing multiple requests one after another, create a Session object at the beginning and close it when the last request is complete.

You set up the connection details with the ContentSource object. You can submit the connection details when you invoke the XCC program with a URL that has the following form:

xcc://username:password@host:port/database

Also, there are discrete arguments to the constructors in the API to set up any or all portions of the connection details.

Point-in-time queries allow you to query older versions of content in a database. In an XCC application, you set up the options for any requests submitted to MarkLogic Server with the RequestOptions class. One of the options you can set is the effective point-in-time option. Therefore, to set up a query to run at a different point in time, you just set that option (the setEffectivePointInTime method in Java) on the RequestOptions. The query will then run at the specified point in time.

There are several things you must set up on MarkLogic Server in order to perform point-in-time queries. For details, see the 'Point-In-Time Queries' chapter of the Application Developer's Guide.

Certain exceptions that MarkLogic Server throws are retryable; that is, the exception is thrown because of a condition that is transitory, and applications can try the request again after getting the exception. XCC will automatically retry retryable exceptions in single-statement transactions. You can control the maximum number of retryable exceptions with the RequestOptions interface.

Multi-statement transactions cannot automatically be retried by the server. Your application must handle retries explicitly when using multi-statement transactions. For details, see Retrying Multi-statement Transactions.

You can use AdhocQuery and Session.submitRequest to evaluate either XQuery or Server-Side JavaScript queries on MarkLogic Server. By default, XCC assumes the query language is XQuery. Use RequestOptions.setQueryLanguage to specify JavaScript instead. For example:

You can the results of your query in the usual way. When a ResultItem in the result sequence is a JsonItem, you can extract the item as Jackson JsonNode and use the Jackson library functions to traverse and access the structure.

Note that there is a difference between returning native JavaScript objects and arrays returning JSON nodes from the database:

A JavaScript object or array corresponds to an atomic result item type. That is, the underlying value type is JS_OBJECT or JS_ARRAY, and ItemType.isAtomic returns true.

A JSON node, such as the result of calling cts.doc or the output from calling a NodeBuilder method, has a node value type. That is, a value type such as OBJECT_NODE, ARRAY_NODE, BOOLEAN_NODE, NUMBER_NODE, or NULL_NODE. Also, ItemType.isNode returns true.

In most cases, your code can ignore this distinction because you can use Jackson to manipulate both kinds of results transparently through the JsonItem interface. For details, see Working With JSON Content.

The XCC interfaces include an integration with Jackson for manipulating JSON data in Java. To use XCC methods such as JsonItem.asJsonNode or the ContentFactory.newJsonContent overload that accepts a JsonNode, you must have an installation of Jackson and put the Jackson jar files on your classpath.

Exactly which libraries you need to add to your classpath depends on the Jackson features you use, but you will probably need at least the Jackson core libraries, available from http://github.com/FasterXML/jackson.

For example, you might need to add the following libraries to your classpath:

You can JSON data into the database the same way you insert other data. If the document URI extension is mapped to the JSON document format in the MarkLogic Server MIME type mappings, then a JSON document is automatically created.

For example, the following code snippet reads JSON data from a file and inserts it into the database as a JSON document.

If you run an ad hoc query that returns JSON (or a JavaScript object or array), you can use Jackson to traverse and manipulate the data in your Java application.

For example, the following code snippet evaluates an ad hoc Server-Side JavaScript query that retrieves a JSON document from the database, and then accesses the value in the document's 'num' property as an integer:

You can also use the Jackson interfaces to manipulate native JavaScript objects and arrays returned by ad hoc Server-Side JavaScript queries. That is, the above conversion to a JsonNode works whether the item is a JSON node or an atomic JS_OBJECT result.

Then you can use any of the Jackson interfaces to manipulate the contents. For example, the following code snippet accesses the value of the 'num' JSON property as an integer:

This section describes how to use a simple Trust Manager for X.509-based authentication. The Trust Manager shown here does not validate certificate chains and is therefore unsafe and should not be used for production code. See your Java documentation for details on how to create a more robust Trust Manager for your specific application or how to obtain a Certificate Authority from a keystore.

To enable SSL access using a trust manager, import the following classes in addition to those described in Coding Basics:

You can use the Java keytool utility to import a MarkLogic certificate into a keystore. See the Java JSSE documentation for details on the use of the keytool and your keystore options.

You can explicitly specify a keystore, as shown in this example, or you can specify a null keystore. Specifying a null keystore causes the TrustManagerFactory to locate your default keystore, as described in the Java Secure Socket Extension (JSSE) Reference Guide.

To enable SSL by accessing certificates in a keystore, import the following classes in addition to those described in Coding Basics:

You can define a KeyManager, if your client application is required to send authentication credentials to the server. The following example adds client authentication to the newTrustOptions method shown in Accessing a Keystore:

When you submit a request to MarkLogic Server, the results are returned to your application in a ResultSequence. By default the XdmItem objects in the sequence are cached. That is, all the result items are read and buffered in memory. Cached results do not tie up any connection resources, and are usually preferred.

A non-cached, or streaming, ResultSequence may only be accessed sequentially and hold the connection to MarkLogic Server open. Individual results may only be read once and on demand, so the result set consumes less memory, at the cost of less efficient access.

If you are retrieving large results, such as a large binary document, you may disable result caching to conserve memory. You may disable result caching per request by creating a RequestOption object with the setting disabled, and associating it with a request, either directly with Request.setOptions or passing it as a parameter to a Request creation method such as Session.newAdhocQuery. You may disable result caching per session by setting the default request options for the session using Session.setDefaultRequestOptions.

By default, all transactions run as single-statement, auto-commit transactions. MarkLogic Server also supports multi-statement, explicitly commited transactions in XQuery, Server-Side JavaScript, and XCC. For more details on transaction concepts, see Understanding Transactions in MarkLogic Server in the Application Developer's Guide. This section covers only related behaviors unique to XCC:

Use the following procedure to use multi-statement, explicitly committed transactions with XCC:

Create a Session object in the usual way.

Call Session.setAutoCommit with a value of false. The next transaction created in the session will run as a multi-statement, explicit commit transaction.

Optionally, call Session.setUpdate to specify an explicit transaction type. By default, MarkLogic determines the transaction type through static analysis of the first statement in a request, but you can explicitly set the transaction type to update or query using Session.setUpdate.

Call Session.submitRequest as usual to operate on your data. All requests run in the same transaction until the transaction is commited or rolled back.

Call Session.commit or Session.rollback to commit or rollback the transaction. If the session ends or times out without explicitly commit or rolling back, the transaction is rolled back.

To restore a session to the default, single-statement transaction model, call Session.setAutoCommit with a value of true and Session.setUpdate with a value of AUTO.

Note that the transaction configuration defined by setAutoCommit and setUpdate remain in effect for all transactions created by a session until explicitly changed. If you override the transaction configuration in an ad hoc query, the override applies only to the current transaction.

Multi-statement query transactions allow all the statements in a transaction to share the same point-in-time view of the database, as discussed in Point-In-Time Queries.

In a multi-statement update transaction, updates performed by one statement (or request) are visible to subsequent statements in the same transaction, without being visible to other transactions.

A multi-statement transaction remains open until it is committed or rolled back. Use Session.commit. to commit a multi-statement transaction and make the changes visible in the database. Use Session.rollback to roll back a multi-statement transaction, discarding any updates. Multi-statement transactions are implicitly rolled back when the containing session ends or the transaction times out. Failure to explicitly commit or rollback a multi-statement update transaction can tie up resources, hold locks unnecessarily, and increase the chances of deadlock.

You may receive a java.lang.IllegalStateException if you call Session.commit from an exception handler when there are no pending updates in the current transaction. Committing from a handler is not recommended.

For a detailed discussion of multi-statement transactions, see the 'Understanding Transactions' chapter of the Application Developer's Guide.

The following example demonstrates using multi-statement transactions in Java. The first multi-statement transaction in the session inserts two documents into the database, calling Session.commit to complete the transaction and commit the updates. The second transaction demonstrates the use of Session.rollback. The third transaction demonstrates implicitly rolling back updates by closing the session.

Calling Session.commit from an exception handler that wraps a request participating in a multi-statement transaction may raise java.lang.IllegalStateException. You may always safely call Session.rollback from such a handler.

Usually, an exception raised during multi-statement transaction processing leaves the Session open, allowing you to continue working in the transaction after handling the exception. However, in order to preserve consistency, exceptions occurring under the following circumstances always roll back the transaction:

After an XQuery statement has finished but before the XCC request is completed

In the middle of an explicit commit or rollback

If such a rollback occurs, the current transaction is terminated before control reaches your exception handler. Calling Session.commit when there is no active transaction raises a java.lang.IllegalStateException. Calling Session.rollback when there is no active transaction does not raise an exception, so rollback from a handler is always safe.

Therefore, it is usually only safe to call Session.commit from an exception handler for specific errors you expect to receive and for which you can predict the state of the transaction.

MarkLogic Server sometimes detects the need to retry a transaction. For example, if the server detects a deadlock, it may cancel one of the deadlocked transactions, allowing the other to complete; the cancelled transaction should be re-tried.

With single-statement, auto-commit transactions, the server can usually retry automatically because it has the entire transaction available at the point of detection. However, the statements in a multi-statement transactions from XCC clients may be interleaved with arbitrary application-specific code of which the server has no knowledge.

In such cases, instead of automatically retrying, the server throws a RetryableXQueryException. The calling application is then responsible for re-trying all the requests in the transaction up to that point. This exception is more likely to occur when using multi-statement transactions.

The following example demonstrates logic for re-trying a multi-statement transaction. The multi-statement transaction code is wrapped in a retry loop with an exception handler that waits between retry attempts. The number of retries and the time between attempts is up to the application.

If you use multi-statement transactions or set the transaction time limit when using XCC version 8.0-2 or later with versions of MarkLogic Server older than 8.0-2, you should set the system property xcc.txn.compatible to true. If you do not set this, then you will get an exception when trying to set the transaction mode or transaction time limit.

You can set the property on the java command line with an argument of the following form:

java -Dxcc.txn.compatible=true

You can also set the property programmatically by calling System.setProperty.

You do not need to set the property if your XCC application does not use multi-statement transactions or if your application communicates with MarkLogic Server version 8.0-2 or later.

To use MarkLogic Server in an XA transaction, use the Session.getXAResource method to register your XCC session as a javax.transaction.xa.XAResource with the XA Transaction Manager.

The following code snippet shows how to enlist an XCC session in a global XA transaction. Once you enlist the Session, any work performed in the session is part of the global transaction. For complete code, see the sample code in com.marklogic.xcc.examples.XA.

When MarkLogic Server acts as an XA transaction Resource Manager, requests submitted to the server are always part of a multi-statement update transaction, with the following important differences:

The Session.setAutoCommit and Session.setTransactionMode settings are ignored. The transaction is always a multi-statement update transaction, even if only a single request is submitted to MarkLogic Server during the global transaction.

The application should not call Session.commit or xdmp:commit. The transaction is committed (or rolled back) as part of the global XA transaction. To commit the global transaction, use the Transaction Manager with which the MarkLogic Server XAResource is registered.

The application may call Session.rollback or xdmp:rollback. Doing so eventually causes rollback of the global XA transaction. Rolling back via the Transaction Manager is usually preferable.

Under extreme circumstances, a MarkLogic Server administrator may need to heuristically complete (intervene to manually commit or rollback) the MarkLogic Server portion of a prepared XA transaction. This section covers the following topics related to heuristic completion:

The unit of work managed by a Resource Manager in an XA transaction is a branch. Manually intervening to force completion of a prepared XA transaction branch is making a heuristic decision, or heuristically completing the branch. The branch may be heuristically completed by either committing or rolling back the local transaction.

The Transaction Manager instructs all participants to prepare to commit.

Each participant reports responds with whether or not it is ready to commit.

If all participants report prepared to commit, the Transaction Manager instructs all participants to commit.

If one or more participants is not prepared to commit, the Transaction Manager instructs all participants to roll back.

If the Transaction Manager goes down due to a failure such as loss of network connectivity or a system crash, the Transaction Manager does not remember the global transaction when it comes back up. In this case, the local transaction times out normally, or may be cancelled with a normal rollback, using xdmp:transaction-rollback.

If the Transaction Manager goes down after the transaction is prepared, the Transaction Manager normally recovers and resumes the flow described above. However, it may not always be possible to wait for normal recovery.

For example, if connectivity to the Transaction Manager is lost for a long time, locks may be held unacceptably long on documents in MarkLogic Server. Under such circumstances, the administrator may heuristically complete a branch of the global transaction to release resources.

Heuristic completion bypasses the Transaction Manager and the Two Phase Commit process, so it can lead to data integrity problems. Use heuristic completion only as a last resort.

The XA protocol requires a Resource Manager to remember the outcome of a heuristic decision, allowing the Transaction Manager to determine the status of the branch when it resynchronizes the global transaction.This remembered state is automatically cleaned up if the global transaction eventually completes with the same outcome as the heuristic decision.

Manual intervention may be required after a heuristic decision. If the Transaction Manager recovers and makes a different commit/rollback decision for the global transaction than the local heuristic decision for the branch, then data integrity is lost and must be restored manually.

For example, if the administrator heuristically completes a branch by committing it, but the global transaction later rolls back, then the heuristically completed branch requires manual intervention to roll back the previously committed local transaction and make the resource consistent with the other global transaction participants.

Heuristic completion is not needed in cases where a global transaction stalls prior to being prepared. In this case, the global transaction is lost and the local branches time out or otherwise fall back on normal failure mechanisms.

Use the xdmp:xa-complete built-in function to heuristically complete the MarkLogic Server branch of a prepared global XA transaction. When using xdmp:xa-complete, you must indicate:

whether or not to commit the local transaction

whether or not MarkLogic Server should remember the heuristic decision outcome (commit or rollback)

Usually, you should rollback the local transaction and remember the heuristic decision outcome.

Forgetting the heuristic decision leads to an error and possibly loss of data integrity when the Transaction Manager subsequently attempts to resynchornize the global transaction. If the outcome is remembered, then the Transaction Manager can learn the status of the branch and properly resume the global transaction.

The following examples demonstrate several forms of heuristic completion. The 3rd parameter indicates whether or not to commit. The 4th parameter indicates whether or not to remember the outcome:

The forest id parameter of xdmp:xa-complete identifies the coordinating forest. Once an XA transaction is prepared, the coordinating forest remembers the state of the MarkLogic Server branch until the global transaction completes. Use the Admin UI or xdmp:forest-status to determine the transaction id and coordinating forest id.

For example, the following query retrieves a list of all transaction participants in transactions that are prepared but not yet comitted. The coordinating forest id for each participant is included in the results. For details, see xdmp:forest-status in XQuery and XSLT Reference Guide.

xquery version "1.0-ml";
for $f in xdmp:database-forests(xdmp:database())
return xdmp:forest-status($f)//*:transaction-participants

Additional cleanup may be necessary if the Transaction Manager resumes and the global transaction has an outcome that does not match the heuristic decision. For details, see Cleaning Up After Heuristic Completion.

Since XA transactions may involve multiple participants and non-MarkLogic Server resources, they may take longer than usual. A slow XA transaction may cause other queries on the same App Server to block for an unacceptably long time.

This section contains important information for environments in which a Layer 3 Load Balancer or a proxy server such as the Amazon Elastic Load Balancer (ELB) sits between your XCC application and MarkLogic Server cluster.

When you use a load balancer, it is possible for requests from your application to MarkLogic Server to be routed to different hosts, even within the same session. This has no effect on most interactions with MarkLogic Server, but queries evaluated in the context of the same multi-statement transaction need to be routed to the same host within your MarkLogic cluster. This consistent routing through a load balancer is called session affinity.

To enable your load balancer to preserve session affinity, you must do the following:

Enabling HTTP compliant mode guarantees the traffic between your XDBC App Server and your XCC client is compliant with the HTTP 1.1 protocol. This enables properly configured load balancers to detect the SessionID cookie generated by MarkLogic Server and use it to enforce session affinity.

To enable this mode for a Java application, set the xcc.httpcompliant system property to true on the Java command line. For example:

java -Dxcc.httpcompliant=true ...

Setting xcc.httpcompliant to true is incompatible with enabling content entity resolution using ContentCreateOptions.setResolveEntities.

If xcc.httpcompliant is not set explicitly, then xcc.httpcompliant is false.

You must also configure your load balancer to use the value in the SessionID cookie for session affinity. Some routers or load balancers may need to have xcc.httpcompliant enabled to allow any traffic through, regardless of session affinity issues.

In addition to setting xcc.httpcompliant to true, you must configure your load balancer to use the SessionID cookie generated by MarkLogic Server for session affinity. You might also need to enable session affinity or sticky sessions in your load balancer. The exact configuration steps depend on the load balancer; see your load balancer documentation for details.

Your load balancer must be HTTP 1.1 compliant and support cookie-based session affinity to use this feature of XCC.