Introduction

Note: in the above schema, when there are arrows between groups of nodes, it means that these arrows actually go to each node, but are not repeated to make the diagram easier on the eyes.

Basically you can see four layers of nodes:

Apache Web HTTP servers: usually a single node can be used, but it is also possible to setup a cluster of web servers. It will be used to proxy the requests to the DX and Unomi nodes, as well as protect the resources from being accessed from the open web. It is also a good idea to use HTTPS to proxy the requests, so that all the external traffic only uses HTTPS, while the "internal" traffic may use HTTP or HTTPS. Currently Apache Unomi only supports HTTP for the public endpoint, so you will need to setup HTTPS-to-HTTP proxying using Apache HTTP (or another equivalent) if you need to have your public traffic secured.

DX nodes: these are instances of the CMS that have the Marketing Factory modules deployed on them. They communicate between each other using Karaf Cellar (starting with DX 7.2) which itself uses Hazelcast as the library for node-to-node communication.

Apache Unomi nodes: the Unomi nodes also use Karaf Cellar and Hazelcast to communicate.

ElasticSearch nodes: the ES nodes use the custom networking protocol specific to ElasticSearch to communicate both between ES nodes and Unomi nodes.

Please note that in order to reduce the amount of hardware needed, it is possible to deploy the Unomi and ElasticSearch nodes on the same (physical or virtual) machines, meaning that in the above graph only 6 (physical or virtual) machines could be used. However in high load scenarios, it might be interesting to separate each node in order to be able to better control the resources the nodes need and maximize resistance to failure.

Ports used

As illustrated in the cluster diagram, a collection of ports is used by a typical Marketing Factory installation. The following summary table lists the ports that will need to be open in firewalls at various layers in the deployment to make sure everything will work as expected.

Port

Name

Visibility

Type

Description

5700-5800

Hazelcast

Unomi

TCP

Used by Apache Karaf Cellar to keep track of cluster nodes and exchange configuration information

Apache Unomi Administration endpoint, used to manage all Apache Unomi objects (segments, profile editing, etc…), also the Unomi secure address (Endpoint used in a secure environment)

9200

ElasticSearch HTTP

DX

TCP

ElasticSearch HTTP interface, used by clients such as Kibana or other applications

9300

ElasticSearch Transport

Unomi

TCP

ElasticSearch Transport TCP interface, highly optimized, used by Apache Unomi to talk to ElasticSearch, as well as between ElasticSearch nodes to exchange data (replications, …)

The visibility column describes the highest layer the application architecture that must have visibility to that port. So for example for port 8181, a "Web" visibility indicates that the port must be visible to the world, although this doesn’t mean it necessarily is exposed directly, it could still be relayed through a proxy (for an example with Apache mod_proxy, see section 4.4)

ATTENTION: If you are deploying Marketing Factory in a network where there are already other instances of Apache Unomis and ElasticSearch servers, you must change the cluster name in the ElasticSearch instances as well as modify ports so that you do not join an existing different cluster. This is very important because data corruption may occur if not setup properly!

Solution overview

Now let’s look at an overview of the elements that compose Marketing Factory.

Apache Unomi: an open source personalization and A/B testing server that is an implementation of the OASIS Context Server specification. Throughout this document we will refer to it as "Apache Unomi". Jahia distributes a custom package of Apache Unomi for integration with Marketing Factory that is tested and supported and that must be paired with the corresponding version of the Marketing Factory DX modules. Apache Unomi is responsible for data collection, rule execution, and serving the personalization context back to the Marketing Factory components so they may build personalized experiences. Apache Unomi itself has the following sub-components:

Apache Karaf: an OSGi runtime that contains a lot of enterprise ready features such as fine-grained logging, clustering, SSH shell console, provisioning, server-side REST framework (CXF) and many more advanced enterprise capabilities. You might think of it as the equivalent of what Tomcat is to Web Application Server but in this case for OSGi Application servers. It is a standalone package that is executed within a Java Virtual Machine.

CXS API: The Context Server API is an implementation of the on-going work of the OASIS Context Server Technical Committee to define a standardized API to collect and serve personalization and A/B testing data. As the specification is still not completed, this API might change but both Marketing Factory and Apache Unomi will be adapted to follow any modifications.

Apache Unomi OSGi bundles: these are the actual components that implement the services that are exposed by the CXS API. They range from cluster management to data collection, rules processing, profile and session management and many other back ends that deliver the personalization and A/B testing functionality.

ElasticSearch: Apache Unomi uses ElasticSearch to store, index and retrieve all the objects that are needed to deliver personalization and A/B testing. All the events, profiles, sessions, definitions, … are stored as JSON documents within the ElasticSearch cluster. As ElasticSearch can scale to dozens or sometimes even hundreds of nodes, it offers a highly scalable data and search back-end.

Minimal System Requirements

Please find below the minimum system requirements to properly run Marketing Factory. You can refer to the document ‘Marketing Factory performances metrics" for more information about system requirements & sizing. This document can be requested to the sales or support team.

OS:

Windows

Linux

Solaris

Mac OSX

Suggested Min. Production Environments:

Quad Core (64 bit CPU and OS)

8 GB RAM

100 GB HDD (SSD recommended)

Java Virtual Machine

In order to run Marketing Factory, you first need to install an Oracle’s Java SE (Java Platform, Standard Edition) in version 8 on your system. Marketing Factory requires the JDK (Java Development Kit) package to run.

To check if Java is already installed on your system, type the following command line at the prompt of your system:

java -version

You should get a message indicating which Java version is installed on your system. Please note that the same message will be displayed if you only have a JRE installed. If an error is returned, you probably don't have a Java Platform installed. If you have installed other versions of the Java Platform, Java Runtime Environment or other Java servers on your system, we recommend that you run a few checks before starting the installation in order to be sure that Digital Factory will run without problems. If you need to obtain and install a new Java SE, you can find both Linux and Windows versions on the Oracle Web site: http://www.oracle.com/technetwork/java/javase/downloads/index.html. To install a Java Virtual Machine on a Windows system, you need to have administrator rights on your computer. Please contact your system administrator if you don’t have sufficient permissions. Although the Apache Unomi tries to detect the location of the installed Java SE, we recommend setting the JAVA_HOME environment variable explicitly to the directory, where you have installed the Java SE. To setup this variable, follow the steps, described in next sections.

Under Windows

i) Open the Control Panel, and the System option. In Windows 7 and Vista it is: Control Panel _ System and Security _ System _ Advanced System Settings. Then, depending on your system: - Select the Advanced tab and click on the Environment Variables button (Windows 7/Vista/XP/2000) - Select the Properties tab and click on the Environment button (Windows NT) ii) Click on New in the "System variables" section to add a new environment variable. Enter the following information: - Variable name: JAVA_HOME - Variable value: c:\Program Files\Java\jdk1.8.0_60 (replace this value with the correct path) Click on OK to validate your entry. The Java Virtual Machine should now be correctly set-up. Please note that on Windows NT you will need to restart your computer to apply the changes.

Under Linux

Set the JAVA_HOME variable to the root directory of your JDK installation. Both examples below suppose you have installed the JDK version 1.8 in your /usr/java directory. The classpath is usually set by typing: In bash or ksh:

export JAVA_HOME=/usr/java/jdk1.8

In csh or tcsh:

export JAVA_HOME /usr/java/jdk1.8

Under Solaris

Set the JAVA_HOME variable to the root directory of your JDK installation. Both examples below suppose you have installed the JDK version 1.8 in your /usr/java directory. The classpath is usually set by typing: In ksh: