Connecting Clients

Connecting clients through service discovery

One of the benefits of running containerized services is that they can be placed anywhere in the cluster. Since they can be deployed anywhere on the cluster, clients need a way to find the service. This is where service discovery comes in.

Discovering Endpoints

Once the service is running, you may view information about its endpoints via either of the following methods:

CLI:

List endpoint types:

dcos nifi endpoints

View endpoints for an endpoint type:

dcos nifi endpoints <endpoint>

API:

List endpoint types:

<dcos-url>/service/nifi/v1/endpoints

View endpoints for an endpoint type:

<dcos-url>/service/nifi/v1/endpoints/<endpoint>

Returned endpoints will include the following:

.autoip.dcos.thisdcos.directory hostnames for each instance that will follow them if they’re moved within the DC/OS cluster.

A HA-enabled VIP hostname for accessing any of the instances (optional).

A direct IP address for accessing the service if .autoip.dcos.thisdcos.directory hostnames are not resolvable.

If your service is on a virtual network such as the dcos overlay network, then the IP will be from the subnet allocated to the host that the task is running on. It will not be the host IP. To resolve the host IP use Mesos DNS (<task>.<service>.mesos).

In general, the .autoip.dcos.thisdcos.directory endpoints will only work from within the same DC/OS cluster. From outside the cluster you can either use direct IPs or set up a proxy service that acts as a front end to your nifi instance. For development and testing purposes, you can use DC/OS Tunnel to access services from outside the cluster, but this option is not suitable for production use.

This JSON array contains a list of valid nodes that the client can use to connect to the nifi cluster. For availability reasons, it is best to specify multiple nodes in configuration of the client. Use the VIP to address any one of the nifi nodes in the cluster.

When TLS is enabled, an endpoint named node-tls should also be listed. To verify a TLS connection from a client, the DC/OS trust bundle with a CA certificate is required.

Accessing the DC/OS NiFi web interface with Edge-LB configuration

Assumptions

DC/OS NiFi is installed on DC/OS without TLS and Kerberos

Edge-LB is installed (with service account and service account secret in strict mode)

Steps

Following are the steps for Edge-LB Pool configuration:

Install the edgelb cli

dcos package install --cli edgelb --yes

Get the DNS address using the following:

dcos nifi endpoints node --name=<service_name>

Create the configuration file edgelb-pool-config.json with required parameters to access nifi on the web.