Scalable VNF monitor policies using Mistral

Currently, Tacker server hosts a local variable to keep the monitoring
of VNFs and uses local threads to do the monitoring. This causes tacker
server not to be scalable and impacts the API performance as well:

If we have a lot of VNFs which needs monitoring, the tacker server will
have to run a lot of threads to monitor them, which will impact the API
function of tacker server.

If the tacker server restarts, the monitor threads will not start.

The system cannot run more than one tacker server, since this is
making tacker server stateful.

Since Mistral is an integral part of tacker system, a long-live Mistral
workflow action can be used to do this kind of task.
Tacker server will generate a VNF monitoring workflow and execute it if
there is a VNF configured with monitor policies. The workflow and execution
will be removed once the monitored target VDU is removed.

Monitor actions cannot access tacker database directly, so we introduce a
Tacker-conductor server to do database access for the mistral actions.

Mistral does not stop long-live running action even if the workflow
execution is deleted.
So a mechanism is devised for action to exit. When the workflow is removed,
the VNFM plugin will kill the mistral action via MSG queue.

Mistral action will use RPC to communicate with conductor server.
To deal with scalability of monitoring, multiple conductors will be deployed.

The mistral action communicates with the conductor via the following interface
VNFPolicyActionRPC:

classVNFPolicyActionRPC(object)# execute policy actiondefexecute_policy_action(context,vnf,monitor_return_value,policy_actions,action_id):# The response message containing the operation result# the status should be the passed in by request or 'bad_action'# if the action is not the wanted action.status=rpccallconductorreturnstatus

After workflow is removed due to VNF change or removal, VNFM plugin kills the mistral
action via the following interface VNFPolicyMonitorRPC asynchronously:

The update method in above interface is used to notify policy monitor that changes
happened on the monitored vnf, for example the VNF was scaled, and was respawned.

Sequence diagram for create VNF:

Monitor policy is divided into two parts: policy monitor and policy action. Policy monitor,
such as ping and http_ping is implemented as mistral task action. Policy action will
be run in tacker conductor.

Each VNF with monitor policies will have a workflow generated, and will be kept as meta
information of VNF instance so that they can be managed.

The mistral workflow action will be run once the workflow is started. The action will do
its job according to monitor policy. When policy action is needed, the monitor action will
call conductor’s execute_policy_action RPC method.

Method execute_policy_action in conductor will call policy action, which will do actual job,
such as respawn, log etc.

If the policy action needs to update the vnf_policy_monitor, it will notify vnf_policy_monitor
the change.

If conductor finds the action is obsolete, it will return bad_action to vnf_policy_monitor,
then the vnf_policy_monitor will exit.

Sequence diagram for update VNF:

No need for this operation to do workflow stuff since VNF update is just used to config
VDUs.