OK, that's a whole different can of worms. :)
TOSCA's Compute capabilities (Container and OperatingSystem) are explicit.
You specify which OS you want, how much RAM you want, how many CPUs, etc.
ARIA's explicit node types (for example, the AWS Compute node) are likewise
explicit. So there is not querying here: the plugin will attempt to spin up
exactly the virtual machine you asked for. If it fails, the workflow will
fail.

Advertising

This is not good enough, I think, for real world scenarios. There are two
possible solutions:
1) Support ranges or fallbacks. So instead of saying "I need 4 GB of RAM"
you can say "I would like 4 GB of RAM, but 2 GB would also be OK." There's
no easy way to do this now without totally changing how the capability
types are designed. But, it may be possible to override this via policies.
So, the capabilities would perhaps specify the minimal requirements, while
policies specify preferences. Some aspects of this were discussed in the
OPEN-O project. DeWayne, has any of this survived in ONAP, or have we not
reached that point in the discussion yet?
2) Incorporate this into the bigger topic of resource orchestration. This a
huge challenge for the industry. The problem field contains not just "I
need X amount of RAM" but also "I want all my virtual machines and
containers running on the same high-performance network backend and have
these two nodes on the same bare-metal machine or at least in the same data
center rack with a NUMA interconnect, and I also don't want all this to
cost more that $100 per hour." That's not crazy: these are real world
requirements for high-performance VNFs and service chaining. Resource
orchestration requires a full map of what is available in the data centers,
a negotiation-based algorithm for properly allocating and placing
resources, connection to billing services, etc. Of course resource
orchestration is not within the scope of ARIA, but it would be great for
ARIA to have plugins for them (and TOSCA be able to model resource
requirement policies) when these become available.
On Fri, Aug 11, 2017 at 3:02 PM, DeWayne Filppi <dewa...@cloudify.co> wrote:
> For the "resource realization" part, I was not even considering
> multicloud/vim. I was considering single cloud even outside of
> composition. Just reqs and caps. If my node "requires" a compute node
> with Suse Linux version X with a minimum of 4GB RAM, how does the
> orchestrator match that without querying the cloud via the plugin? If it
> does query it, which it seems it must in addition to the implicit quota
> query, how is this done? TOSCA seems to not really care, which is fine and
> perhaps a good idea. But ARIA has to care.
>
> DeWayne
>
> On Fri, Aug 11, 2017 at 12:34 PM, Tal Liron <t...@cloudify.co> wrote:
>
> > In my opinion, the new composite service should indeed be a single
> service,
> > so the ARIA CLI will show it as one (if a substituting service already
> > existed, it would be "dissolved" into the new top-level service). The
> > composition will show its traces if you look more deeply, because you'll
> > see that some node templates come from a different service template.
> > Perhaps our CLI can detect this and tell the user which service templates
> > were used to create the nodes in current service.
> >
> > The rest of what you described I think is not related directly to service
> > composition, but resource realization. The way we currently handle this
> is
> > by being very explicit: you have to use the derived Compute types
> included
> > in our AWS or Openstack plugins, for example.
> >
> > I think we should still keep this, because sometimes you want to be very
> > explicit (and use specific features of the platform), but I actually have
> > some ideas for multi-VIM support that would work differently: the idea is
> > to use the basic Compute type, with detailed properties in its Container
> > and OperatingSystem capabilities. There would then be a policy, perhaps
> > aria.Realization, that would hint at what kind of Compute nodes you are
> > looking for (you can apply the policy to specific nodes or groups, or
> > generally). It might be possible to still have strong platform-specific
> > feature support here, perhaps by implementing a general string key-value
> > dict that could include hints for the specific plugin to use.
> >
> > Anyway, just ideas at this point...
> >
> >
> > On Fri, Aug 11, 2017 at 2:25 PM, DeWayne Filppi <dewa...@cloudify.co>
> > wrote:
> >
> > > Good stuff. Obviously a "fancy include" was the wrong metaphor. The
> > > lifecycles are linked though. When a create the referencing service
> > (aria
> > > service create), ARIA will match the other service template, and run
> > create
> > > service on it, and then continue on. Uninstall of the referencer will
> > > destroy the referenced service. After such an instantiation, would
> > listing
> > > all services produce two (perhaps with some parent indicator) or one?
> > From
> > > what I've seen, reqs and caps don't have the equivalent of a lifecycle
> > > interface associated. Maybe I missed it. It is implicit in the
> > > orchestrator to match stuff up at service creation time, I'm assuming,
> > and
> > > using plugins somehow? The sort of flagship scenario is matching cloud
> > > compute resources without specifying an image. Since discovering
> > whether a
> > > particular cloud has a given capability, I guessing a plugin for that
> > cloud
> > > has to search cloud inventory and perhaps quota to provide info for the
> > > match. Since plugins (at least the Cloudify variety) tend to be
> > triggered
> > > due to operations that they connect to lifecycle methods, and there is
> no
> > > "materialize_capabilities" interface that I noticed, some other kind of
> > > magic must be performed. Maybe this is an orchestrator-specific detail
> > > outside of TOSCA.
> > >
> > > DeWayne
> > >
> > > On Fri, Aug 11, 2017 at 10:57 AM, Tal Liron <t...@cloudify.co> wrote:
> > >
> > > > OK, let me try to organize this differently. Three potential
> > conceptions
> > > > here:
> > > >
> > > > 1) A "fancy include," as you say. All that would happen here is that
> > the
> > > > TOSCA parser would automatically find the service template to include
> > > from
> > > > some kind of repository of recognized service templates, and just
> > include
> > > > that. The language in the TOSCA spec suggests that this is not the
> > > > intention: it is something that happens at the "orchestrator," not
> the
> > > > "parser."
> > > >
> > > > 2) Static service composition. This happens not at the parsing stage,
> > but
> > > > rather the instantiation stage, where reqs-and-caps happens. I think
> > this
> > > > is what is intended: substitution mapping is specifically about
> mapping
> > > > reqs-and-caps. And this is also where things like scalability and
> > > placement
> > > > happen: think of a requirement matching a capability, but that
> > capability
> > > > not having enough capacity. So, the node might be able to scale out:
> in
> > > the
> > > > case of substitution, this would mean duplicating an entire service
> > > > instance. My understanding is that this is what is intended by TOSCA,
> > and
> > > > it's entirely within the scope of what we can do. We've recently just
> > > > refactored the instantiation phase into a new "topology" module where
> > > > exactly all this logic is concentrated. So think of it this way --
> ARIA
> > > has
> > > > three parts: "parser," "topology manager," and "workflow engine"
> > > > ("orchestrator").
> > > >
> > > > (I think I might have confused some thing here a bit when mentioning
> a
> > > > "logical proxy node." I do not mean an actual piece of proxy software
> > > > running on a machine somewhere. I mean just a data point in the
> > > > ARIA-generated topology that can be used as a barrier of sorts when
> > > > constructing the task graph -- because the task graph follows
> > > > relationships, the edges of the topology. It could be that we
> discover
> > in
> > > > our POC that this is not needed, because actually the substitution
> node
> > > is
> > > > already part of our topology data model and we might be able to
> easily
> > > take
> > > > that into account when generating the task graph.)
> > > >
> > > > 3) Live service composition. I think this is a fantasy of some
> people:
> > > that
> > > > ARIA would be able to take existing, "running" service chains and run
> > > > workflows with them. I do think this is a solvable problem, but not
> via
> > > > substitution mapping per se. A solution could involve deployment of a
> > > proxy
> > > > service (which could actually be encapsulated in a substitution
> > mapping,
> > > > but doesn't have to be), or configuring specialized virtual ports via
> > an
> > > > SDN controller, or via logical proxy nodes created via an inspection
> > > tool,
> > > > etc. I cannot believe that there is a one-size-fits-all solution to
> > this
> > > > problem. The dream of NFV might be to have everything connect to each
> > > other
> > > > like LEGO blocks, but there are far too many protocols,
> configuration,
> > > and
> > > > security standards. I think point #2, what I called "static service
> > > > composition," is a realistic compromise and within the scope of what
> > > TOSCA
> > > > promises.
> > > >
> > > >
> > > > On Fri, Aug 11, 2017 at 12:08 PM, DeWayne Filppi <
> dewa...@cloudify.co>
> > > > wrote:
> > > >
> > > > > To my eyes, the spec doesn't speak of runtime service substituion,
> > but
> > > > > parsetime template composition. IOW, substitution mapping is a
> fancy
> > > > > "include", or the equivalent of an interface definition. Is it
> > > > understood
> > > > > by the ARIA team that this includes proxying of running services?
> > IOW,
> > > > if
> > > > > my template requires a database service that my template does *not*
> > > want
> > > > to
> > > > > control the lifecycle of, I can "substitution map" an instance of a
> > > > > template (i.e. a running service)? This would be a lovely feature,
> > but
> > > > > it's not really a "substitution map", rather more of a "service
> > proxy"
> > > > (as
> > > > > implemented as a plugin in Cloudify). Just trying to clarify.
> > Maybe
> > > > the
> > > > > community thinks that "substitution map" as something that occurs
> > > beyond
> > > > > parsetime, or should.
> > > > >
> > > > > On Fri, Aug 11, 2017 at 9:52 AM, Tal Liron <t...@cloudify.co>
> wrote:
> > > > >
> > > > > > Well, DJ, it's all just opinion at this stage and feedback is
> > > welcome!
> > > > > >
> > > > > > Option 1:
> > > > > > > To look at satisfying nodes present in a substituting
> > > > service,
> > > > > > > Have these nodes part of the newly created service and remove
> the
> > > > > > > substituting service(nodes with different ID's. Also we are
> very
> > > much
> > > > > in
> > > > > > > favor of UUID )
> > > > > > > With this approach I guess the substituting service
> > should
> > > > not
> > > > > > > have any associated workflows running. If at all an workflow
> > > > execution
> > > > > is
> > > > > > > already triggered I hope this service will not be considered
> for
> > > > > > > substitution.
> > > > > > > I hope this is the correct approach when we are looking
> > at
> > > a
> > > > > > > service for the substitution
> > > > > > >
> > > > > >
> > > > > > Yes, this is a good idea. It would be easy to discover this
> > according
> > > > to
> > > > > > the stored node state -- just make sure that all nodes are in a
> > > stable
> > > > > > state before composition.
> > > > > >
> > > > > > This leads to a general issue: before composition, the
> substituting
> > > > > > services must be validated in some way before composition begins.
> > > > > >
> > > > > > Also, let's all start using TOSCA terminology here: the
> containing
> > > > > service
> > > > > > is called the "top-level service," and the contained services are
> > > > called
> > > > > > "substituting services."
> > > > > >
> > > > > > Also, we keep using trivial examples, but it's definitely
> possible
> > > for
> > > > a
> > > > > > top-level service to require several substituting services at the
> > > same
> > > > > > time. I can definitely see such things happening in NFV with even
> > > > simple
> > > > > > service chains. Basically every VNF could be a substituting
> > service.
> > > > > >
> > > > > > So, actually one of the validations would be to make sure you do
> > not
> > > > > create
> > > > > > circular composition: if the top-level also has substitution
> > > mappings,
> > > > > you
> > > > > > need to make sure that one of the substituting ones doesn't
> require
> > > it.
> > > > > :)
> > > > > > Not a very likely situation, but it would lead to failure.
> > > > > >
> > > > > >
> > > > > >
> > > > > > > Option 2:
> > > > > > > While creating a service look at the req-caps at the
> > > > > > > service-template level and create a service including the nodes
> > > > > provided
> > > > > > by
> > > > > > > the substituting service-template. With this approach there
> would
> > > not
> > > > > be
> > > > > > > any service created from the service-template which is
> > providing
> > > > the
> > > > > > > substitution functionality. The service-template would remain
> the
> > > > same
> > > > > > but
> > > > > > > only the service would be added with extra nodes.
> > > > > > >
> > > > > > > Are you considering both option 1 & 2 for the implementation ?
> If
> > > not
> > > > > > > which one do you feel which take priority. I see option 2 at
> this
> > > > stage
> > > > > > > could be the best possible approach
> > > > > > > Also could you please let me know a tentative time for this
> > feature
> > > > to
> > > > > be
> > > > > > > available?
> > > > > > >
> > > > > >
> > > > > > I think both options 1 and 2 make sense and are useful, and
> > actually
> > > > one
> > > > > is
> > > > > > a subset of the other.
> > > > > >
> > > > > > With option #2 (substituting a service template), it means new
> > nodes
> > > > are
> > > > > > instantiated and the composed service would include all nodes.
> So,
> > an
> > > > > > "install" workflow would install everything at once. In this case
> > we
> > > do
> > > > > > need to fix the lifecycle workflows to be "boundary aware," so
> that
> > > > > > workflows of substituting service nodes are part of their own
> task
> > > > > graph. I
> > > > > > think that possibly using a logical proxy node in between might
> > solve
> > > > > this
> > > > > > situation automatically.
> > > > > >
> > > > > > With option #1 (substituting a service) the substituting nodes
> > might
> > > > > > already be installed. Or not (they might have been instantiated
> but
> > > > still
> > > > > > not installed). So, the lifecycle workflows should only work on
> the
> > > > nodes
> > > > > > that have not yet been installed.
> > > > > >
> > > > > > The point is that we just need to beef up the lifecycle workflows
> > to
> > > > > > properly work with boundaries and with a mix of nodes in
> different
> > > > > states.
> > > > > > If they can do that, then they can handle any kind of service
> > > > > composition,
> > > > > > whether it's option #1 or option #2.
> > > > > >
> > > > > > I don't think we can provide a timeline yet, but I will say that
> we
> > > are
> > > > > in
> > > > > > the research phase and may have a POC soon. Avia is in charge of
> > this
> > > > > JIRA,
> > > > > > so I will let him chime in with the current state of things.
> > > > > >
> > > > >
> > > >
> > >
> >
>