@Pchelolo, hmmmm. eventgate in prod will need to have the event-schemas repo(s) available somehow. I'm working on getting the docker images and helm charts figured out. For the initial deployment prototype, I'm considering just making a blubber and CI based docker image that will be included in the eventgate docker image somewhere. This will work for a trial, but will be a bit inflexible, since it will mean that a new schema will require a rebuild and redeploy of eventgate.

Perhaps we should consider deploying a quick and easy (non kubernetes based) schema registry? We could set up something simple via puppet: a webserver and a git clone ensure => latest of the event-schemas repo. Thoughts?

Regarding a little http server - if it's gonna only be used for a test deployment - I'm ok with that, but it will be a blocker for putting any production load on the deployed eventgate. We can not have a production service depend on something like this.

I assume they will be behind varnish so they can take advantage of request level throttling and similar implemented at the varnish layer.

Sure the analytics one will not be exposed bare to the internet, I meant it will have endpoints that are available to the public, so in theory, it could be DDOSed.

I think we will be good with tier-1 and tier-2 installs, however since both will be in kubernetes, thus running on the same physical hardware, they will not be entirely separated as our Kafka clusters are. But I suppose it should be ok, but I am not quite sure how the isolation of different support tier services will work in kubernetes.

There's a number of ways to make a workload have a higher priority. Some are more ready that others. Namely priority and preemption [1] is not yet (it's beta in 1.11, we are at 1.10 currently) whereas affinity and anti-affinity[2] is more ready. Resource quotas[3] are also ready and we already use them. But note that there is no high level first class resource for representing"tier" (I don't love the term btw) . We will have to figure out what it means for us and implement it respectively.

@Pchelolo, hmmmm. eventgate in prod will need to have the event-schemas repo(s) available somehow. I'm working on getting the docker images and helm charts figured out. For the initial deployment prototype, I'm considering just making a blubber and CI based docker image that will be included in the eventgate docker image somewhere. This will work for a trial, but will be a bit inflexible, since it will mean that a new schema will require a rebuild and redeploy of eventgate.

The same issue is more or less met by ORES where updating the models requires a redeploy

Perhaps we should consider deploying a quick and easy (non kubernetes based) schema registry? We could set up something simple via puppet: a webserver and a git clone ensure => latest of the event-schemas repo. Thoughts?

Depending on the size and structure of the registry (I am not sure what it even is yet) we could indeed create a configMap[4] object and store the data there and have it be mounted in the container. It's valid as an approach and we already do it for the config.yaml file. But it won't scale well to large amounts of data and in fact the limit is 1MB. Other solutions like hostPath[5] would not have that limitation but will require deployments to the kubernetes hosts themselves which is ugly.

Regarding a little http server - if it's gonna only be used for a test deployment - I'm ok with that, but it will be a blocker for putting any production load on the deployed eventgate. We can not have a production service depend on something like this.

Agreed on premise, but noting that it depends on the implementation.

We can have the software reach out once during (re)-initialization to fetch data from some endpoint (HTTP(S) or otherwise). During that phase the readiness probe should return effectively a "not ready" message informing kubernetes to not route traffic to this pod. That would be fine as a pattern. More than that would probably cause problematic interactions.

@akosiaris In your doc , in the Start Up section, you mention 'Open grafana'. Can you elaborate here? :)
OH! Nevermind I see, that isn't an instruction...but a summary of what we are doing, never mind!

I 've updated the docs anyway to make it a tad more descriptive. Thanks for pointing it out!

@fselles, I'm not able to get the requirements.yamlrepository to work. The only reason my symlink works is I think because the dependency will be looked for in charts/ by default. No matter what I put for repository, if I don't have the symlink in charts/, I get

Error: found in requirements.yaml, but missing in charts/ directory: kafka-single-node

I had to manually copy the schema repo into /etc/eventgate-ci/ on deployment-eventgate-analytics-1 so that the container could access schemas, but this will change when we have a remote schema registry and when the schemas are baked into the image.