Advertising

> Further thinking on implementation for k8s, we need to improve in 3 places.
>
> 1.) Need to introduce min=0 for autoscaling policies
> kubectl autoscale rc foo --min=0 --max=5 --inflight-request-count=80
>
> 2.) Have to config auto scaler for use load balancing factor
> (inflight-request-count) - K8S have extension in Auto scaler
>
> 3.) Improve load balancer for hold first request (until service running)
>
Do you mean load balancers like nginx, haproxy? Or can we get this done
with gateway itself without worrying about load balancers being used?
thank you.
>
> On Tue, Oct 11, 2016 at 2:24 PM, Imesh Gunaratne <im...@wso2.com> wrote:
>
>> On Mon, Oct 10, 2016 at 8:01 PM, Sanjeewa Malalgoda <sanje...@wso2.com>
>> wrote:
>>
>>>
>>> When we do container based deployment standard approach we discussed so
>>> far was,
>>>
>>> - At the first request check the tenant and service from URL and do
>>> lookup for running instances.
>>> - If matching instance available route traffic to that.
>>> - Else spawn new instance using template(or image). When we spawn
>>> this new instance we need to let it know what is the current tenant and
>>> data sources, configurations it should use.
>>> - Then route requests to new node.
>>> - After some idle time this instance may terminate.
>>>
>>> ​If we were to do this with a container cluster manager, I think we
>> would need to implement a custom scheduler (an entity similar to HPA in
>> K8S) to handle the orchestration process properly. Otherwise it would be
>> difficult to use the built-in orchestration features such as auto-healing
>> and autoscaling with this feature.
>>
>> By saying that this might be a feature which should be implemented at the
>> container cluster manager.
>>
>> *Suggestion*
>>> If we maintain hot pool(started and ready to serve requests) of servers
>>> for each server type(API Gateway, Identity Server etc) then we can cutoff
>>> server startup time + IaaS level spawn time from above process. Then when
>>> requests comes to wso2.com tenants API Gateway we can pick instance
>>> from gateway instance pool and set wso2.com tenant context and data
>>> source using service call(assuming setting context and configurations is
>>> much faster).
>>>
>>
>> ​I think with this approach tenant isolation will become a problem. It
>> would be ideal to use tenancy features at the container cluster manager
>> level. For an example namespaces in K8S.
>>
>> Thanks
>>
>>>
>>> *Implementation*
>>> For this we need to implement some plug-in to instance spawn process.
>>> Then instead of spawning new instance it will pick one instance from the
>>> pool and configure it to behave as specific tenant.
>>> For this each instance running in pool can open up port, so load
>>> balancer or scaling component can call it and tell what is the tenant and
>>> configurations.
>>> Once it configured server close that configuration port and start
>>> traffic serving.
>>> After some idle time this instance may terminate.
>>>
>>> This approach will help us if we met following condition.
>>> (Instance loading time + Server startup time + Server Lookup) *>*
>>> (Server Lookup + Loading configuration and tenant of running server from
>>> external call)
>>>
>>> Any thoughts on this?
>>>
>>> Thanks,
>>> sanjeewa.
>>> --
>>>
>>> *Sanjeewa Malalgoda*
>>> WSO2 Inc.
>>> Mobile : +94713068779
>>>
>>> <http://sanjeewamalalgoda.blogspot.com/>blog
>>> :http://sanjeewamalalgoda.blogspot.com/
>>> <http://sanjeewamalalgoda.blogspot.com/>
>>>
>>>
>>>
>>
>>
>> --
>> *Imesh Gunaratne*
>> Software Architect
>> WSO2 Inc: http://wso2.com
>> T: +94 11 214 5345 M: +94 77 374 2057
>> W: https://medium.com/@imesh TW: @imesh
>> lean. enterprise. middleware
>>
>>
>
>
> --
> Lakmal Warusawithana
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
> _______________________________________________
> Architecture mailing list
> Architecture@wso2.org
> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
>
>
--
Manjula Rathnayaka
Technical Lead
WSO2, Inc.
Mobile:+94 77 743 1987