[ https://issues.apache.org/jira/browse/ATLAS-683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-683:
------------------------------
Assignee: venkata madugundu
> Refactor local type-system cache with cache provider interface
> --------------------------------------------------------------
>
> Key: ATLAS-683
> URL: https://issues.apache.org/jira/browse/ATLAS-683
> Project: Atlas
> Issue Type: Sub-task
> Affects Versions: 0.7-incubating
> Reporter: venkata madugundu
> Assignee: venkata madugundu
> Priority: Critical
> Labels: high-availability, performance, scalability
> Fix For: 0.7-incubating
>
>
> As noted in ATLAS-488, local type-system cache makes Atlas runtime stateful and prevents
multiple Atlas instances to be active in a cluster. Either the type-cache should be synched
across Atlas instances (on all type create/update requests) or the type-cache should be moved
out of Atlas to something like a distributed cache.
> 1. As a first step, the local type-cache code in TypeSystem.java can be refactored to
be carved out as an interface like TypeCacheProvider (whose default implementation for a standalone
Atlas server would just use in-process local cache). The cache provider implementation itself
could be specified as an optional configuration property. Expert users of Atlas can choose
to inject a custom cache provider which can likely hit a distributed cache. We are evaluating
the use of a distributed cache.
> 2. As a second step, some more refactoring can be done to minimize/optimize the calls
made to TypeSystem for type lookup queries. Essentially, in a given transaction/request, once
a type lookup is done, it should not be requeried again. A request scoped variable (guice
would probably help with that scoping) can hold all the lookups made in a request. This might
sound like a cache of a cache, but I think it should help in reducing the hits to cache provider
if the provider is hitting a remote cache.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)