directory-dev mailing list archives

Re: Question about Replication of Config Partition and Schema Partition

Date

Tue, 17 Jul 2012 00:06:16 GMT

On Tue, Jul 17, 2012 at 1:28 AM, Alex Karasulu <akarasulu@apache.org> wrote:
>
>
> On Mon, Jul 16, 2012 at 6:50 PM, Emmanuel Lécharny <elecharny@gmail.com>wrote:
>
>>
>> - But i'm not against on replicating configuration completely. Some
>>>>
>>>>> one
>>>>>
>>>>> would want to simply clone the server in its entirety. ( This is
>>>>> not
>>>>> touched by RFC as i see ) Because our OSGI distribution is Karaf
>>>>> based, we
>>>>> can use Karaf Cellar here. Cellar is used to keep multiple Karaf
>>>>> instances
>>>>> synched in terms of Bundles,Features, and Configuration.(We can't
>>>>> use
>>>>> its
>>>>> configuration aspects because we're keeping our component
>>>>> configurations in
>>>>> ApacheDS itself). I quickly looked through its functionality and
>>>>> code. It's
>>>>> incomplete but promising. Most importantly it provides API to
>>>>> initiate
>>>>> Karaf Instance replication.(Again not complete). So when
>>>>> replication
>>>>> system
>>>>> is revisited we can make use of Cellar.
>>>>> - I looked at Apache ZooKeeeper a bit, and we should definetely
>>>>> leverage
>>>>>
>>>>> it in our LDAP replication system. It is simply a distributed
>>>>> data and
>>>>> notification system for clusters. It has strength in node
>>>>> management
>>>>> in
>>>>> clusters and good for implementing infrastructure related parts
of
>>>>> replication system.
>>>>>
>>>>> So i see no threat to OSGI from LDAP Replication. However i believe
>>>>> ou=config should be excluded from replication in any scenario.
>>>>>
>>>>> Not sure.
>>>>
>>>> Not completely as i say, they must be controlled through other
>>> repliccation
>>> system. I say they shouldn't be managed by LDAP replication, because as i
>>> said: 2 server having the same configuration are not have to have same
>>> runtime behaviour because of the different bundle configurations.
>>>
>> Two servers can't have the same configuration, otherwise we have a
>> problem !
>>
>> There are very few informations that are not to be replicated, because
>> they belongs to the local server (the three I mentionned : IP, port, and
>> name). That means we need to find a better way to store those informations.
>> And if we store them in the DIT, then this part of the DIT must *not* be
>> replicated.
>>
>>
> Partial and fractional replication will come in handy here where we say
> this attribute or this entry does not get replicated and it does not. That
> simple.
>
Yeah that is simple to implement. The important thing that matters here is
that, just replicating configuration means nothing. Lets say one ApacheDS
instance have installed some custom partition bundle and have a
configuration entry pointing to that custom partition implementation. So
replicating that configuration to the some other ApacheDS instance will not
magically make this custom partition available unless we implement that
magic into replication procedure. I think we can use Karaf-Cellar here if
it becomes stable enough for us to use when we review replication system.
If we can't use Cellar than we can simply describe how to replicate
configuration between 2 servers, pointing out the importance of syncing 2
server in terms of installed bundles. No problem here !
>
>
>> If this is what you were thinking of, then yes, I'm on the same page. May
>> be we should put thos information somewhere else than on the ou=config sub
>> tree (root DSE ?). We could specifically address those points.
>>
>>
> That's not something I'm feeling good about. We're going to change how we
> laid down the organization of configurations just for replication?
> Replication needs to advance to take into account these considerations
> instead of us injecting one offs to handle these replication scenarios.
>
Replication must be smart of course, through use of replication flags we
don't have to store instance local properties in some other place. Because
i believe there will be more local properties other than ip/port pair in
the future. And the same local property type can be appeared in
configuration at many place under different names, like kdc-server-port,
ldap-server-port. So Replication flags must be designed for future
flexibility. Tweaking the configuration layout does not help here.
>
>
>> I was pretty much thinking that we could store those informations in a
>> plain text file, but that would be a bit overkilling, when we can store
>> them in a the DIT too. Maybe storing those information sinto the ou=config
>> entry could be the right thing to do, assuming that the ou=config is not
>> replicated (we will only replicate what's under ou=config, ie, its children)
>>
>>
> Please please please let's not fuck with this. This is the worst idea I've
> heard of yet. We don't need another one off here.
>
Actually i don't see that much harm here. I also once thought about how it
would be to use configurations on plain-text files (OSGI Standart for so
much framework and component). Though, i don't know historical decisions
those led to this current configuration system. Maybe you can explain me
why it's that fragile?
Current OSGI layer's code have a lot of code to deal with configuration
kept in and tracked from ApacheDS Partition. It has pros and cons of
course.(pros outweight cons IMO) And i like the way we can tweak the server
from DIT. So I have no plan to change the way we store configuration.
>
>
>>
>>
>>>
>>> I believe if
>>>>
>>>>> we're gonna provide some way to replicate some ApacheDS instance's
>>>>> runtime
>>>>> layout, we should not do it by LDAP Replication. Because :
>>>>>
>>>>> - It is not just config partition manages the server anymore.
>>>>> - We just can't be sure that 2 server having the same
>>>>> configuration
>>>>> will
>>>>>
>>>>> have same runtime behavior, because they might have different
>>>>> composition
>>>>> of bundles.
>>>>> - Replicating ou=config is hell of a job because of site specific
>>>>> configuration parameters.
>>>>>
>>>>> Unless the server can detect which configuration applies to itself.
>>>>
>>>> Bottom line, there are a very litte set of information a server needs to
>>>> keep local :
>>>> - its IP address
>>>> - its port
>>>> - its name
>>>>
>>>> The name will be used to identify the configuration that applies to the
>>>> local server.
>>>>
>>> I don't believe simply keeping those 3 information local will help to
>>> make
>>> replicating configuration information across multiple nodes consistent.
>>> When thinking in terms of some system, there will be always some more
>>> local
>>> configuration points other than IP/port/hostname IMO.
>>>
>> See upper with the suggestion I made about using pu=config entry but not
>> replicating it. We could even extend the ObjectClass used to store the 3
>> pieces of informations to store more, if needed. ATM, though, this is all
>> what we need locally. Everything else can be safely replicated, assuming
>> that the code that reads and handle the configuration knows that it must be
>> combined with the logal configuration.
>>
>>
>>
>> --
>> Regards,
>> Cordialement,
>> Emmanuel Lécharny
>> www.iktek.com
>>
>>
>
>
> --
> Best Regards,
> -- Alex
>
>