I am wondering does metadata of Kafka stores producer/consumer information.

As I start any producer,I provide a broker-list to it.But producer must beconnecting to one of the broker to send data(as per Im gettiing thethings).Similarly I can start more producer by giving a broker-list in thestart command.

So a broker must be storing the information about which producer ishandshaking with it(as a producer will connect to only one broker at anytime).

Same case goes with consumer.

is there any way to find the details about Producer/Consumer from Kafka,ifI am right in explaining the above details?

>> I am wondering does metadata of Kafka stores producer/consumer information.>> As I start any producer,I provide a broker-list to it.But producer must be> connecting to one of the broker to send data(as per Im gettiing the> things).Similarly I can start more producer by giving a broker-list in the> start command.>> So a broker must be storing the information about which producer is> handshaking with it(as a producer will connect to only one broker at any> time).

Not really - well, brokers obviously have to maintain theseconnections from producers but it maintains no additional per-producerstate.

>> Same case goes with consumer.

Same for consumers. For high-level consumers, the consumers themselvesmaintain state (e.g., registration, partition ownership, offsets) inzookeeper but there is no per-consumer state on the broker (althoughwith KAFKA-1000, we will be moving offset management to the broker).

> Hi,>>>> I am wondering does metadata of Kafka stores producer/consumer information.>> As I start any producer,I provide a broker-list to it.But producer must be> connecting to one of the broker to send data(as per Im gettiing the> things).Similarly I can start more producer by giving a broker-list in the> start command.>> So a broker must be storing the information about which producer is> handshaking with it(as a producer will connect to only one broker at any> time).>> Same case goes with consumer.>> is there any way to find the details about Producer/Consumer from Kafka,if> I am right in explaining the above details?>> --> *Moniii*>

So Joel,can we get the simple details that how many producers's connectiona broker is maintaining?On Fri, Oct 25, 2013 at 8:25 PM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Details of the consumer implementation can be found at>> http://research.microsoft.com/en-us/um/people/srikanth/netdb11/netdb11papers/netdb11-final12.pdf>> Thanks,>> Jun>>> On Thu, Oct 24, 2013 at 11:13 PM, Monika Garg <[EMAIL PROTECTED]>> wrote:>> > Hi,> >> >> >> > I am wondering does metadata of Kafka stores producer/consumer> information.> >> > As I start any producer,I provide a broker-list to it.But producer must> be> > connecting to one of the broker to send data(as per Im gettiing the> > things).Similarly I can start more producer by giving a broker-list in> the> > start command.> >> > So a broker must be storing the information about which producer is> > handshaking with it(as a producer will connect to only one broker at any> > time).> >> > Same case goes with consumer.> >> > is there any way to find the details about Producer/Consumer from> Kafka,if> > I am right in explaining the above details?> >> > --> > *Moniii*> >>

> So Joel,can we get the simple details that how many producers's connection> a broker is maintaining?

I don't think we directly maintain that information but it is readilyavailable through standard OS-level tools (e.g., netstat).

Thanks,

Joel

>>> On Fri, Oct 25, 2013 at 8:25 PM, Jun Rao <[EMAIL PROTECTED]> wrote:>>> Details of the consumer implementation can be found at>>>> http://research.microsoft.com/en-us/um/people/srikanth/netdb11/netdb11papers/netdb11-final12.pdf>>>> Thanks,>>>> Jun>>>>>> On Thu, Oct 24, 2013 at 11:13 PM, Monika Garg <[EMAIL PROTECTED]>>> wrote:>>>> > Hi,>> >>> >>> >>> > I am wondering does metadata of Kafka stores producer/consumer>> information.>> >>> > As I start any producer,I provide a broker-list to it.But producer must>> be>> > connecting to one of the broker to send data(as per Im gettiing the>> > things).Similarly I can start more producer by giving a broker-list in>> the>> > start command.>> >>> > So a broker must be storing the information about which producer is>> > handshaking with it(as a producer will connect to only one broker at any>> > time).>> >>> > Same case goes with consumer.>> >>> > is there any way to find the details about Producer/Consumer from>> Kafka,if>> > I am right in explaining the above details?>> >>> > -->> > *Moniii*>> >>>>>>> --> *Moniii*

NEW: Monitor These Apps!

All projects made searchable here are trademarks of the Apache Software Foundation.
Service operated by Sematext