Hi,I am a newbie, I was just checking if there has been any updates on the REST proxy.I have tried to come up with a draft of REST API and can post it for review and discussion if it is still an open topic.

-DavidOn 7/18/13 6:19 PM, Yogita Bijani wrote:> Hi,> I am a newbie, I was just checking if there has been any updates on the REST proxy.> I have tried to come up with a draft of REST API and can post it for review and discussion if it is still an open topic.>> Thanks> Yogita>> Sent from my iPad

I think this is really cool and would be useful for a lot of people. David,do you think it would really benefit from being a contrib package in kafkaversus a separately hosted github project?

Here was our experience: contrib packages tend to get added but the authorsand don't necessarily want involvement in kafka internals or necessarilyfollow the mailing list to answer questions. When we take then they end upkind of stagnating because there is overhead for the code review process,and the main kafka committers aren't generally qualified to fully maintainthe code.

What we have seen is that external projects that host add ons do reallywell. The author tends to get more credit for their work, directly respondsto questions from users, etc.

On the other hand maybe we feel this is central enough that it should justbe a top-level package in kafka. I think this makes sense for things whichare closely tied to internals, are general enough that there is only onegood way to do it, and will be very common for kafka users to want. If thissounds better for the REST proxy the important thing is just that someonehas bandwidth to be able to support it and ensure we we document it, havesufficient tests to keep it in sync, etc.

Let me know what you think, if the preference is to add it to kafka then Ican help with code reviews and stuff.

> I did some work on this a while ago: https://github.com/mumrah/**> kafka/commits/rest <https://github.com/mumrah/kafka/commits/rest>>> I think people would still be interested. I know I would :)>> -David>>>> On 7/18/13 6:19 PM, Yogita Bijani wrote:>>> Hi,>> I am a newbie, I was just checking if there has been any updates on the>> REST proxy.>> I have tried to come up with a draft of REST API and can post it for>> review and discussion if it is still an open topic.>>>> Thanks>> Yogita>>>> Sent from my iPad>>>>

I think there would be many advantages to having a REST API in Kafka, as a core component. Not just for data services (producing/consuming messages), but also for admin. I could imagine a Kafka web console that showed various stats, allowed for maintenance operations, showed pretty viz of the cluster, etc.

The REST prototype I put together using Jetty was really a simplistic POC. In reality, more modern web techniques like chunked transfer encoding and streaming should be used for high throughput (in, and out). Netty would probably be a good choice for a networking library.

I'd be willing to bet that a sufficiently high performance and easy to use REST API could put a lot of the 3rd party client libraries out of business, so-to-speak.

To more directly answer your question, Jay, I think if something like this is done it should be done as a top-level component in Kafka. Is contrib even maintained/updated anymore? Hasn't Camus supplanted the hadoop stuff?

On 7/19/13 4:24 PM, Jay Kreps wrote:> I think this is really cool and would be useful for a lot of people. David,> do you think it would really benefit from being a contrib package in kafka> versus a separately hosted github project?>> Here was our experience: contrib packages tend to get added but the authors> and don't necessarily want involvement in kafka internals or necessarily> follow the mailing list to answer questions. When we take then they end up> kind of stagnating because there is overhead for the code review process,> and the main kafka committers aren't generally qualified to fully maintain> the code.>> What we have seen is that external projects that host add ons do really> well. The author tends to get more credit for their work, directly responds> to questions from users, etc.>> On the other hand maybe we feel this is central enough that it should just> be a top-level package in kafka. I think this makes sense for things which> are closely tied to internals, are general enough that there is only one> good way to do it, and will be very common for kafka users to want. If this> sounds better for the REST proxy the important thing is just that someone> has bandwidth to be able to support it and ensure we we document it, have> sufficient tests to keep it in sync, etc.>> Let me know what you think, if the preference is to add it to kafka then I> can help with code reviews and stuff.>> -Jay>>> On Thu, Jul 18, 2013 at 8:15 PM, David Arthur <[EMAIL PROTECTED]> wrote:>>> I did some work on this a while ago: https://github.com/mumrah/**>> kafka/commits/rest <https://github.com/mumrah/kafka/commits/rest>>>>> I think people would still be interested. I know I would :)>>>> -David>>>>>>>> On 7/18/13 6:19 PM, Yogita Bijani wrote:>>>>> Hi,>>> I am a newbie, I was just checking if there has been any updates on the>>> REST proxy.>>> I have tried to come up with a draft of REST API and can post it for>>> review and discussion if it is still an open topic.>>>>>> Thanks>>> Yogita>>>>>> Sent from my iPad>>>>>

> I think there would be many advantages to having a REST API in Kafka, as a> core component. Not just for data services (producing/consuming messages),> but also for admin. I could imagine a Kafka web console that showed various> stats, allowed for maintenance operations, showed pretty viz of the> cluster, etc.>> The REST prototype I put together using Jetty was really a simplistic POC.> In reality, more modern web techniques like chunked transfer encoding and> streaming should be used for high throughput (in, and out). Netty would> probably be a good choice for a networking library.>> I'd be willing to bet that a sufficiently high performance and easy to use> REST API could put a lot of the 3rd party client libraries out of business,> so-to-speak.>> To more directly answer your question, Jay, I think if something like this> is done it should be done as a top-level component in Kafka. Is contrib> even maintained/updated anymore? Hasn't Camus supplanted the hadoop stuff?>>> On 7/19/13 4:24 PM, Jay Kreps wrote:>>> I think this is really cool and would be useful for a lot of people.>> David,>> do you think it would really benefit from being a contrib package in kafka>> versus a separately hosted github project?>>>> Here was our experience: contrib packages tend to get added but the>> authors>> and don't necessarily want involvement in kafka internals or necessarily>> follow the mailing list to answer questions. When we take then they end up>> kind of stagnating because there is overhead for the code review process,>> and the main kafka committers aren't generally qualified to fully maintain>> the code.>>>> What we have seen is that external projects that host add ons do really>> well. The author tends to get more credit for their work, directly>> responds>> to questions from users, etc.>>>> On the other hand maybe we feel this is central enough that it should just>> be a top-level package in kafka. I think this makes sense for things which>> are closely tied to internals, are general enough that there is only one>> good way to do it, and will be very common for kafka users to want. If>> this>> sounds better for the REST proxy the important thing is just that someone>> has bandwidth to be able to support it and ensure we we document it, have>> sufficient tests to keep it in sync, etc.>>>> Let me know what you think, if the preference is to add it to kafka then I>> can help with code reviews and stuff.>>>> -Jay>>>>>> On Thu, Jul 18, 2013 at 8:15 PM, David Arthur <[EMAIL PROTECTED]> wrote:>>>> I did some work on this a while ago: https://github.com/mumrah/**>>> kafka/commits/rest <https://github.com/mumrah/**kafka/commits/rest<https://github.com/mumrah/kafka/commits/rest>>>> >>>>>>>>>> I think people would still be interested. I know I would :)>>>>>> -David>>>>>>>>>>>> On 7/18/13 6:19 PM, Yogita Bijani wrote:>>>>>> Hi,>>>> I am a newbie, I was just checking if there has been any updates on the>>>> REST proxy.>>>> I have tried to come up with a draft of REST API and can post it for>>>> review and discussion if it is still an open topic.>>>>>>>> Thanks>>>> Yogita>>>>>>>> Sent from my iPad>>>>>>>>>>>>

NEW: Monitor These Apps!

All projects made searchable here are trademarks of the Apache Software Foundation.
Service operated by Sematext