I have a build of Flume 1.4.0 on FreeBSD 8, installed into /usr/local/flume. I manually created /usr/local/flume/plugins.d/flume-ng-cassandra-sink anduntarred flume-ng-cassandra-sink-1.0.0-SNAPSHOT-dist.tar.gz (from a localpull/build of btoddb's sources on github), so I have a lib/ directorythat's completely populated with both the sink plugin and any possiblejarfiles it might rely on.

I've likewise tried just copying flume-ng-cassandra-sink-1.0.0-SNAPSHOT.jarto flume/lib/ for it to get picked up.

Leave the Cassandra sink jar with its other jars in a folder some where. InFLUME_HOME/conf, open flume-env.sh and go to the FLUME_CLASSPATH line.Uncomment it if needed. Append the path to your cassandra sink and itsassociated jars, and see if that works.

There doesn't appear to be a version incompatability; simply, the Cassandrajar is not found in the classpath. And the reason for that is it looks likeyou have a typo in your flume.conf.

The reason I don't recommend using it is that I didn't know it existed :P

That seems like a great place to put it. I've done quite a bit ofcustomization, so for me it's easier to just point Flume to the ./targetdirectory of my java projects. But the plugins.d seems like a great placefor third-party libraries.

> Awesome, thanks; it was that typo.>> The plugins.d directory seems to be working, is there a particular reason> you don't recommend using? I'm brand-spanking new to Flume, so I'm curious.>> Thanks again,>> Jeff>>> On Mon, Jan 14, 2013 at 9:06 PM, Connor Woodson <[EMAIL PROTECTED]>wrote:>>> Leave the Cassandra sink jar with its other jars in a folder some where.>> In FLUME_HOME/conf, open flume-env.sh and go to the FLUME_CLASSPATH line.>> Uncomment it if needed. Append the path to your cassandra sink and its>> associated jars, and see if that works.>>>> There doesn't appear to be a version incompatability; simply, the>> Cassandra jar is not found in the classpath. And the reason for that is it>> looks like you have a typo in your flume.conf.>>>> com.*t*btoddb.flume.sinks.cassandra.CassandraSink>>>> should probably be>>>> com.btoddb.flume.sinks.cassandra.CassandraSink>>>> - Connor>>>>>> On Mon, Jan 14, 2013 at 5:57 PM, Jeff Bachtel <[EMAIL PROTECTED]>wrote:>>>>> I have a build of Flume 1.4.0 on FreeBSD 8, installed into>>> /usr/local/flume . I manually created>>> /usr/local/flume/plugins.d/flume-ng-cassandra-sink and>>> untarred flume-ng-cassandra-sink-1.0.0-SNAPSHOT-dist.tar.gz (from a local>>> pull/build of btoddb's sources on github), so I have a lib/ directory>>> that's completely populated with both the sink plugin and any possible>>> jarfiles it might rely on.>>>>>> I've likewise tried just>>> copying flume-ng-cassandra-sink-1.0.0-SNAPSHOT.jar to flume/lib/ for it to>>> get picked up.>>>>>> When I have a CassandraSink define in flume.conf as such:>>> webserver.sinks.cassandraSink.type >>> com.tbtoddb.flume.sinks.cassandra.CassandraSink>>> webserver.sinks.cassandraSink.channel = memoryChannel>>> webserver.sinks.cassandraSink.hosts = localhost>>> webserver.sinks.cassandraSink.cluster-name = MyCluster>>> webserver.sinks.cassandraSink.keyspace-name = ApacheLogs>>> webserver.sinks.cassandraSink.records-coldfam = Requests>>>>>> and try to start Flume with:>>>>>> /usr/local/flume/bin/flume-ng agent -c /usr/local/flume/conf/ -f>>> /usr/local/flume/conf/flume.conf -n webserver>>> -Dflume.root.logger=DEBUG,console>>>>>>>>> I get the following output (and obviously, no success):>>> 2013-01-14 19:55:33,594 (conf-file-poller-0) [ERROR ->>> org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:142)]>>> Failed to load configuration data. Exception follows.>>> org.apache.flume.FlumeException: Unable to load sink type:>>> com.tbtoddb.flume.sinks.cassandra.CassandraSink, class:>>> com.tbtoddb.flume.sinks.cassandra.CassandraSink>>> at>>> org.apache.flume.sink.DefaultSinkFactory.getClass(DefaultSinkFactory.java:69)>>> at>>> org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.java:41)>>> at>>> org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:415)>>> at>>> org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:103)>>> at>>> org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)>>> at>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)>>> at>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)>>> at>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)

Great to hear the plugin directory is being used. Just to confirm, doyou have the following:

/usr/local/flume/plugins.d/flume-ng-cassandra-sink/lib/

correct?

Generally the plugin.d was created so this:

/usr/local/flume/plugins.d/plugin/lib/

would have the plugin jar and this:

/usr/local/flume/plugins.d/plugin/libext/

would have any jars the plugin depended on.

Brock

On Mon, Jan 14, 2013 at 6:17 PM, Connor Woodson <[EMAIL PROTECTED]> wrote:> The reason I don't recommend using it is that I didn't know it existed :P>> That seems like a great place to put it. I've done quite a bit of> customization, so for me it's easier to just point Flume to the ./target> directory of my java projects. But the plugins.d seems like a great place> for third-party libraries.>> Glad I could help!>> - Connor>>> On Mon, Jan 14, 2013 at 6:12 PM, Jeff Bachtel <[EMAIL PROTECTED]>> wrote:>>>> Awesome, thanks; it was that typo.>>>> The plugins.d directory seems to be working, is there a particular reason>> you don't recommend using? I'm brand-spanking new to Flume, so I'm curious.>>>> Thanks again,>>>> Jeff>>>>>> On Mon, Jan 14, 2013 at 9:06 PM, Connor Woodson <[EMAIL PROTECTED]>>> wrote:>>>>>> Leave the Cassandra sink jar with its other jars in a folder some where.>>> In FLUME_HOME/conf, open flume-env.sh and go to the FLUME_CLASSPATH line.>>> Uncomment it if needed. Append the path to your cassandra sink and its>>> associated jars, and see if that works.>>>>>> There doesn't appear to be a version incompatability; simply, the>>> Cassandra jar is not found in the classpath. And the reason for that is it>>> looks like you have a typo in your flume.conf.>>>>>> com.tbtoddb.flume.sinks.cassandra.CassandraSink>>>>>> should probably be>>>>>> com.btoddb.flume.sinks.cassandra.CassandraSink>>>>>> - Connor>>>>>>>>> On Mon, Jan 14, 2013 at 5:57 PM, Jeff Bachtel <[EMAIL PROTECTED]>>>> wrote:>>>>>>>> I have a build of Flume 1.4.0 on FreeBSD 8, installed into>>>> /usr/local/flume . I manually created>>>> /usr/local/flume/plugins.d/flume-ng-cassandra-sink and untarred>>>> flume-ng-cassandra-sink-1.0.0-SNAPSHOT-dist.tar.gz (from a local pull/build>>>> of btoddb's sources on github), so I have a lib/ directory that's completely>>>> populated with both the sink plugin and any possible jarfiles it might rely>>>> on.>>>>>>>> I've likewise tried just copying>>>> flume-ng-cassandra-sink-1.0.0-SNAPSHOT.jar to flume/lib/ for it to get>>>> picked up.>>>>>>>> When I have a CassandraSink define in flume.conf as such:>>>> webserver.sinks.cassandraSink.type >>>> com.tbtoddb.flume.sinks.cassandra.CassandraSink>>>> webserver.sinks.cassandraSink.channel = memoryChannel>>>> webserver.sinks.cassandraSink.hosts = localhost>>>> webserver.sinks.cassandraSink.cluster-name = MyCluster>>>> webserver.sinks.cassandraSink.keyspace-name = ApacheLogs>>>> webserver.sinks.cassandraSink.records-coldfam = Requests>>>>>>>> and try to start Flume with:>>>>>>>> /usr/local/flume/bin/flume-ng agent -c /usr/local/flume/conf/ -f>>>> /usr/local/flume/conf/flume.conf -n webserver>>>> -Dflume.root.logger=DEBUG,console>>>>>>>>>>>> I get the following output (and obviously, no success):>>>> 2013-01-14 19:55:33,594 (conf-file-poller-0) [ERROR ->>>> org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:142)]>>>> Failed to load configuration data. Exception follows.>>>> org.apache.flume.FlumeException: Unable to load sink type:>>>> com.tbtoddb.flume.sinks.cassandra.CassandraSink, class:>>>> com.tbtoddb.flume.sinks.cassandra.CassandraSink>>>> at>>>> org.apache.flume.sink.DefaultSinkFactory.getClass(DefaultSinkFactory.java:69)>>>> at>>>> org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.java:41)>>>> at>>>> org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:415)