I tried all the ways, but all didn't work.(I'm using CDH Hue and oozie workflow) Following were what I tried with the 4 ways:

For way 1:

It recommended "oozie.libpath=/path/to/jars,another/path/to/jars"

I add oozie.libpath=hdfs://ip-10-0-4-248.us-west-1.compute.internal:8020/user/oozie/share/lib/lib_20151201085935/spark or oozie.libpath=hdfs://ip-10-0-4-248.us-west-1.compute.internal:8020/user/oozie/share/lib/lib_20151201085935/spark/guava-16.0.1.jar

and oozie.use.system.libpath=true is by default.

Both don't work.

For way 2:

I added guava-16.0.1.jar into “lib” next to current workspace workflow.xml in HDFS, it doesn't work.

For way 3:

I can not find any <archive> tag in a Spark action with the path to a single jar, so I have no way to try way3.

For way 4:

I added guava-16.0.1.jar to the ShareLib (e.g. hdfs://ip-10-0-4-248.us-west-1.compute.internal:8020/user/oozie/share/lib/lib_20151201085935/spark) and set oozie.use.system.libpath=true in job.properties, it still doesn't work.

Could you please give any suggestion? Thanks very much for any of your help! I appreciated!

Re: how to add external guava-16.0.1.jar in CDH oozie classpath

I think you need to re-read those 4 ways a little more carefully. #1 clearly states:

There is no need to ever point [oozie.libpath] at the ShareLib location. (I see that in a lot of workflows.) Oozie knows where the ShareLib is and will include it automatically if you set oozie.use.system.libpath=true in job.properties.

which is exactly what you tried.

Any of the 4 methods should work. (Except for #3: we're currently aware of a known issue where the Spark Action does not allow <file> or <archive> tags; we're planning on fixing that in a later release.)

Though keeping in mind that you're trying to replace a jar in the Sharelib, you need to go with #4 anyway, and replace the jar in the Spark Sharelib subdir (which you did already).

Though remember that this means any Spark Action that anyone runs will have this modification. If you want to protect other users and workflows from your changes, you can create a new dir in the Sharelib, say "spark_guava_16", and set "oozie.action.sharelib.for.spark" to "spark_guava_16". This is also described in the blog post in the "Overriding the ShareLib" section. If that's not a concern, then you don't need to bother.

Please check the following:

Run the oozie admin -shareliblist spark command. It will print out a list of the jars from the Spark sharelib directory that Oozie is currently aware of and using. If you did replace the guava 14 jar with the 16 jar there, it should show up in that output. If not, you need to restart the Oozie server or run the oozie admin -sharelibupdate command. Also pay attention to the lib_<timestamp> directory in the output; perhaps you're changing an old directory

When you run the job, look at the stdout from the Launcher Job. It prints out a lot of useful information, including the classpath. Do you see the guava 16 or 14 jar there?

However, as I said in the email thread in the oozie mailing list, Spark is expecting guava 14. Guava tends to not be very compatible across major versions; so you may encounter other problems if you force it to use guava 16.