Spark Job Server first steps

I started by developing and submitting a simple job and the JAR check gave me errors on a seemingly good jar. I disabled the fingerprint checking on the jar and I could submit it, but when I tried to submit the job, it could not find it's classpath. Therefore I decided to take a couple of steps backwards and go through the example in the docs.

Using the (Hello)WordCount example, upload is OK and the jar in in the UI as well, but when I submit the job, I get the same classpathNotFound error as before:

Re: Spark Job Server first steps

Hi Gerard,

We're using the Spark Job Server in production, from GitHub [master] running against a recent Spark-1.0 snapshot so it definitely works. I'm afraid the only time we've seen a similar error was an unfortunate case of PEBKAC.

First and foremost, have you tried doing an unzip -l "/tmp/spark-jobserver/filedao/data/test-2014-05-22T18:44:09.254+02:00.jar"on the JAR uploaded to the server to make sure the class is where you're expecting it to be?

It's not uncommon for a package statement to be neglected when moving classes around in an IDE like Eclipse.

This email and any files transmitted with it are confidential and may also be privileged. It is intended only for the person to whom it is addressed. If you have received this email in error, please inform the sender immediately. If you are not the intended recipient you must not use, disclose, copy, print, distribute or rely on this email.

I started by developing and submitting a simple job and the JAR check gave me errors on a seemingly good jar. I disabled the fingerprint checking on the jar and I could submit it, but when I tried to submit the job, it could not find it's classpath. Therefore I decided to take a couple of steps backwards and go through the example in the docs.

Using the (Hello)WordCount example, upload is OK and the jar in in the UI as well, but when I submit the job, I get the same classpathNotFound error as before:

We're using the Spark Job Server in production, from GitHub [master] running against a recent Spark-1.0 snapshot so it definitely works. I'm afraid the only time we've seen a similar error was an unfortunate case of PEBKAC.

First and foremost, have you tried doing an unzip -l "/tmp/spark-jobserver/filedao/data/test-2014-05-22T18:44:09.254+02:00.jar"on the JAR uploaded to the server to make sure the class is where you're expecting it to be?

It's not uncommon for a package statement to be neglected when moving classes around in an IDE like Eclipse.

This email and any files transmitted with it are confidential and may also be privileged. It is intended only for the person to whom it is addressed. If you have received this email in error, please inform the sender immediately. If you are not the intended recipient you must not use, disclose, copy, print, distribute or rely on this email.

I started by developing and submitting a simple job and the JAR check gave me errors on a seemingly good jar. I disabled the fingerprint checking on the jar and I could submit it, but when I tried to submit the job, it could not find it's classpath. Therefore I decided to take a couple of steps backwards and go through the example in the docs.

Using the (Hello)WordCount example, upload is OK and the jar in in the UI as well, but when I submit the job, I get the same classpathNotFound error as before: