MR-279: 0 map, 0 reduce job fails with Null Pointer Exception

Details

Description

A 0 map, 0 reduce job fails with an NPE. This case works fine on hadoop-0.20.x. The job should succeed and run setup/cleanup code - with no tasks. Below is the stacktrace:

11/06/05 19:35:37 WARN mapred.ClientServiceDelegate:
StackTrace: java.lang.NullPointerException
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.getTaskAttemptCompletionEvents(JobImpl.java:498)
at
org.apache.hadoop.mapreduce.v2.app.client.MRClientService$MRClientProtocolHandler.getTaskAttemptCompletionEvents(MRClientService.java:290)
at
org.apache.hadoop.mapreduce.v2.api.impl.pb.service.MRClientProtocolPBServiceImpl.getTaskAttemptCompletionEvents(MRClientProtocolPBServiceImpl.java:139)
at
org.apache.hadoop.yarn.proto.MRClientProtocol$MRClientProtocolService$2.callBlockingMethod(MRClientProtocol.java:195)
at org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$TunnelResponder.call(ProtoOverHadoopRpcEngine.java:168)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:420)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1406)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1402)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1094)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1400)

Hadoop QA
added a comment - 23/Jun/11 15:53 -1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12483611/MAPREDUCE-2618.patch
against trunk revision 1138301.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/412//console
This message is automatically generated.

Jeffrey Naisbitt
added a comment - 23/Jun/11 18:46 I thought I had run all the tests, but apparently I didn't pay enough attention to the results...
I originally made checkJobCompleteSuccess public for testing, but I changed things so it does only need to be package-scoped now - good catch.
I renamed logJobHistory as you recommended.
Thanks for the review! I'll submit an updated patch soon.

Jeffrey Naisbitt
added a comment - 23/Jun/11 18:51 New patch addresses Siddharth's comments.
The failing unit test was actually testing the condition this patch reverses. I modified that test and removed the equivalent test I had placed in my new tests.

Hadoop QA
added a comment - 23/Jun/11 19:03 -1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12483631/MAPREDUCE-2618-v2.patch
against trunk revision 1138301.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/414//console
This message is automatically generated.