I'm making my own scheduler. Since I had errors, I rewrote the scheduler todo just a simple thing. When I give it a job, it allocates 1 map() and 1reduce() to it. But that isn't happening. I used a barebones WordCount pgm.I'm stuck at map: 0% reduce 0%. And in the jobtracker logs, I see:

Here's some snippets of code. The scheduler logic is inside theassignTasks():

We get a JobInProgressListener in the constructor-this.jobQueueJobInProgressListener = new JobQueueJobInProgressListener();

In start() -this.taskTrackerManager.addJobInProgressListener(jobQueueJobInProgressListener);

Then I just take the last element in that queue (I assume its the latestsubmitted job) and assign to incomingJob (of type JobInProgress).

Then I obtain a new map task as follows and append this task to a listwhich is the return value of the assignTasks():Task createdMap = incomingJob.obtainNewMapTask(taskTrackerStatus,numTaskTrackers, taskTrackerManager.getNumberOfUniqueHosts());I've attached the scheduler. Can any expert point out if there's anyignorant mistakes from my side?

I'm making my own scheduler. Since I had errors, I rewrote the scheduler to do just a simple thing. When I give it a job, it allocates 1 map() and 1 reduce() to it. But that isn't happening. I used a barebones WordCount pgm. I'm stuck at map: 0% reduce 0%. And in the jobtracker logs, I see:

Here's some snippets of code. The scheduler logic is inside the assignTasks():We get a JobInProgressListener in the constructor-this.jobQueueJobInProgressListener = new JobQueueJobInProgressListener();

In start() -this.taskTrackerManager.addJobInProgressListener(jobQueueJobInProgressListener);

Then I just take the last element in that queue (I assume its the latest submitted job) and assign to incomingJob (of type JobInProgress).

Then I obtain a new map task as follows and append this task to a list which is the return value of the assignTasks():Task createdMap = incomingJob.obtainNewMapTask(taskTrackerStatus, numTaskTrackers, taskTrackerManager.getNumberOfUniqueHosts());

I've attached the scheduler. Can any expert point out if there's any ignorant mistakes from my side?

> I'm making my own scheduler. Since I had errors, I rewrote the scheduler> to do just a simple thing. When I give it a job, it allocates 1 map() and 1> reduce() to it. But that isn't happening. I used a barebones WordCount pgm.> I'm stuck at map: 0% reduce 0%. And in the jobtracker logs, I see:>> INFO org.apache.hadoop.mapred.JobInProgress: Cannot create task split for> <my-job-id>>> DETAILED INFO:>> Here's some snippets of code. The scheduler logic is inside the> assignTasks():>> We get a JobInProgressListener in the constructor-> this.jobQueueJobInProgressListener = new JobQueueJobInProgressListener();>> In start() ->> this.taskTrackerManager.addJobInProgressListener(jobQueueJobInProgressListener);>> Then I just take the last element in that queue (I assume its the latest> submitted job) and assign to incomingJob (of type JobInProgress).>> Then I obtain a new map task as follows and append this task to a list> which is the return value of the assignTasks():> Task createdMap = incomingJob.obtainNewMapTask(taskTrackerStatus,> numTaskTrackers, taskTrackerManager.getNumberOfUniqueHosts());>>> I've attached the scheduler. Can any expert point out if there's any> ignorant mistakes from my side?>

*Yet the **Cannot create task split for <my-job-id>** is still there*.About errors relating to the JobID, is this significant? --- INFOorg.apache.hadoop.mapred.JobInProgress: job_201307111248_0002: nMaps=1nReduces=1 max=-1

> I've attached the relevant portion of the log.>> For easy testing, *I've attached the maven project* for the> EDFJobScheduler. Just make a JAR from it and put it in $HADOOP_HOME/lib and> add this property to *mapred-site.xml*:>> *<property>> *> * <name>mapred.jobtracker.taskScheduler</name>> <value>org.apache.hadoop.mapred.EDFJobScheduler</value>> *> *</property>*>>> On Thu, Jul 11, 2013 at 12:19 PM, Sreejith Ramakrishnan <> [EMAIL PROTECTED]> wrote:>>> I'm making my own scheduler. Since I had errors, I rewrote the scheduler>> to do just a simple thing. When I give it a job, it allocates 1 map() and 1>> reduce() to it. But that isn't happening. I used a barebones WordCount pgm.>> I'm stuck at map: 0% reduce 0%. And in the jobtracker logs, I see:>>>> INFO org.apache.hadoop.mapred.JobInProgress: Cannot create task split for>> <my-job-id>>>>> DETAILED INFO:>>>> Here's some snippets of code. The scheduler logic is inside the>> assignTasks():>>>> We get a JobInProgressListener in the constructor->> this.jobQueueJobInProgressListener = new JobQueueJobInProgressListener();>>>> In start() ->>>> this.taskTrackerManager.addJobInProgressListener(jobQueueJobInProgressListener);>>>> Then I just take the last element in that queue (I assume its the latest>> submitted job) and assign to incomingJob (of type JobInProgress).>>>> Then I obtain a new map task as follows and append this task to a list>> which is the return value of the assignTasks():>> Task createdMap = incomingJob.obtainNewMapTask(taskTrackerStatus,>> numTaskTrackers, taskTrackerManager.getNumberOfUniqueHosts());>>>>>> I've attached the scheduler. Can any expert point out if there's any>> ignorant mistakes from my side?>>>>

You don't need to explicitly call the initTasks(), you can find this invocation in JobTracker.java class.

>*Yet the **Cannot create task split for <my-job-id>** is still there*.>About errors relating to the JobID, is this significant? --- INFO>org.apache.hadoop.mapred.JobInProgress: job_201307111248_0002: nMaps=1>nReduces=1 max=-1

Here it shows max=-1, which means max configured(i.e default) tasks for the Job is -1.

*Yet the **Cannot create task split for <my-job-id>** is still there*.About errors relating to the JobID, is this significant? --- INFOorg.apache.hadoop.mapred.JobInProgress: job_201307111248_0002: nMaps=1nReduces=1 max=-1

> I've attached the relevant portion of the log.>> For easy testing, *I've attached the maven project* for the > EDFJobScheduler. Just make a JAR from it and put it in > $HADOOP_HOME/lib and add this property to *mapred-site.xml*:>> *<property>> *> * <name>mapred.jobtracker.taskScheduler</name>> <value>org.apache.hadoop.mapred.EDFJobScheduler</value>> *> *</property>*>>> On Thu, Jul 11, 2013 at 12:19 PM, Sreejith Ramakrishnan < > [EMAIL PROTECTED]> wrote:>>> I'm making my own scheduler. Since I had errors, I rewrote the >> scheduler to do just a simple thing. When I give it a job, it >> allocates 1 map() and 1>> reduce() to it. But that isn't happening. I used a barebones WordCount pgm.>> I'm stuck at map: 0% reduce 0%. And in the jobtracker logs, I see:>>>> INFO org.apache.hadoop.mapred.JobInProgress: Cannot create task split >> for <my-job-id>>>>> DETAILED INFO:>>>> Here's some snippets of code. The scheduler logic is inside the>> assignTasks():>>>> We get a JobInProgressListener in the constructor- >> this.jobQueueJobInProgressListener = new >> JobQueueJobInProgressListener();>>>> In start() ->>>> this.taskTrackerManager.addJobInProgressListener(jobQueueJobInProgres>> sListener);>>>> Then I just take the last element in that queue (I assume its the >> latest submitted job) and assign to incomingJob (of type JobInProgress).>>>> Then I obtain a new map task as follows and append this task to a >> list which is the return value of the assignTasks():>> Task createdMap = incomingJob.obtainNewMapTask(taskTrackerStatus,>> numTaskTrackers, taskTrackerManager.getNumberOfUniqueHosts());>>>>>> I've attached the scheduler. Can any expert point out if there's any >> ignorant mistakes from my side?>>>>