I have a hadoop cluster with 3 nodes, the network topology is like this:1. For each DataNode, its IP address is like :192.168.0.XXX;2. For the NameNode, it has two network cards: one is connect with theDataNodes as a local LAN with IP address 192.168.0.110, while the other oneis connect to the company network(which eventually connect to the Internet);--now I'm trying to debug a MapReduce program on a computer which is in thecompany network. Since the jobtracker in this scenario is 192.168.0.110:9001,I was wondering how could I connect to the cluster by using eclipse?

From: Jason Yang [mailto:[EMAIL PROTECTED]]Sent: miércoles, 04 de julio de 2012 11:25To: mapreduce-userSubject: How to connect to a cluster by using eclipse

Hi, all

I have a hadoop cluster with 3 nodes, the network topology is like this:1. For each DataNode, its IP address is like :192.168.0.XXX;2. For the NameNode, it has two network cards: one is connect with the DataNodes as a local LAN with IP address 192.168.0.110, while the other one is connect to the company network(which eventually connect to the Internet);--now I'm trying to debug a MapReduce program on a computer which is in the company network. Since the jobtracker in this scenario is 192.168.0.110:9001<http://192.168.0.110:9001>, I was wondering how could I connect to the cluster by using eclipse?

--YANG, Lin________________________________Subject to local law, communications with Accenture and its affiliates including telephone calls and emails (including content), may be monitored by our systems for the purposes of security and the assessment of internal compliance with Accenture policy.______________________________________________________________________________________

However, I was still wonder whether I could debug a MR application in thisway.

I have read some posts talking about using NAT to redirect all the packetsto the network card which connect to the local LAN, but it does not work asI tried to redirect by using iptables :(

在 2012年7月4日星期三， 写道：

> Jason,>>> the easiest way to debug a MapRedupe program with eclipse is working> on hadoop local.> http://hadoop.apache.org/common/docs/r0.20.2/quickstart.html#Local In> this mode all the components run locally on the same VM and can be easily> debugged using Eclipse. Hope this will be useful.>>>> *From:* Jason Yang [mailto:[EMAIL PROTECTED] <javascript:_e({},> 'cvml', '[EMAIL PROTECTED]');>]> *Sent:* miércoles, 04 de julio de 2012 11:25> *To:* mapreduce-user> *Subject:* How to connect to a cluster by using eclipse>>>> Hi, all>>>> I have a hadoop cluster with 3 nodes, the network topology is like this:>> 1. For each DataNode, its IP address is like :192.168.0.XXX;>> 2. For the NameNode, it has two network cards: one is connect with the> DataNodes as a local LAN with IP address 192.168.0.110, while the other one> is connect to the company network(which eventually connect to the Internet);>> -->> now I'm trying to debug a MapReduce program on a computer which is in the> company network. Since the jobtracker in this scenario is> 192.168.0.110:9001, I was wondering how could I connect to the cluster by> using eclipse?>>>> -->> YANG, Lin>>>> ------------------------------> Subject to local law, communications with Accenture and its affiliates> including telephone calls and emails (including content), may be monitored> by our systems for the purposes of security and the assessment of internal> compliance with Accenture policy.>> ______________________________________________________________________________________>> www.accenture.com>-- YANG, Lin

Jason,Ramon is right.The best way to debug a MapReduce job is mounting a local cluster, andthen, when you have tested enough your code, then, you candeploy it in a real distributed cluster.On 07/04/2012 10:00 PM, Jason Yang wrote:> ramon,>> Thank for your reply very much.>> However, I was still wonder whether I could debug a MR application in> this way.>> I have read some posts talking about using NAT to redirect all the> packets to the network card which connect to the local LAN, but it> does not work as I tried to redirect by using iptables :(>> 锟斤拷 2012锟斤拷7锟斤拷4锟斤拷锟斤拷锟斤拷锟斤拷 写锟斤拷锟斤拷>> Jason,>>> the easiest way to debug a MapRedupe program with eclipse is> working on hadoop local.> http://hadoop.apache.org/common/docs/r0.20.2/quickstart.html#Local>>> In this mode all the components run locally on the same VM and> can be easily debugged using Eclipse.>>> Hope this will be useful.>> *From:*Jason Yang [mailto:[EMAIL PROTECTED]> <javascript:_e({}, 'cvml', '[EMAIL PROTECTED]');>]> *Sent:* mi锟斤拷rcoles, 04 de julio de 2012 11:25> *To:* mapreduce-user> *Subject:* How to connect to a cluster by using eclipse>> Hi, all>> I have a hadoop cluster with 3 nodes, the network topology is like> this:>> 1. For each DataNode, its IP address is like :192.168.0.XXX;>> 2. For the NameNode, it has two network cards: one is connect with> the DataNodes as a local LAN with IP address 192.168.0.110, while> the other one is connect to the company network(which eventually> connect to the Internet);>> -->> now I'm trying to debug a MapReduce program on a computer which is> in the company network. Since the jobtracker in this scenario is> 192.168.0.110:9001 <http://192.168.0.110:9001>, I was wondering> how could I connect to the cluster by using eclipse?>> -- >> YANG, Lin>>> ------------------------------------------------------------------------> Subject to local law, communications with Accenture and its> affiliates including telephone calls and emails (including> content), may be monitored by our systems for the purposes of> security and the assessment of internal compliance with Accenture> policy.> ______________________________________________________________________________________>> www.accenture.com <http://www.accenture.com>>>>> -- > YANG, Lin>

> Jason,> Ramon is right.> The best way to debug a MapReduce job is mounting a local cluster, and> then, when you have tested enough your code, then, you can> deploy it in a real distributed cluster.> On 07/04/2012 10:00 PM, Jason Yang wrote:>> ramon,>> Thank for your reply very much.>> However, I was still wonder whether I could debug a MR application in> this way.>> I have read some posts talking about using NAT to redirect all the> packets to the network card which connect to the local LAN, but it does not> work as I tried to redirect by using iptables :(>> 在 2012年7月4日星期三， 写道��>>> Jason,>>>>>> the easiest way to debug a MapRedupe program with eclipse is working on>> hadoop local.>> http://hadoop.apache.org/common/docs/r0.20.2/quickstart.html#Local In>> this mode all the components run locally on the same VM and can be easily>> debugged using Eclipse. Hope this will be useful.>>>>>>>> *From:* Jason Yang [mailto:[EMAIL PROTECTED]]>> *Sent:* miércoles, 04 de julio de 2012 11:25>> *To:* mapreduce-user>> *Subject:* How to connect to a cluster by using eclipse>>>>>>>> Hi, all>>>>>>>> I have a hadoop cluster with 3 nodes, the network topology is like this:>>>> 1. For each DataNode, its IP address is like :192.168.0.XXX;>>>> 2. For the NameNode, it has two network cards: one is connect with the>> DataNodes as a local LAN with IP address 192.168.0.110, while the other one>> is connect to the company network(which eventually connect to the Internet);>>>> -->>>> now I'm trying to debug a MapReduce program on a computer which is in the>> company network. Since the jobtracker in this scenario is>> 192.168.0.110:9001, I was wondering how could I connect to the cluster>> by using eclipse?>>>>>>>> -->>>> YANG, Lin>>>>>>>> ------------------------------>> Subject to local law, communications with Accenture and its affiliates>> including telephone calls and emails (including content), may be monitored>> by our systems for the purposes of security and the assessment of internal>> compliance with Accenture policy.>>>> ______________________________________________________________________________________>>>> www.accenture.com>>>>> --> YANG, Lin>>> --> **>> Marcos Luis Ortíz Valmaseda> *Data Engineer && Sr. System Administrator at UCI*> about.me/marcosortiz> My Blog <http://marcosluis2186.posterous.com>> @marcosluis2186 <http://twitter.com/marcosluis2186>> **>>>> <http://www.uci.cu/>>>

for mapred.java.child.opts for the job (sorry forgot the exact param name,maybe not exactly mapred.java.child.opts)then use eclipse to connect to the tasktracker , note you need to choosebetween "Listen/attach" in the"Run remote application" window in eclipse

On Wed, Jul 4, 2012 at 7:00 PM, Jason Yang <[EMAIL PROTECTED]> wrote:

> ramon,>> Thank for your reply very much.>> However, I was still wonder whether I could debug a MR application in this> way.>> I have read some posts talking about using NAT to redirect all the packets> to the network card which connect to the local LAN, but it does not work as> I tried to redirect by using iptables :(>> 在 2012年7月4日星期三��� 写道：>>> Jason,>>>>>> the easiest way to debug a MapRedupe program with eclipse is working>> on hadoop local.>> http://hadoop.apache.org/common/docs/r0.20.2/quickstart.html#Local In>> this mode all the components run locally on the same VM and can be easily>> debugged using Eclipse. Hope this will be useful.>>>>>>>> *From:* Jason Yang [mailto:[EMAIL PROTECTED]]>> *Sent:* miércoles, 04 de julio de 2012 11:25>> *To:* mapreduce-user>> *Subject:* How to connect to a cluster by using eclipse>>>>>>>> Hi, all>>>>>>>> I have a hadoop cluster with 3 nodes, the network topology is like this:>>>> 1. For each DataNode, its IP address is like :192.168.0.XXX;>>>> 2. For the NameNode, it has two network cards: one is connect with the>> DataNodes as a local LAN with IP address 192.168.0.110, while the other one>> is connect to the company network(which eventually connect to the Internet);>>>> -->>>> now I'm trying to debug a MapReduce program on a computer which is in the>> company network. Since the jobtracker in this scenario is>> 192.168.0.110:9001, I was wondering how could I connect to the cluster>> by using eclipse?>>>>>>>> -->>>> YANG, Lin>>>>>>>> ------------------------------>> Subject to local law, communications with Accenture and its affiliates>> including telephone calls and emails (including content), may be monitored>> by our systems for the purposes of security and the assessment of internal>> compliance with Accenture policy.>>>> ______________________________________________________________________________________>>>> www.accenture.com>>>>> --> YANG, Lin>>

NEW: Monitor These Apps!

All projects made searchable here are trademarks of the Apache Software Foundation.
Service operated by Sematext