HBase import job does not complete

Issue -The hbase import job fails in the end of import. If the splits were say 4000 and the mappers created for import were 4000, the last set of tasks which have to run will get struck in NEW State and never completes. The number of tasks which get struck varies between 1 to 8 in the 4-5 times we have tried. There are no errors/killed tasks for the compelted ones. There are no errors in the container log or task logs which say something obvious was a problem. The region server error also does not show any errors at the time this happens. Once this happens the import job never completes and gets struck in the same state and the only option would be to kill the job using the yarn application kill command.

We initially thought that the problem with the files which were exported due to which the import was failing, upon importing the files into the table individually (1 by one using the import command) the import goes through.

Export input details1. Folder size of the exported data - 554G2. Total number of sequence files in the exported folder - 7413. Size of these files - varies between 290MB to 1.1 GB

How was the exported files transferred from the source to the target cluster1. Once the table data was exported on the source cluster, it was moved to local file system on the source cluster using copyToLocal command of hadoop.2. The files were transferred to the target cluster using rsync.3. The files were moved to hdfs on the target cluster using the copyFromLocal command of hadoop.