LINTE
added a comment - 22/Jun/15 13:13 Hi,
There is no Java stackstrace for this bug.
I think that the property yarn.resourcemanager.max-completed-applications is in cause (default value is 10000), but it doesn't work properly.
Maybe yarn.resourcemanager.max-completed-applications is only effective on ResourceManager GUI.
Regards,

LINTE, This seems to be sorting issue with respect to the app ids. It is just considering the first four digits of the application number for sorting in ascending/descing order, due to this it is not showing the application ids having more that 9999 based on the order and it is mixing up with the other apps considering only the first 4 digits. You can see the attached image RMApps.png which shows displaying the apps having id>9999 with other apps,

Please check and confirm whether is it happening same or not in your case by searching for the specific app id in the search box. Thanks.

Devaraj K
added a comment - 23/Jun/15 11:55 LINTE , This seems to be sorting issue with respect to the app ids. It is just considering the first four digits of the application number for sorting in ascending/descing order, due to this it is not showing the application ids having more that 9999 based on the order and it is mixing up with the other apps considering only the first 4 digits. You can see the attached image RMApps.png which shows displaying the apps having id>9999 with other apps,
Please check and confirm whether is it happening same or not in your case by searching for the specific app id in the search box. Thanks.

The datatable string sort algorithm has limitation.
It can not properly sort the string having having the combination of the string and numeric value. The application id "application_numericValue"
ie is why the sort is not working properly.

To fix the same we can use the datatable plugins natural sort alogorithm.

Mohammad Shahid Khan
added a comment - 27/Jun/15 06:31 The datatable string sort algorithm has limitation.
It can not properly sort the string having having the combination of the string and numeric value. The application id "application_numericValue"
ie is why the sort is not working properly.
To fix the same we can use the datatable plugins natural sort alogorithm.
sb.append( "[\n" )
.append( "{'sType':'natural', 'aTargets': [0]" )
.append( ", 'mRender': parseHadoopID }" )
plugin - ref:
https://github.com/DataTables/Plugins/blob/1.10.7/sorting/natural.js

Devaraj K
added a comment - 01/Jul/15 06:05 Thanks Mohammad Shahid Khan for the patch.
Patch is not getting applied, can you please check it?
There are some comments at high level.
1. Can you also check the same for Node Manager Web UI Running Applications and Running Containers tables?
2. Newly added js code is missing the format, Please follow this coding guide line.
Indent two spaces per level, not four.

Please ignore the first patch. in the current patch have taken the nodemanager web ui issue as well.
The current patch is using the natural sort algorithm of the natural,js a plugin used for datatable to sort the data.
The natural sort plugin " https://github.com/DataTables/Plugins/blob/1.10.7/sorting/natural.js " is having the MIT license.
As per MIT license we can redistribute the code but we have to keep the license header.
The haddop patch verification tool does not allow the auther info
As of now i have not removed the @author tag from the path file.
Please help me to address this issue.

Mohammad Shahid Khan
added a comment - 01/Jul/15 10:09 Hi Devaraj K ,
Please ignore the first patch. in the current patch have taken the nodemanager web ui issue as well.
The current patch is using the natural sort algorithm of the natural,js a plugin used for datatable to sort the data.
The natural sort plugin " https://github.com/DataTables/Plugins/blob/1.10.7/sorting/natural.js " is having the MIT license.
As per MIT license we can redistribute the code but we have to keep the license header.
The haddop patch verification tool does not allow the auther info
As of now i have not removed the @author tag from the path file.
Please help me to address this issue.

You can probably package the plugin js file which is compatible for the dt-1.9.4 release(currently YARN uses) similar to the one we have already in hadoop-yarn-project\hadoop-yarn\hadoop-yarn-common\src\main\resources\webapps\static\dt-1.9.4\js\jquery.dataTables.min.js.gz.

Devaraj K
added a comment - 01/Jul/15 11:12 You can probably package the plugin js file which is compatible for the dt-1.9.4 release(currently YARN uses) similar to the one we have already in hadoop-yarn-project\hadoop-yarn\hadoop-yarn-common\src\main\resources\webapps\static\dt-1.9.4\js\jquery.dataTables.min.js.gz.

Did anyone do a performance test with this change? Recently I'm seeing the main RM UI page taking a lot longer to load with a full set of 10,000+ applications, and it appears to all be client-side processing. It takes so long the client-side browser is complaining:

A script on this page may be busy, or it may have stopped responding. You can stop the script now, open the script in the debugger, or let the script continue.
Script: http://rmhost:8088/static/dt-plugin-1.10.7/sorting/natural.js

Jason Lowe
added a comment - 16/Nov/15 18:34 Did anyone do a performance test with this change? Recently I'm seeing the main RM UI page taking a lot longer to load with a full set of 10,000+ applications, and it appears to all be client-side processing. It takes so long the client-side browser is complaining:
A script on this page may be busy, or it may have stopped responding. You can stop the script now, open the script in the debugger, or let the script continue.
Script: http://rmhost:8088/static/dt-plugin-1.10.7/sorting/natural.js

Varun Saxena
added a comment - 18/Nov/15 06:27 Jian He , Shahid is in my team and currently on leave.
Let me see if I can find a solution to this before he comes back.
natural.js has a lot of extraneous logic which is not directly related to what we want to do for sorting appIds'. Let me see if removing code improves performance.

Varun Saxena
added a comment - 20/Nov/15 21:35 Rewrote the logic for sorting.
In my tests, with 10000 finished apps and RM idle(no running apps), it takes more than 1 minute to load the apps page with previous script.
With the script in patch, it takes a few seconds.

I have only changed the javascript file(natural.js). Rewrote the logic for natural sort from scratch keeping in mind the way our application ids', attempt ids', task ids', etc. are.
Other code is exactly the same as what Shahid had written.

Varun Saxena
added a comment - 21/Nov/15 18:13 Jian He / Jason Lowe , kindly review.
Test failures unrelated.
Raised YARN-4379 and MAPREDUCE-6555 for 2 of the failures. For others, JIRA exist as well.
Findbugs warnings are unrelated as well. Had raised YARN-4298 for it earlier.
I have only changed the javascript file(natural.js). Rewrote the logic for natural sort from scratch keeping in mind the way our application ids', attempt ids', task ids', etc. are.
Other code is exactly the same as what Shahid had written.