Raja,
I don't have time to test this right now, so this might require a little debugging, but here are the basic changes that you need to make in the source code to limit the number of records from the Report Table.

Make backup copies of the files workflow/engine/src/ProcessMaker/Services/Api/Project/ReportTable.php and workflow/engine/src/ProcessMaker/BusinessModel/Table.php before you make any changes to them.

Edit workflow/engine/src/ProcessMaker/Services/Api/Project/ReportTable.php and change the definition of the doGetReportTableData() function starting on line 77 to:

Then you should be able to get rows from the Report Table by calling the endpoint multiple times like this:
GET project/1234567890abcde1234567890abcde/report-table/abcde1234567890abcde1234567890/data?limit=1000
GET project/1234567890abcde1234567890abcde/report-table/abcde1234567890abcde1234567890/data?start=1001&limit=1000
GET project/1234567890abcde1234567890abcde/report-table/abcde1234567890abcde1234567890/data?start=2001&limit=1000
GET project/1234567890abcde1234567890abcde/report-table/abcde1234567890abcde1234567890/data?start=3001&limit=1000

If this doesn't work, then post here what errors you are seeing and I will debug it when I find extra some time.