Description

If the sections are large and has many links, templates, cxserver will take a good amount of time to finish the processing.
For example, big tables in en:List_of_English_football_champions, like one in section "Football League First Division (1892–1992)".

The request will timeout in that cases. In the case of the above table, I observed that the request did not finish in 2minutes and timed out.

This ticket is to analyse and detect bottlenecks if any in the processing that we can improve and reduce the time.

I did a timing analysis for the big table in en:List_of_English_football_champions, the one un section "Football League First Division (1892–1992)". Target language: es. MT: Apertium

I changed the TitleInfoRequest and TitlePairRequest to Get methods instead of Post:

Once the cache for cxserver is ready, a re-attempt to translation, using "Reset translation" gives

If I use the POST method in TitleInfoRequest and TitlePairRequest (no modification)

Observations

The table is now translatable

Using get request has slight improvement in timing compared to POST. VE API Request cache uses get. We changed the cxserver implmentation to use POST, but I don't think that is relevant now, since we never send more than 50 titles in go.

Is the table translatable with cold caches? It doesn't really mean anything if it is translatable with warm caches, as users will see a failure (and probably won't retry)

Surprising that there is a diff with GET/POST. Is this consistent pattern? I'm okay with switching to GET as long as we don't get silent (or non-silent) failures due to it. Though, it was not about number of titles, but maximum query length. 50*300 bytes is too much (namespace + 255 bytes).

The first, and the last data in the above comment are immediately after cxserver start, means cache was empty. And it was translatable after the api cache fix patches. In master, the table is not translatable at all. I waited for 4 mins and then the request timed out.

There is nothing to translate for an MT client in this. All elements are template and its fragements. Adaptation on data-mw definition will be happening anyway. It is a waste to translate the table content.

If we can figure out the above fact in MTClient class translateHtml method, we can save serveral MT service requests.