Well-known member

XenForo developer

This is made much more common by the custom thread field search on your side. That appears to be implemented in such a way that it has to be done in SQL, so we "overfetch" to account for this (later filtering down). Similarly, when grouping by thread, we overfetch and filter down. Each multiplies the number of documents fetched by 4, but in combination, that ends up meaning by 16. Out of the box this can happen, but I don't think it would be anywhere near 20 times per day, even on a big board.

Regardless, I have set up a cap at fetching 10000 results. This is the out of the box limit for ES now, and fetching more than that to filter is probably unnecessary.

You can workaround this by adding:

Code:

$dsl['size'] = min(10000, $dsl['size']);

Before:

Code:

$response = XenES_Api::search($this->_indexName, $dsl);

In library/XenES/Search/SourceHanderl/ElasticSearch.php.

(There are two instances of the original line. Technically it only applies to the first one, though it shouldn't hurt to apply it to the second.)

Well-known member

This is made much more common by the custom thread field search on your side. That appears to be implemented in such a way that it has to be done in SQL, so we "overfetch" to account for this (later filtering down). Similarly, when grouping by thread, we overfetch and filter down. Each multiplies the number of documents fetched by 4, but in combination, that ends up meaning by 16. Out of the box this can happen, but I don't think it would be anywhere near 20 times per day, even on a big board.

Regardless, I have set up a cap at fetching 10000 results. This is the out of the box limit for ES now, and fetching more than that to filter is probably unnecessary.

You can workaround this by adding:

Code:

$dsl['size'] = min(10000, $dsl['size']);

Before:

Code:

$response = XenES_Api::search($this->_indexName, $dsl);

In library/XenES/Search/SourceHanderl/ElasticSearch.php.

(There are two instances of the original line. Technically it only applies to the first one, though it shouldn't hurt to apply it to the second.)