-
Task
-
Resolution: Done
-
L3 - Default
-
None
-
None
Context:
During the work on OPT-3524 we encountered a weird exception on the elastic side related to the the startup of Optimize and the initial load of alarms:
05:12:17 Suppressed: org.elasticsearch.client.ResponseException: method [POST], host [http://localhost:9200], URI [/optimize-process-instance/_search?typed_keys=true&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&search_type=query_then_fetch&batched_reduce_size=512&ccs_minimize_roundtrips=true], status line [HTTP/1.1 429 Too Many Requests] 05:12:17 {"error":{"root_cause":[{"type":"circuit_breaking_exception","reason":"[parent] Data too large, data for [<http_request>] would be [8094251846/7.5gb], which is larger than the limit of [8094194073/7.5gb], real usage: [8094247352/7.5gb], new bytes reserved: [4494/4.3kb]","bytes_wanted":8094251846,"bytes_limit":8094194073,"durability":"PERMANENT"}],"type":"circuit_breaking_exception","reason":"[parent] Data too large, data for [<http_request>] would be [8094251846/7.5gb], which is larger than the limit of [8094194073/7.5gb], real usage: [8094247352/7.5gb], new bytes reserved: [4494/4.3kb]","bytes_wanted":8094251846,"bytes_limit":8094194073,"durability":"PERMANENT"},"status":429} 05:12:17 at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:260) 05:12:17 at org.elasticsearch.client.RestClient.performRequest(RestClient.java:238) 05:12:17 at org.elasticsearch.client.RestClient.performRequest(RestClient.java:212) 05:12:17 at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1433) 05:12:17 ... 79 common frames omitted
no matter how big the heap was sized it always hit the limit and given there are even no alarms stored this appeared to be a bug of the memory circuit breaker.
Testing revealed this affects Elasticsearch 7.0.0-7.2.1, but from 7.3.0 it never occured.
AT:
- track down cause of the issue
- reset default es version used in the query performance test
This is the controller panel for Smart Panels app
- is related to
-
OPT-3472 Evaluate all elasticsearch aggregations prone to hitting bucket size limit
- Done