L3 - Default Brief summary of the bug. What is it ? Where is it ?
Running a k8s init container in order to perform an Optimize update may lead into an edge-case where the init container is started but Elasticsearch is not yet reachable. This is more likely for newly created clusters.
The Update then fails because of a Connection Refused error but the process does not exit so that k8s can retry by creating a new init container.
Steps to reproduce:
1. Start the update without having Elasticsearch running
Actual result :
2. The Update fails with a connection refused error, the process however does not exit.
17:53:05.640 [main] DEBUG o.c.o.u.e.ElasticsearchHighLevelRestClientBuilder - Elasticsearch username and password not provided, skipping connection credential setup. Exception in thread "main" java.lang.ExceptionInInitializerError Caused by: org.camunda.optimize.service.exceptions.OptimizeRuntimeException: Failed retrieving aliases for dynamic index prefix camunda-activity-event- at org.camunda.optimize.service.es.schema.MappingMetadataUtil.retrieveAllDynamicIndexKeysForPrefix(MappingMetadataUtil.java:181) at org.camunda.optimize.service.es.schema.MappingMetadataUtil.retrieveAllCamundaActivityEventIndices(MappingMetadataUtil.java:145) at org.camunda.optimize.service.es.schema.MappingMetadataUtil.getAllDynamicMappings(MappingMetadataUtil.java:73) at org.camunda.optimize.service.es.schema.MappingMetadataUtil.getAllMappings(MappingMetadataUtil.java:67) at org.camunda.optimize.upgrade.es.SchemaUpgradeClientFactory.createSchemaUpgradeClient(SchemaUpgradeClientFactory.java:37) at org.camunda.optimize.upgrade.es.SchemaUpgradeClientFactory.createSchemaUpgradeClient(SchemaUpgradeClientFactory.java:24) at org.camunda.optimize.upgrade.main.UpgradeProcedureFactory.create(UpgradeProcedureFactory.java:22) at org.camunda.optimize.upgrade.main.UpgradeMain.<clinit>(UpgradeMain.java:42) Caused by: java.net.ConnectException: Connection refused
Expected result:
If elasticsearch cannot be reached the update should either retry or fail and exit the process with an exit code != 0.