Details
-
Bug Report
-
Resolution: Fixed
-
L3 - Default
-
None
-
None
Description
There are ES import jobs of two kinds: data import jobs, which insert the data to ES and index update jobs, which are executed to save the current state of import.
Given:
- the data import job fails (e.g. connection error)
When:
- the corresponding index update job is executed successfully
Then:
- data from the data import job is lost. The data won't be reimported, as the import index contains an entry that states that the data is imported.
AT:
- The data doesn't get lost in case of scenario above.
Notes:
- We need to retry the failing import job infinitely, which would cause the import to stop until the possible ES issue resolves
- In case multiple importing threads are enabled, we need to think how to avoid the race condition between the data import job and the corresponding index update, maybe merging the two kinds of import jobs into one.