-
Task
-
Resolution: Duplicate
-
L3 - Default
-
None
-
None
-
2
-
S
Context:
In the context of OPT-3748, we decided that the best option to resolve the issues caused by the nested documents limitation is to split our data into several indices instead of using nested documents. For example, we could have one index for process instances and one index for related activities, with a reference in the activities index to which process instance each activity belongs to. However, before implementing such a big change to our index structure, we should dedicate some time into evaluating the impact and effort of this change first. This could for example be done by trying to extract the variables into their own dedicated index first.
Meeting notes: https://confluence.camunda.com/display/CO/2020-10-22+Handle+10k+nested+document+limit
Note: This should first be scope-limited to variables, given that we now flatten Object variables by default and potentially create a lot more nested documents than before
AT:
- Get a better idea of the impact of this change, for example with regards to performance impact and other required changes
- Evaluate which nested documents are affected and should be extracted like this
- Get a better idea of effort involved to implement this for all nested documents
Justification:
This would likely improve index read/write performance. We also would be able to remove the limitation on nested documents where data can be skipped, resulting in partial data sets for customers