Uploaded image for project: 'Camunda Optimize'
  1. Camunda Optimize
  2. OPT-3378

Reevaluate default shardCount for indices

    XMLWordPrintable

Details

    • Task
    • Resolution: Done
    • L3 - Default
    • 3.0.0
    • 3.0.0
    • backend
    • None

    Description

      Context:
      With 7.x.x Elastic decreased the default shard count per index from 5 to 1. As well as introduced a limit of a maximum of 1000 open shards per node.
      We reconsider the default shardCount in the Optimize configuration (currently being 5) as well as whether there are certain indexes that may need different defaults than others.

      E.g. with the introduction of camunda events OPT-2955 we introduced per definition indexes for which a lower shardcount compared to the overall processInstance index count would be feasible. As for those indexes query performance is not that crucial + they are read sequentially anyway so search queries are not really a use case there. This also applies to the event index.

      AT:

      • default shard count config is reevaluated
      • documentation is updated to better explain the purpose of configurable shard size
      • different use case indexes are identified and if considered reasonable a different default shardCount is introduced to be used, potential different use cases are e.g.:
        • the event index, camunda activity event indices which are written and read sequentially
        • event trace & Count indexes are not that crucial to yield the best query performance so a shard size of 1 could be fine for them
        • every index except the decision/process instance indices usually does not contain much data, the shardSize could easily get reduced to 1 for others

      mgm-controller-panel

        This is the controller panel for Smart Panels app

        Attachments

          Activity

            People

              Unassigned Unassigned
              sebastian.bathke Sebastian Bathke
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Salesforce